Creating Your Own Dashboard

The queries you have created so far can be transferred to dashboards, which then have less of an ad hoc character and present a large amount of visual information at a glance. You can create a new workbook by clicking on Workbooks in the Monitoring section of the AAD portal. You will then see an overview of available workbooks. Microsoft has started to publish predefined reports as part of the engine changeover. Of course, you can leverage these as a source of inspiration. The Workbooks process acquires data from Log Analytics and your own requests. Microsoft provides some logs on sign-ins, such as which sign-ins occurred in your tenant, whether legacy authentication was used, and how conditional access influenced this.

You can create a new, custom dashboard with + New or under Quick start . An empty dashboard will be loaded showing only the commands for new elements: Add text , Add query , and some others. Start with Add text , insert the text SignIns with Admin Account , and confirm with Done Editing . Then, create the dashboard by entering a title, the Azure Subscription, and the Resource Group where the Log Analytics workbook should end up. Click Save , and you can continue with the editing. Now insert the following query under your text:

SigninLogs
| where UserPrincipalName == "admin@testtenant.onmicrosoft.com"
| where TimeGenerated > ago(7d)
| summarize count() by TimeGenerated
| render timechart

Run Query gives you a rendered time history showing the logons of the last seven days for the specified User Principal Name (UPN). Size lets you change the size of the graph, and Done Editing terminates draft mode and displays the curve below the text label. In this way, various elements are added to the dashboard. You can arrange the elements with the Up and Down arrow keys in edit mode.

Additionally, a settings menu contains the means, for example, to change the width of the elements on the dashboard. To do this, click on the gear icon at the bottom of the item and select the desired size in Make this item a custom width . You can add a short description for the element's heading in Chart Title . If no data is available, a message in No data message will help. If no chart is rendered, but a list is generated from the data, you can adjust the output columns to display only the relevant data. Once you have formulated the query, Column Settings lets you enable the relevant columns.

Queries previously created on the fly can be transferred to the dashboard without changes in most cases. The data presentation can be adjusted to make the most important findings visible at a glance. To make the previous query, which shows changes to a specific group, far more attractive, add two more lines and display a pie chart instead of a flat list (Figure 3):

AuditLogs
| where TargetResources contains "a0fdc91a-a1b2-4ec5-b352-03bda610be0e"
| where TimeGenerated > ago(7d)
| summarize a = count() by ActivityDisplayName
| render piechart

Figure 3: Configuring dashboards to suit individual information requirements.

A trick lets you use the dashboard or individual requests from the dashboard on the AAD portal splash page to store the most important information there. Click on the pin and choose between Pin Workbook and Pin All . The first option creates a link to the dashboard on the splash page, whereas Pin All also displays the individual elements of the dashboard directly on the splash page. You then need to adjust the size and arrangement of the individual elements.

Azure Sentinel as SIEM

Microsoft recently announced a SIEM-in-the-cloud solution named Azure Sentinel that can accept and analyze logs and data from any source. The tool is designed to help IT managers gain insights into previously acquired data without needing to build a complex infrastructure and purchase additional software. The preview version of Sentinel can also provide insights into AAD logs. However, here too, the data must be available in a Log Analytics workspace.

Sentinel already has two predefined AAD dashboards that are designed to supplement your own overviews by helping you search for irregularities or operations in general. When you create an Azure Sentinel workspace, you will find an overview of the installed dashboards under Dashboards . To connect AAD, you first need to specify it as a data source in Data Connectors and then select Azure Active Directory and click on Connect . Back in Dashboards , selecting All reveals two new reports: Azure AD Audit Logs and Azure AD Sign-in Logs. Click View Dashboard to install the dashboards, after which you can display them immediately – provided you have already exported data to AAD.

Conclusions

Whether or not you use a SIEM tool in your company, the Kusto query language that you use for the exported AAD logs can help the directory team, because not all relevant data is always available in SIEM.

Often, "only" the security-relevant data is of interest to SIEM and is then served up to the identity team; however: What about group changes to assignments? How successful is identity provisioning from AAD to a software-as-a-service application? What methods of multifactor authentication identity verification are most commonly used with Exchange Online? These questions can be answered quite easily with data from the logs and presented neatly in a dashboard, which means that the SIEM team does not have to respond constantly to questions about access and changes to the tool.