Check Mailbox Audit Configurations to Make Sure that New Audit Events are Ingested into Audit Log

Last October, I wrote about Microsoft’s glacial progress in making important audit events used for forensic investigations available to customers with Purview Audit standard licenses. This followed a July 19 statement where Microsoft agreed to expose the audit events to audit log searches run by Purview Audit standard customers and to extend the retention period for audit events from 90 to 180 days. Nothing seems to move quickly in the world of auditing. Perhaps they need a Copilot to help?

The good news is that a May 20 post in the Microsoft technical community post says that the long-anticipated delivery of 19 new audit events are coming in public preview. Once the update reaches your tenant (looks like June 2024 according to the Microsoft 365 roadmap), you should see these events turn up for accounts with Purview Audit standard licenses in the results of audit log searches run through the Purview portal, the Search-UnifiedAuditLog cmdlet, or the AuditLogQuery Graph API.

Searching for the New Audit Events

Here’s an example of using the Search-UnifiedAuditLog cmdlet to search the audit log for some of the new events. Note that I use the SessionCommand parameter to make sure that all results are returned (necessary after an unannounced and unexplained change made by Microsoft last year). Sorting the results by identity removes duplicates:

[array]$Records = Search-UnifiedAuditLog -Operations MailItemsAccessed, Send, messageSent -StartDate (Get-Date).AddDays(-10) -EndDate (Get-Date).AddDays(-1) -ResultSize 5000 -Formatted -SessionCommand ReturnLargeSet
$Records = $Records | Sort-Object Identity -Unique

$Records | Group Operations -Noelement | select name, count

Name              Count
----              -----
MailItemsAccessed  1792
MessageSent          61
Send                 49

You could get the same results by running a high completeness search, but you’d wait much longer for the output (if the search doesn’t hit an internal server error as in Figure 1). In Microsoft’s defense, high completeness searches are a preview feature.

The Question of Exchange Mailbox Logging

What’s interesting from Microsoft’s announcement is that the Send and MailItemsAccessed events are added automatically to the set of events captured for mailboxes UNLESS you’ve updated the audit configuration for a mailbox. In other words, Microsoft doesn’t attempt to update custom mailbox audit configurations.

I guess I understand the logic. If administrators changed mailbox audit configurations, they presumably do so for good reason and Microsoft doesn’t want to mess with that configuration. On the other hand, an arguable case exists that these events are so important that they should be added to the audit configuration for all mailboxes.

Updating the Mailbox Audit Configuration for New Audit Events

Microsoft suggests two options: revert mailboxes to the default audit configuration or update mailbox audit configurations to add the new events. I suggest that the latter is the better option. Here’s some code I used to update mailboxes in my tenant. The script uses the Get-MgUser cmdlet from the Microsoft Graph PowerShell SDK to find accounts with Office 365 E3 licenses (including Purview Audit standard).

For each mailbox, the script:

• Checks to see if the default audit set for owner actions is present. If it is, we don’t need to update the audit configuration because Microsoft will add the new events to the default set.
• Checks the audit configuration for owner actions to see if the set includes MailItemsAccessed. If not, update the configuration for the owner and delegate sets.
• Checks the audit configuration for owner actions to see if the set includes the Send action. If not, update the owner set.
• Runs Set-Mailbox to enable the updated audit configuration. I have no idea why Microsoft insists that this needs to be done manually for Purview Audit standard. It isn’t required for mailboxes with Purview Audit (Premium) licenses.

Connect-MgGraph -NoWelcome -Scopes User.Read.All
Connect-ExchangeOnline
[array]$Users = Get-MgUser -filter "assignedLicenses/any(s:s/skuId eq 6fd2c87f-b296-42f0-b197-1e91e994b900)" -All | Sort-Object DisplayName
[int]$Updates = 0
ForEach ($User in $Users) {
    # See if the mailbox uses the default audit set
    Write-Host ("Checking mailbox audit configuration for {0}" -f $User.displayName)
    [array]$DefaultAuditSet = (Get-Mailbox -Identity $User.UserPrincipalName).DefaultAuditSet
    If ("Owner" -notin $DefaultAuditSet) {
        # There's a non-default owner audit configuration, so let's update the custom set
        [array]$AuditConfiguration = (Get-Mailbox -Identity $User.userPrincipalName).AuditOwner
        If ("MailItemsAccessed" -notIn $AuditConfiguration) {
            Write-Host ("Updating mailbox audit configuration for {0}" -f $User.displayName) -ForegroundColor Yellow
            Set-Mailbox -Identity $User.UserPrincipalName -AuditOwner @{Add="MailItemsAccessed"} -AuditDelegate @{Add="MailItemsAccessed"} -ErrorAction SilentlyContinue
            $Updates++
        }
        If ("Send" -notIn $AuditConfiguration) {
            Set-Mailbox -Identity $User.UserPrincipalName -AuditOwner @{Add="Send"} -ErrorAction SilentlyContinue
        }
        # Make sure that the new audit configuration is enabled
        Set-Mailbox -Identity $User.UserPrincipalName -AuditEnabled $true -WarningAction SilentlyContinue
    }
}
Write-Host ("All done. {0} of {1} mailboxes updated" -f $Updates, $Users.Count)

New Audit Events are A Step Forward

It’s good that Microsoft has finally deployed the new audit events. It’s not so good that tenant administrators need to intervene to ensure that mailbox audit configurations are correctly set up. Further details are available in Microsoft’s documentation.

Learn about using Exchange Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Time to Review Mailbox Auditing Configurations

Updated 8 November 2023

Paul Robichaux’s recent article describing five errors Microsoft made which led to the Storm-0558 attack made me think about the MailItemsAccessed event. This was the first “premium” or high-value audit event launched by Microsoft in an attempt to monetize auditing through the introduction of what is now Microsoft Purview Audit (Premium) (aka Microsoft 365 advanced auditing). Purview Audit Premium is included in Office 365 E5 and Microsoft 365 E5 and other add-on licenses. Purview Audit Standard is available to Office 365 E3 and Microsoft 365 E3 customers.

Update: Microsoft says that Office 365 E3 customers won’t see the MailItemsAccessed event until the summer of 2024.

In his article, Paul points out that tenant administrators for a federal executive civilian branch agency noted unusual activity captured in MailItemsAccessed events. Exchange Online captures these events (Figure 1) when mailboxes belonging to licensed accounts access mail messages. Being able to know that someone (or some process) other than the owner accessed messages in a mailbox is a good indication that something’s wrong.

To emphasize the point about how important MailItemsAccessed events can be, Microsoft’s documentation explains how to use the events in a forensic investigation. This is what might have happened to detect some of the Storm-0588 infiltration. According to a Cybersecurity and Infrastructure Security Agency (CISA) report analyzing Storm-0558, “The affected FCEB agency identified suspicious activity by leveraging enhanced logging—specifically of MailItemsAccessed events—and an established baseline of normal Outlook activity (e.g., expected AppID). The MailItemsAccessed event enables detection of otherwise difficult to detect adversarial activity.”

The Cost of Security

As Paul notes, some organizations don’t use MailItemsAccessed because they didn’t want to pay for enhanced auditing. Although avoiding cost is a reasonable perspective, it does raise the issue of why Microsoft insists that customers pay extra to log events that are so important for investigation of potential incidents. Some feel it’s an example of extracting additional revenue from a captive market. After all, the 400 million Office 365 monthly active users don’t exactly have a choice of auditing provider.

On July 19, Microsoft decided that it was best to reverse course and announced that they would make enhanced logging available to Office 365 E3/Microsoft 365 E3 tenants, saying “customers will receive deeper visibility into security data, including detailed logs of email access and more than 30 other types of log data previously only available at the Microsoft Purview Audit (Premium) subscription level. In addition to new logging events becoming available, Microsoft is also increasing the default retention period for Audit Standard customers from 90 days to 180 days.”

Audit Updates Coming in September 2023

According to Microsoft, they will deploy the necessary updates to expose the additional audit events and to increase audit event retention to 180 days to all commercial and government customers during September 2023. The update hasn’t reached my tenant yet because any attempt to enable the MailItemsAccessed event for a mailbox with an Office 365 E3 license fails as follows:

Set-Mailbox -Identity Lotte.Vetler -AuditOwner @{Add="MailItemsAccessed"}

Set-Mailbox: |Microsoft.Exchange.Management.Tasks.RecipientTaskException|Auditing of MailItemsAccessed event is only available for users with appropriate license. Please visit the documentation to know more about this.

When the update lands, Microsoft hasn’t said if they will retrospectively enable the MailItemsAccessed event for mailboxes with Office 365 E3 or Microsoft 365 E3 licenses. It’s entirely possible that Microsoft will not update mailbox audit configurations to add the MailItemsAccessed event for existing mailboxes. We also don’t know if Microsoft will enable new mailboxes for the event in the same way that they enable the event automatically for mailboxes licensed for Purview Audit Premium. A arguable case exists that managing mailbox audit configurations is an operation best left to tenants, especially if tenants use non-standard mailbox auditing configurations.

My advice is to take control of the situation and:

• Check that mailbox auditing is enabled for all mailboxes. This note in Microsoft documentation implies that mailboxes with Purview Audit Standard still need to enable auditing to force flow of mailbox audit events from Exchange Online to the unified audit log. This was certainly the case, but a quick test with a new mailbox created today saw mailbox events appear in the unified audit log. In any case, it’s best to be sure.
• Include the MailItemsAccessed event in the audit configuration for all mailboxes. Some years ago, I wrote a script to make sure that auditing was enabled for all mailboxes. It is easy to adapt the script to update mailbox audit configuration with the MailItemsAccessed event.
• Consider a more automated approach to maintain mailbox audit configurations. Using a scheduled PowerShell runbook managed by Azure Automation is a mechanism well suited to this kind of task. If the runbook operated on a weekly basis, the user accounts created during the last week can be found with code like this:

$LastWeek = (Get-Date).AddDays(-7)
$T = Get-Date $LastWeek -format "yyyy-MM-ddThh:mm:ssZ"
[array]$Users = Get-MgUser -Filter "createdDateTime ge $T" -Property Id, UserPrincipalName, CreatedDateTime, DisplayName

The MailItemsAccessed Event Really is High-Value

No one likes being caught on the back foot when things go wrong. But if problems occur, it’s good to have as much data as possible. The MailItemsAccessed event increases the amount of information available about what attackers might have done inside Exchange Online mailboxes. That’s one good reason to make sure to capture the events and know how to use them during forensic investigations.

Create a task for yourself to check mailbox audit configurations at the end of September 2023 and make sure that the MailItemsAccessed event is captured. You know it makes sense.

Learn about using Exchange Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Capturing Crucial Office 365 Audit Data Requires E5 Licenses

In January 2019, Microsoft announced that they were adding an event called MailItemsAccessed to the set of audited operations captured in the Office 365 audit log. Microsoft claimed that the new event would “capture details of when a message in a mailbox is opened by the mailbox owner, delegate (someone with read access to the mailbox) or using administrative access” leading to audit information delivering “comprehensive forensic coverage of mailbox accesses.”

Time moved on and in March 2019, Microsoft said that they had halted the deployment of MailItemsAccessed to Office 365 tenants. Software has a habit of hitting delays and it was speculated that the overhead involved in gathering a massive number of message access events would place a strain on Exchange Online.

All went quiet for a while, which prompted me to ask Microsoft in June what was happening. They provided an odd statement that faintly indicated that the MailItemsAccessed event might appear in Q3 (July to September).

Crucial Security or Compliance Audit Events

Q3 came and went without a trace of any message access being captured in the Office 365 audit log. But last month Microsoft released documentation for Advanced Audit in Microsoft 365 (now Purview Audit Premium) which makes it clear that MailItemsAccessed is now regarded as the first example of a “crucial” security or compliance-related audit event included in their advanced audit offering. Previously, Microsoft called these events “high-value.” In either case, Microsoft defines the event as “one that can help you investigate possible breaches or other forensic-related investigations.”

Update October 19: Microsoft has released three additional crucial events to handle email sends and searches of mailboxes and sites.

In a nutshell, if you want to see information about who accessed an item in a mailbox, you need to buy some Office 365 E5, Microsoft 365 E5 or Microsoft 365 E3 with Compliance licenses.

Some MailItemsAccessed records can be found in the Office 365 audit log for my tenant audit and viewed using the Search-UnifiedAuditLog cmdlet or the Audit log search (Figure 1). But all the records that have turned up so far (in about a month) are for “sync” activities for various folders like the Inbox. Sync records aren’t very exciting because all they record is the synchronization of a complete folder using a client like Outlook desktop. The really interesting data lie in bind records, which record access to individual messages.

It’s also interesting to learn that Exchange Online applies throttling for MailItemsAccessed events. If a mailbox generates more than 1,000 bind events in a 24-hour period, Exchange Online stops recording MailItemsAccessed events for bind operations for another 24 hours before resuming capture of these events. Microsoft says that less than 1% of mailboxes are subject to throttling.

You can download an example of how to extract and report MailItemsAccessed audit events from GitHub.

Audit Log Retention Policies

Apart from capturing crucial audit events, the advanced audit feature also allows tenants to configure audit log retention policies. These policies work much like mailbox retention policies. You define a retention policy for selected audit events with a set retention period and Office 365 removes those items after that period. A tenant supports up to 50 audit log retention policies.

This example runs the New-UnifiedAuditLogRetentionPolicy cmdlet to create an audit retention policy to remove any SearchQueryPerformed event executed by the background app@sharepoint process after three months instead of the twelve-month retention of audit events if the tenant has E5 licenses.

New-UnifiedAuditLogRetentionPolicy -Name "90-day Retention SearchQueryPerformed by app@sharepoint" -Description "Remove SearchQueryPerformed events from the app@sharepoint process after 90 days" -RecordTypes SharePoint -Operations SearchQueryPerformed -UserIds "app@sharepoint" -RetentionDuration ThreeMonths -Priority 8

You can only manage audit log retention policies with PowerShell using cmdlets accessible by connecting to the Compliance Center endpoint.

Purging the Office 365 Audit Log

You can choose to apply retention for any of the events captured in the Office 365 audit log and keep them for three, six, nine, or twelve months. That is, you can keep audit events for longer than 90 days for accounts with E5 licenses. Office 365 restricts E3 accounts to a 90-day retention period, which is also the period for which you can search audit events in the Compliance Center. Searches earlier than this point must be done with the Search-UnifiedAuditLog PowerShell cmdlet.

It’s a good idea for tenants who either want precise control over how long audit data is retained or want to clean up events that don’t add much value in terms of investigations. SharePoint is a notoriously “chatty” application when it comes to the capture of audit events, so I can see why tenants  might decide to keep important events like FileUploaded or FileAccessed for as long as possible while removing some of the chatter after 90 days.

Communication Woes

I don’t have any issue with Microsoft classifying the MailItemsAccessed event as crucial and demanding a premium for its capture into the audit log. Only some tenants will be interested in these events and they might well have E5 licenses already. I can also see the sense of not imposing a huge overhead on Office 365 to capture these events for E3 tenants. It’s just a pity that the communication around the introduction of MailItemsAccessed and its evolution to become a crucial audit event has been so fractured and incoherent. Microsoft can do better.

We track developments in Office 365 auditing, including the kind of events you can extract from the audit log, in a chapter in the Office 365 for IT Pros eBook. Knowing what goes on in a tenant is important and the audit log holds the answers to many mysteries.

Announced in January, Pulled in April, Back in September?

In April 2019, Microsoft rolled back the deployment of code to capture MailItemsAccessed events in the Office 365 audit log (and Exchange mailbox audit log, which feeds into the Office 365 log). At the time, Microsoft said that they planned to restart the rollout “soon,” but they haven’t given an update since.

I asked Microsoft for a status and received this:

“The M365 Auditing product group has an update on audit capabilities. Earlier this year, we announced that availability of Exchange [Online] MailItemsAccessed event was being rolled back. We are actively working on getting these events added into the audit logs and expect staged roll out to start in Q3 of this calendar year. Exact licensing requirements to access these events will be announced closer to roll out. In addition to this, we are working on enabling Longer term retention capability for audit events. Later in the year, we also plan to add more events and capabilities to our audit feature set. More details will be made available closer to release dates. Customers that want to evaluate these new events and features before subscribing will be able to do so with trial subscriptions. We are excited to get these capabilities in the hands of our customers and look forward to getting their feedback.”

Interpreting the Words of the Wise

Here’s what I took out of the statement:

• Microsoft expects to restart the roll-out in Q3. That could be next week, it could be at the end of September.
•  “Exact licensing requirements to access these events” is both interesting and worrying. The MailItemsAccessed event captures details of when an email is opened. I can’t imagine any scenario where Microsoft could justify licensing of the ingestion of these events into the Office 365 audit log and reporting (by the tenant) thereafter. ISVs will also be interested in using these events for their reports. But perhaps Microsoft has something interesting up their sleeves where they use these events for deeper analysis and understanding of how email is used within a tenant (like Workplace Analytics or MyAnalytics). If so, they might be able to justify an add-on license.
• I’m not sure why so much effort is needed to get the MailItemsAccessed events back into the Office 365 audit log unless a) the events are not being captured reliably or b) the storage of so many items (for 180 million active Office 365 users open a lot of messages daily) is proving to be a challenge. In either case, Microsoft isn’t saying.
• “More events and capabilities to our audit feature set.” Well, more data is welcome, if the audit events aren’t truncated or otherwise malformed when they get to the Office 365 audit log.
• “We are excited, etc.” Well, it’s July 4 next week, so I shall let that piece of motherhood and apple pie go by without comment.

In a nutshell, we’re working on getting MailItemsAccessed events back into the Office 365 audit log and it’ll be done by the end of Q3. Or something like that.

We await further developments.

Update October 23: Microsoft is bringing the MailItemsAccessed audit event back, but you’ll have to pay for a new Microsoft Audit 365 feature to get it. See Petri.com for more details.

Do you struggle to keep up-to-date with changing situations in Office 365? The Office 365 for IT Pros team specializes in tracking developments and then reporting what we find in the book You should have a copy!

Exchange Online Promises Forensic Coverage of Mailbox Accesses

In January, we reported Microsoft’s announcement that a new mailbox audit record called MailItemsAccessed in the set of actions that can be captured for mailbox activity. At the time, they said “The new action will capture details of when a message in a mailbox is opened by the mailbox owner, delegate (someone with read access to the mailbox), or using administrative access.” According to Microsoft, the data gathered gives
“comprehensive forensic coverage of mailbox accesses.”

Sometimes things don’t go quite to plan in the cloud, and Office 365 Admin Center notification MC176515 published on 26 March 2019 contained the blunt message that “We have rolled back the feature, at this time, and so the MailItemsAccessed action will no longer be available.” The additional information link in the notification leads to a discussion about how to manage mailbox auditing for Exchange Online that doesn’t mention MailItemsAccessed at all and the title of the notification could be clearer, meaning that administrators could easily miss it.

All-in-all, given that the new audit record opened the possibility of comprehensive forensic coverage of mailbox accesses, Microsoft’s terse statement deserved some interrogation.

Microsoft’s Explanation

I reached out to Greg Taylor, Marketing Director for Exchange, who told me that: “There were technical challenges that during the process of rolling out of MailItemsAccessed to the different regions. Keeping in mind the necessity of complete accuracy and availability of data, we decided to roll the changes back, make the fixes and re-initiate the rollout. We will begin the rollout again soon, and will be sharing more details with respect to the rollout plan and availability.”

Reading between the lines, we can say that:

• Bugs were discovered. Speculating what might have happened, perhaps not all accesses to messages were captured in audit records , or the audit records were not correctly ingested from Exchange Online into the Office 365 audit log (something that has happened before).
• Microsoft detected the problem and because it involves data (loss?), they decided to pull the code that generates the new audit record.
• They’re working on the fixes and will restart the rollout when the new code is available. No timeline is available for when this might be.

Audit Records are Important

I think everyone will agree that audit records are important. Office 365 must generate audit records when expected, the audit records must contain the correct data, be immutable, and discoverable. The problem found by Microsoft with the MailItemsAccessed audit record might belong in either or both of the first two buckets, so it’s good that they have taken the action to find and fix the problem.

Now, if only someone could teach the people who write the Office 365 notifications how to use clear, concise, and informative language, we’d all be in a happier place.

We cover mailbox auditing and the Office 365 audit log in Chapter 21 of the Office 365 for IT Pros eBook. The advent of the MailItemsAccessed audit record is covered there. We’ll add a caveat now and remove it after Microsoft restarts its deployment. It’s what we do in the ePublishing world!