Why Humans Should Apply Their Knowledge of Office 365 When Reviewing OCAS Alerts

Don’t Assume Everything a Computer System Spits Out is the Truth

As Office 365 for IT Pros subscribers know, we publish a new edition annually. Part of the preparation for a new edition is an end-to-end technical review of all content. This happens to make sure that our material is current and accurate. The review picks up issues like dead hyperlinks, unnecessary (some might say verbose) text, and outdated graphics. It’s a good process to keep our authors focused on delivering the best possible book, something that’s only possible because of our ePublishing model.

Office 365 Client App Security

Microsoft 365 applications update GUIs on an ongoing basis. Sometimes it’s just a matter of adding a new option or changing the words on a button. Other times it’s a more fundamental makeover, such as the introduction of a new interface for content searches. Office 365 Cloud App Security (OCAS) is available to tenants with Office 365 E5 licenses. OCAS is a subset of the full Microsoft 365 Cloud App Security product, tailored for Office 365.

Figuring Out Impossible Travel

OCAS analyzes the data ingested from multiple workloads into the Office 365 audit log to identify anomalies and other potential issues. As we reviewed the chapter on reporting and auditing, the technical editor highlighted the need to refresh some screen shots to reflect the new OCAS GUI, which brings us to Figure 1, which shows how OCAS highlights a potential impossible travel activity issue.

In other words, the IP addresses captured by OCAS for client connection events over a certain period originate in multiple countries where it would be impossible for the user to travel between those countries during that time. In this case, the alert flagged interactions from Ireland and the Netherlands within a 99-minute period. It’s possible to fly from Dublin to Schiphol in this time, so that’s probably why OCAS uses this period to test for suspicious connections.

Applying the Human Touch

On the surface, this looks like a problem which deserves investigation to understand if an attacker has compromised the user’s account. In fact, it’s a good example of how human intelligence can quickly make sense of activity which a computer deems suspicious. At first glance, the facts are:

• The user signed in from two different IP addresses within a short period.
• The IP addresses indicate connections from Ireland and the Netherlands.
• In both cases, the application was Teams.

But when we examine the detailed records, we see a continuous set of connections first originating from The Netherlands and then switching to Ireland, all within a very short time (Figure 2). Most of the records are for login events. Some others (not shown here) record SharePoint Online activities like opening a document.

Searching the audit log with the Search-UnifiedAuditLog cmdlet to find the underlying records confirms that the user connected multiple times to work with Teams and SharePoint Online over the period. The IP addresses are correct, the connections valid, so what’s happening? Everything makes more sense when you consider that:

• Teams and its associated applications use Azure AD secure token service (AzureActiveDirectoryStsLogon) logons to validate user credentials. The logged sign-in events all use the token service.
• The tenant is in Microsoft’s EMEA datacenter region, and the Teams service runs in the region.
• The EMEA datacenter region includes datacenters in Ireland and the Netherlands.

Therefore, the most likely explanation is that the Teams client attempted to use its access token to connect. During this process, the server handling the request changed from a server in the Netherlands to one in Ireland. Azure AD captured details of the connections and sent them to the Office 365 audit log where OCAS picked up the information, analyzed the events, and concluded that a potential impossible travel situation exists. As it happens, I know that this is exactly what transpired, but it’s a great example of how tenant administrators need to apply their knowledge of Office 365 and how Microsoft’s datacenter infrastructure operates to assess and resolve a flagged alert.

Administrator in Office 365

Another thing to consider is that OCAS notes that the user is an administrator in Office 365. This doesn’t mean that the account is a tenant administrator. It means that the account holds an administrative role. In this case, the account holds the SharePoint administrator role. Again, when probing details of an incident, check before assuming the worst.

Resolving Issues

This case did not take much to resolve. Other OCAS alerts require substantially more effort to understand and conclude. The point I make is that OCAS is a tool to highlight issues to administrators which deserve some attention. Just because OCAS flags an alert isn’t evidence that a problem exists. Always use human intelligence to validate computer indications when resolving alerts. You’ll get better results that way.