Reporting Spam to Make Exchange Online Protection Better

Office 365 Admins and Users can Report Spam and Phishing

From time to time, reports come out to criticize the performance of Exchange Online Protection (EOP), mainly its inability to detect spam and phishing messages. Invariably, the report is authored by a vendor anxious to sell their mail hygiene service with promises that a much higher proportion of bad email will be caught if Office 365 tenants would sign up. It’s true that routing email through multiple cleansing services can have a benefit; what’s not so clear is if third parties do any better than Microsoft’s own Advanced Threat Protection (ATP), which serves the same purpose.

In any case, all the services that aim to block spam and malware depend on intelligence to understand the latest tactics taken by attackers to trick defenses and allow their email to get to user mailboxes. If you want to see EOP do a better job of blocking malware, you can help Microsoft by reporting messages that get through.

Two methods are available:

• The Report Message add-in for Outlook allows users to report messages as junk, phishing, or a false positive (not junk). Figure 1 shows how to use the Report Message add-in with the new OWA. The add-in works for Outlook desktop (Windows and Mac) as well and should be a basic part of the Outlook configuration for Office 365 clients.
• The Submissions section under Threat Management in the Security and Compliance Center allows admins to report messages. This is a relatively new feature described in this Microsoft post.

In both cases, reported messages are sent to Microsoft for analysis so that they can tweak EOP to do a better job.

Administrator Submissions for EOP Processing

Before administrators can submit a report to Microsoft through the Security and Compliance Center, they need some details about a bad message that only a user can give. Every message has a network message identifier that should be unique. An easy way to find the message identifier is to run the Outlook’s Message Header Analyzer add-in (also available as a GitHub project) and look for the X-MS-Exchange-Organization-Network-Message-Id property (Figure 2).

Another method is to use OWA’s Show Message Details option (Figure 3). The equivalent in Outlook desktop is to look at the message properties through the File menu.

In either case, I prefer to use the Message Header Analyzer because it’s easier to locate the message identifier. Once you have the message identifier, you can submit a new report. Go to the Threat Management section of the Security and Compliance Center, select Submissions, and then New submission. Fill in the information about the problem message (Figure 4) using the network identifier to find the message. You need to select one of the message recipients too. If you have a copy of the message (EML format), you can upload it too. Indicate if you think the message should have been blocked or passed, select what kind of problem you see in the message (spam, phishing, or malware), and submit the message for processing.

The Submissions dashboard (Figure 5) shows you a breakdown of user (via the Report message add-in) and admin submissions.

For admin submissions, the reported messages show when EOP has finished analyzing their content. Select a completed message to see what the verdict is. In the case of the message verdict shown in Figure 6, the user had complained that obvious spam had reached their Inbox. The clue to why this was so was in the policy type “Sender domain in safe list.” The user’s junk email settings accepted all email from outlook.com senders, so even though EOP had marked it as spam, the user’s preference had overridden the analysis. The learning from this is to educate users not to mark consumer email domains like outlook.com and gmail.com as safe because spammers often create throwaway accounts in these domains to use to send mail. It’s perfectly acceptable to mark individual known accounts from these domains as safe senders.

Of course, automated detection systems can only go so far. Some spam and malware will get through and it’s then up to user intelligence to recognize and suppress bad email. And hopefully, when they do see spam arriving in their inbox, they’ll know how to report the messages themselves or how to give admins the necessary information to make the report on their behalf.

---

There’s lots more to learn about Exchange Online Protection and Advanced Threat Management in the Office 365 for IT Pros eBook. Be informed and be secure!