Using Teams Compliance Records for eDiscovery

Sometimes Web Content Isn’t True

Updated September 16, 2021

Always verify what you read in a blog before you accept it as fact. A myriad of reasons might make text unreliable. People make mistakes as they write, or in their understanding of a topic, or use different versions of software to what you have. The problem doesn’t exist only in independent blogs. Microsoft publications get things wrong too. A recent example in when they updated their guidance about what’s captured in Teams compliance records and can be used for eDiscovery.

It’s good when Microsoft does this because there’s a ton of misconception in the technical community about the purpose and usage of Teams compliance records. I have been told that it is possible to backup Teams by copying the compliance records in an Exchange Online backup, something that is complete and unadulterated rubbish. You can copy the compliance records, but you’ll never be able to restore those items into Teams. As explained below, Microsoft updated their page because it contained some errors (correct information is now online).

How the Microsoft 365 Substrate Captures Teams Compliance Records

In summary, here’s what happens. When someone posts a message to Teams, the Microsoft 365 substrate captures a copy of the message as an Exchange mail item. A Teams message is not a mail item, so some transformation occurs in the capture. For this reason, referring to this process as journaling is incorrect. Unlike email journaling, a perfect legally-defensible copy of the original item does not result.

The Microsoft 365 substrate writes Teams compliance records for personal chats into user mailboxes. Compliance records for channel messages go into group mailboxes. In both cases, the mail items are stored in the TeamsMessagesData folder in the non-IPM (system) part of the mailbox. This folder is hidden from clients. Originally, Teams stored its compliance records in a folder under the Conversation History folder. Teams changed the folder it uses in October 2020.

Storing records in Exchange Online mailboxes has been done since Teams first generated compliance records in 2017. The substrate captures compliance records for all Teams conversations, including those involving hybrid users whose mailboxes are on on-premises Exchange servers and guest users. Compliance records are also captured for federated chats with Skype consumer users. In these cases, special hidden mailboxes store the compliance records. I have heard these mailboxes referred to as phantom, shard, or cloud-based mailboxes.

Conversations in private channels are a special case. Private channels don’t have a group mailbox, so the substrate writes copies of these messages into the personal mailboxes of channel members and tags the items as belonging to a private channel.

Teams compliance records are also used for retention policy processing. The Exchange Managed Folder Assistant removes expired records from mailboxes according to policy. Those deletions are synchronized back to Teams, which then removes the real messages from its store.

Teams and eDiscovery

Because the compliance records are in Exchange Online mailboxes, they are indexed and discoverable by content searches. eDiscovery never operates against the “real” Teams message data, which remains in Azure Cosmos DB. All content searches use the indexes populated by Exchange Online, so the items returned by a content search (Figure 2) come whatever can be found in Exchange Online, including records for conversations involving hybrid, guest, and federated users.

When Teams compliance records are found by a content search, they can be exported as individual items or to a PST. Figure 3 shows an item found with a content search as viewed through Outlook. The compliance record contains important information like the title of the topic and the name of the team the item is posted to. Inline images, GIFs, links, and tables are also visible.

Everything seems good, if you understand and appreciate two facts: first, the compliance records stored in Exchange Online are copies and not real Teams data. Second, the transformation process to copy a Teams message into an Exchange mail item means that some Teams content does not end up in the searchable content.

Moving Content from Teams to Exchange Online

When the substrate copies a Teams message to create a mail item in Exchange Online, the following information is included:

• Links to any embedded emojis, stickers, inline images, and GIFs.
• Tables.
• Embedded deep links to other Teams messages.
• Sharing links to files in SharePoint Online document libraries.
• For channel messages, the subject of the message is recorded (if available) as is the name of a team a message is posted to. For personal chats, the names of the people involved in the conversation are captured.

However, problems occur with these elements of Teams messages:

• Reactions (for example, a like, heart, or smile) given to messages. In an eDiscovery context, reactions can be important signs that certain individuals have seen a conversation in the same way that changing the read status of an email from “unread” tells you that the message was opened.
• Recordings of audio messages.

Update April 2021: Teams compliance records omitted code snippets created in Teams messages in the past. Retesting shows that these elements are now captured as .DAT attachments for the compliance records. The .DAT attachments hold the HTML-formatted content for the code snippet. They can be opened and examined with a text editor.

In addition, compliance records captured for praise messages only include the text of the praise and not the graphics. Compliance messages for messages with quoted text include the text but not the formatting to mark the text as a quote.

An Insight into the Exchange Items

You can use the MFCMAPI utiliity to see what’s in the Teams compliance records captured as mail items in Exchange mailboxes. Examining a Teams compliance record with MFCMAPI very quickly tells you what the item does and does not contain. Mail items are collections of MAPI properties and the content of those properties constitute what clients display, what Office 365 indexes, and what’s discoverable by a content search.

Figure 4 shows an example of how to review the properties of a Teams compliance item. The PR_HTML property stores the HTML-formatted content of the item that clients like Outlook display. In this case, you can see the HTML code describing a “smile” sticker and pointer to a GIF (stored online in a Teams content delivery network).

Webinar Information

Teams webinar information such as the event information, speakers, and attendance report is captured in a set of Microsoft Lists in the meeting organizer’s OneDrive for Business account to make the data available for search and eDiscovery,

Some Teams Data is Invisible for Compliance

Good as the substrate is at capturing Teams messages, some Teams data remains invisible from a compliance perspective, including:

• Voice memos recorded with the Teams mobile client.
• Whiteboards used during Teams meetings (Teams recordings don’t capture whiteboard activity in the video feed and the whiteboard service is not indexed for eDiscovery).
• Teams meeting recording stored in Stream are not available for eDiscovery either. The spoken text in recordings stored in OneDrive for Business and SharePoint Online is captured for search but is not yet available for eDiscovery.

The point is that you shouldn’t assume that everything done in Teams is captured for compliance purposes.

Private Channels

Teams private channels pose another challenge for compliance administrators. The compliance records for conversations in these channels aren’t captured in group mailboxes. Instead, the substrate creates copies in the personal mailboxes of channel members. The messages for private channels and personal chats are mixed up. However, compliance records for private channel messages have different tags to the items for chats. See this post for more information.

In addition, the documents created in the SharePoint sites belonging to private channels won’t be included in content searches unless the URLs for the sites are added to the search locations.

Call and Meeting Records

Teams compliance records are captured for meetings and calls. These records (Call Detail Records or CDRs) note when the meeting or call happened, the participants, and when each participant joined and left the call. For instance, a call between two users might be captured like this:

Start Time (UTC): 5/20/2020 12:57:02 PM
End Time (UTC): 5/20/2020 1:27:36 PM
Duration: 00:30:34.1604437
[5/20/2020 12:57:02 PM (UTC)] vasil@michev.xxx joined.
[5/20/2020 1:27:36 PM (UTC)] vasil@xxx left.
[5/20/2020 12:57:02 PM (UTC)] Tony.Redmond@xxx  joined.
[5/20/2020 1:27:36 PM (UTC)] Tony.Redmond@xxx left.

Because compliance records are captured as mail items, the person who starts the call is noted as the sender (for meetings, it’s the person who schedules the meeting). The people who participate are captured as message recipients and the subject will be something like:

Call (Complete)/Thread Id: /Communication Id: 58365f07-e530-4b69-bef8-71dca438e65a/Vasil Michev (MVP) (Guest),Tony Redmond

The subject for a meeting looks like:

Meeting (ScheduledMeeting)/Thread Id: 19:meeting_NzI1ZTA3ODUtZDhiZC00MTRiLWE5NDEtNmRiMWFlMzI5MmZj@thread.v2/Communication Id: 0f03bcd9-f48b-4bdc-887f-a78f3087d772

When meetings or calls are with visitors, you’ll see them noted with addresses like:

teamsvisitor:13c3a4132a584b918b9c6528b6dabef9 <13c3a4132a584b918b9c6528b6dabef9>

And the transcript will look like this. All we can say is that four anonymous visitors joined a meeting hosted by a tenant user. There’s no way to resolve the addresses signed to visitors into their email addresses.

Start Time (UTC): 6/20/2020 3:54:56 PM
End Time (UTC): 6/20/2020 5:05:28 PM
Duration: 01:10:31.9773921
[6/20/2020 3:54:56 PM (UTC)] teamsvisitor:13c3a4132a584b918b9c6528b6dabef9 joined.
[6/20/2020 5:05:27 PM (UTC)] teamsvisitor:13c3a4132a584b918b9c6528b6dabef9 left.
[6/20/2020 4:02:46 PM (UTC)] teamsvisitor:748997de34a346149f3c72a8d3c6c50f joined.
[6/20/2020 4:36:21 PM (UTC)] teamsvisitor:748997de34a346149f3c72a8d3c6c50f left.
[6/20/2020 4:08:42 PM (UTC)] teamsvisitor:ba1c0907edae4a6791f9fdc17adb9fc1 joined.
[6/20/2020 5:05:28 PM (UTC)] teamsvisitor:ba1c0907edae4a6791f9fdc17adb9fc1 left.
[6/20/2020 4:21:17 PM (UTC)] teamsvisitor:d1cce625a5ee4a29911a7a954a89f1ee joined.
[6/20/2020 4:23:29 PM (UTC)] teamsvisitor:d1cce625a5ee4a29911a7a954a89f1ee left.
[6/20/2020 3:59:38 PM (UTC)] Tony.Redmond@xxx joined.
[6/20/2020 5:05:27 PM (UTC)] Tony.Redmond@ left.

Guest accounts or people authenticated with another Office 365 tenant have their email address noted.

Compliance records for meetings and calls are searched along with other items. If you want to refine a search to look specifically for records for meetings or calls, you can do this by searching for the MAPI item class used for these items (Figure 5).

• To search for all Teams items (messages, calls, and meetings), search using the message kind condition and look for MicrosoftTeams. Warning: Always add extra search parameters (like certain users or a date range) as looking for items based on the message kind will return every item of that kind. In the case of Teams, this could be millions of items.
• To search for Teams meeting records, use the keyword Itemclass:IPM.AppointmentSnapshot.SkypeTeams.Meeting. Again, make sure to add extra search parameters to restrict the number of items returned by the search.
• To search for Teams calls records, use the keyword Itemclass:IPM.AppointmentSnapshot.SkypeTeams.Call.
• To search for both, use the OR operator: Itemclass:IPM.AppointmentSnapshot.SkypeTeams.Meeting OR Itemclass:IPM.AppointmentSnapshot.SkypeTeams.Call

Remember that Teams compliance records, including call detail records, are stored in Exchange Online mailboxes, so always search Exchange Online locations rather than SharePoint Online when looking for Teams content.

The Problems of Assuming Web Text is Accurate

Coming back to the problems of web text that’s wrong. Microsoft published its article on April 15. The text prompted some online commentary about Teams compliance, all blissfully repeating the errors in Microsoft’s article. Here’s an especially egregious example from April 22, complete with erroneous text pasted in from Microsoft’s article. Microsoft updated its page on April 29 with accurate information. It will be interesting to see if those who repeated the incorrect text now recant.

The experience proves that you should always check and verify text found in the web before you trust and depend on it. Even this text!

---

Compliance is a complex area. If you need to know more about compliance, subscribe to the Office 365 for IT Pros eBook where you’ll find Microsoft 365 compliance explained in depth.