Table of Contents
Error in Retention Processing Removes Personal Chats
Reading Monday’s report in the Register about the problem KPMG suffered when erasing Teams personal chat data for 145,000 users, you might ask the question “how did this happen”? The answer is that an error was made in a change applied to an Office 365 retention policy applied to Teams personal chat. Instead of removing a user from the policy, the update applied the policy to the entire KPMG deployment.
Teams compliance records for personal chats are captured in user mailboxes. These records are used by content searches, eDiscovery cases, and other data governance features like communications compliance policies. Office 365 retention policies to control Teams data use the Exchange Managed Folder Assistant (MFA) to process the compliance records in mailboxes according to policy settings. Exchange Online synchronizes the deletions made by MFA to remove compliance records from the mailboxes to Teams, which then removes the items from its data store in Azure Cosmos DB. The cycle completes when the deletions synchronize from Teams to clients. The overall process used to take far longer than it used to.
KPMG Error
Errors happen in life and in IT. I don’t have direct knowledge of what happened in this case, but it looks very like an administrator updated the retention policy for Teams personal chat and applied it to everyone instead of excluding a user from the policy. It’s easy to do if you’re not paying attention (Figure 1).
Only messages in personal chats were affected. Files shared in chats remained unaffected in user OneDrive for Business accounts. Different settings in retention policies for Teams apply to channel conversations, so these messages were unaffected too.
Restrictive Policies for Chats
Many organizations apply restrictive retention policies to Teams personal chat, which is one of the reasons why Microsoft enabled a 1-day retention period for this data. The logic is that personal chat is much like the discussions in WhatsApp and all business discussion should be conducted through Teams channels. That’s a reasonable approach but one that founders on the simple fact that Teams supports group chats for up to 250 users. You can do a lot of business with 249 others, especially if you follow the advice to move debates into chats before presenting an agreed position in a team channel.
Avoiding Errors in Retention Processing
I’m sure KPMG has good change control policies in place to make sure that the right change is made at the right time. You could question making a change to retention policies manually in such a large organization, but on the face of it the proposed change seemed straightforward and easy.
Other large tenants develop and deploy PowerShell scripts to automate management operations and test and debug those processes in test tenants (as the Register report notes, “to automate service execution and remove human intervention in policy management”). However, it can be argued that the nature of retention policies is that they don’t change all that often, making the investment to automate this operation less attractive than others.
The basic fix for something like this is to make sure that anyone who makes a change understands the technology and knows what will happen if they update a retention policy. Asking someone qualified to check that the right change is being made before it is committed is also possible. Such is the benefit of hindsight.
Proponents of backup services for Office 365 will say that they could rescue the situation by being able to restore the deleted compliance records back to user mailboxes. This will certainly solve the eDiscovery gap. Regretfully, no API exists to allow restore back so that the missing data appears as chats, leaving users no better off than they are now. There’s no silver restore bullet available in this scenario.
Retention policies can be complicated and they work differently across Exchange Online, SharePoint Online, OneDrive for Business, and Teams. Learn how to master retention processing by reading Office 365 for IT Pros.
Hello, again, Tony.
While I know no third-party backup could have fixed this particular issue, do you not see how this proves the point that Retention Policies are not that easy to use? Or at the very least, mistakes can be made that you can’t recover from? What if they had made this mistake with EOL and immediately wiped out all its retained data?
I’m sure you will mention that you can activate the “even you can’t delete history” feature, but I’m guessing most people don’t turn that on because they’re worried about not being able to change their mind if the storage costs get out of hand – which I’m hearing that they can be.
“Retention policies are not that easy to use”: Rather, retention policies are too easy to set up and change (which happened in this case). It’s the knowledge of how retention works and how retention policies are processed that causes the problems.
“Even you can’t delete history”: do you mean retention in SharePoint and its effect on storage quotas and potential cost? I have covered this extensively in the past (https://petri.com/how-retention-impacts-office-365-storage for instance), and as this doesn’t affect the issue under debate (Teams chats), it’s not covered here.