Table of Contents
Good Tactical Move but Not the Right Strategy
Although I understand the tactical necessity for Microsoft to enable OAuth 2.0 authorization for programs wishing to use the POP3 and IMAP4 protocols to retrieve emails from Exchange Online, I don’t think it is the correct strategy for Exchange Online IMAP4 and POP3 access. Essentially, Microsoft is facilitating the continued use of antique messaging protocols instead of forcing developers to change to the Microsoft Graph APIs. I think that’s wrong, but I know why Microsoft is doing it.
The big basic authentication turnoff for Exchange Online is now just 88 days away. October 1 marks the point when Microsoft begins to disable seven email connectivity protocols for Exchange Online. It will take time for Microsoft to process all tenants, but eventually those wishing to use protocols like POP3, IMAP4, and Exchange ActiveSync will have no choice but to use modern authentication. In other words, a username and password won’t be enough.
Microsoft has been preparing to remove basic authentication from Exchange for over two years. The scale of Exchange Online, the product’s history, and the number of protocols and devices combine to create a myriad of complexities. Automatically upgrading the profiles for Apple’s device Mail app on iOS and iPadOS devices is a good example of the kind of detailed planning and technical execution involved in this project.
I don’t know how many companies have applications that use POP3 or IMAP4 to programmatically retrieve messages from mailboxes. Enough must exist for Microsoft’s fabled telemetry to detect a potential customer satisfaction problem should the turnoff proceed without an answer released with sufficient time for customers to prepare.
Registered Apps and IMAP4/POP3 Permissions
Microsoft’s solution is to allow customers to create Azure AD registered apps and assign the necessary permissions to allow the apps to use IMAP4 or POP3 to interact with mailboxes. Figure 1 shows the assignment of the IMAP.AccessAsApp permission to an app. The equivalent permission for POP3 access is POP.AccessAsApp.
After assigning the permissions, an administrator must grant consent to allow the app to use the permissions to access user mailboxes.
There’s nothing strange here. Apps follow the same process to allow them to use Graph API permissions to access other kinds of information from user accounts to Microsoft 365 groups. The only difference is that two specific permissions exist to control access via the IMAP4 and POP3 protocols.
Service Principals and Exchange Online
Registered apps have service principals, which are, in the words of a Graph API architect, “convenient holders for permissions.” Exchange Online boasts a new PowerShell cmdlet called New-ServicePrincipal to make the service principal of an app holding IMAP4 or POP3 permissions known. In this example, the $ClientId variable holds the application or client identifier for the app, the $ServiceId variable holds the object identifier for the service principal, and the $TenantId variable holds the tenant identifier.
$ServiceId = (Get-MgServicePrincipal -All | ? {$_.displayname -eq "POP3 and IMAP4 OAuth 2.0 Authorization"} | Select -ExpandProperty Id)
$TenantId = (Get-MgOrganization).Id
$ClientId = (Get-MgServicePrincipal -All | ? {$_.displayname -eq "POP3 and IMAP4 OAuth 2.0 Authorization"} | Select -ExpandProperty AppId)
New-ServicePrincipal -AppId $ClientId -ServiceId $ServiceId -Organization $TenantId -DisplayName "OAuth for POP3 and IMAP4"
Once a service principal is registered with Exchange Online, administrators can run the Add-MailboxPermission cmdlet to assign receive permissions to the service principal, just like the granting of regular delegate access to mailboxes.
Add-MailboxPermission -Identity "Kim.Akers@office365itpros.com" -User $ServiceId -AccessRights FullAccess
In passing, I should note that this is all theoretical on my part because the New-ServicePrincipal cmdlet is not available yet in any tenant that I have access to. In any case, the theory is clear:
- Create a registered app.
- Assign the appropriate permission(s) to the app.
- Make the app’s service principal known to Exchange Online.
- Use the app’s service principal to gain delegate access to specific mailboxes.
Application Access
I have no need to use IMAP4 or POP3 to access Exchange Online mailboxes, but I did want to test that I could get an OAuth 2.0 access token containing the necessary permissions. In production use, an app should use a certificate for authentication. To test, I used a client secret and ran this PowerShell code:
$Uri = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$Body = @{
client_id = $ClientId
scope = "https://ps.outlook.com/.default"
client_secret = $AppSecret
grant_type = "client_credentials" }
# Get OAuth 2.0 Token
$TokenRequest = Invoke-WebRequest -Method Post -Uri $Uri -ContentType "application/x-www-form-urlencoded" -Body $Body -UseBasicParsing
# Unpack Access Token
$Token = ($tokenRequest.Content | ConvertFrom-Json).access_token
I then checked the access token and found that the expected permissions were present (Figure 2). All is well and the app has authorization to access the mailboxes.
Pragmatic, but Wrong Strategy
Developers will probably welcome Microsoft’s approach because it means minimal change for their code. All they need to do is replace the code to sign into a mailbox using basic authentication with code to get an access token. Afterward, the rest of the app code to access messages in a mailbox should work.
Pragmatic as it is, I think Microsoft’s approach is a short-term tactical win. The long-term solution is to move to the Outlook Graph API to access mailboxes. This uses the same registered app approach with different permissions, but it’s more functional. And anyway, app developers will have to embrace the Graph sooner or later to send email via SMTP. The SMTP AUTH protocol is a current exception to Microsoft’s effort to remove basic authentication for email connectivity, but that exception won’t last forever.
I guess that October 1 date is just too close to ask developers to recode their applications. But if you’re in the position where your tenant has some apps that exploit Exchange Online IMAP4 or POP3 mailbox access , consider dumping these old protocols and laying a better foundation for the future. If you have the time…
Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.
how it is possible that i can’t use the cmdlet New-ServicePrincipal ?
It look like this command don’t even exist, i see Microsoft talk about this cmdlet here too : https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
But when you click on the link, it’s 404…
As Microsoft noted in their blog, the cmdlet is rolling out across Office 365. It takes time to get to all tenants.
Hi, should I take it that IMAP4/POP3 will be avaiable via legacy auth with this steps after 1st of October?
No, IMAP4 and POP3 clients will need to use modern authentication after Microsoft disables basic authentication for those protocols. Clients need to be updated to use modern authentication as per https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth?WT.mc_id=M365-MVP-9501. If they are not, clients will stop being able to connect to Exchange Online.
i have my code using pop3 anywasy i have generated the token but how to utilise the token and use?
POP3 pop = new POP3(MailSender, strTo, strPassword);
pop.Port = POP3PortNo;
AdvancedIntellect.Ssl.SslSocket ssl = new AdvancedIntellect.Ssl.SslSocket();
pop.LoadSslSocket(ssl);
pop.Connect(strTo, strPassword, MailSender);
I’m sorry, but I can’t debug code for you. I suggest you file a support request with Microsoft to seek assistance.
Tony, I followed the instruction step by step above, retrieved the token, got the POP.AccessAsApp role in it. Then send the POP3 request, but got rejected:
AUTH XOAUTH2
dXNlcj14dXlhbmc3NDIx….
<= -ERR Authentication failure: unknown user name or bad password.
The same code with the same email account working fine with the OAuth2 authorization code flow generated token. Don't know anyone successfully get the POP3 or IMAP to work or not.
I’m waiting for the cmdlets to show up in my tenant to try things out… So I am not much help at present.
Tony, I think it might be the ExchangeOnlinManagement module you used, it should be:
PS C:\Windows\System32> Import-Module ExchangeOnlineManagement; Get-Module ExchangeOnlineManagement
ModuleType Version PreRelease Name ExportedCommands
———- ——- ———- —- —————-
Script 2.0.6 Preview6 ExchangeOnlineManagement {Get-CustomNudge, Get-CustomNudgeAssignment, Get-…
Then you must run the
PS C:\Windows\System32> Connect-ExchangeOnline
then the cmdlets will work. (at least that’s my case)
I did use that module, but not the preview version. I’m still using the production version (2.0.5). Anyway, the cmdlet showed up and I was able to create a new service principal in EXO. I then noticed that I hadn’t included a displayname, so I removed the service principal (there’s no Set-ServicePrincipal cmdlet), and now I can’t add the service principal back. It might just be a caching thing, so I shall wait a while and try again.
Is the display name significant? I was trying to delete a ServicePrincipal after using a wrong object_id. It was painful as it kept saying the appId already in use. And I end up with creating a new app, and go over the registration etc again.
The display name is not significant, but the cmdlet should work properly. I’ve flagged this to Microsoft and am waiting their response.
Tony, did you successfully make an end to end Pop3/Imap request with your instructions? I am still getting rejected for a valid email account. Have no weapon to attack the problem.
I am working through this issue with Microsoft and have asked them for sample code to show how to authenticate once a service principal is in place… Stay tuned. It takes time to make things happen.
Understand, you made a great detailed instruction than what Microsoft posted. And I thought it should just work. If they don’t shutdown basic auth in <3 months, I won't panic. lol. I also start looking convert my pop3/smpt to graph, which seems doable in couple of weeks.
Have you looked at https://github.com/DanijelkMSFT/ThisandThat/blob/main/Get-IMAPAccessToken.ps1? That’s where Microsoft pointed me to find information about IMAP4 logins.
Thanks Tony, I will check that code. (I was on POP3 side, but this may help too)
Tony, I just got my Microsoft ticket resolved by using the scope “https://outlook.office365.com/.default” in the token request. Please make correction in your instruction.
$Body = @{
client_id = $ClientId
scope = “https://outlook.office365.com/.default”
client_secret = $AppSecret
grant_type = “client_credentials” }
In large enterprises there are SO many legacy apps that arent going to completely re-develop an application to use MS-Graph, that Microsoft had to allow for those older protocols to work.
One thing that i believe gets overlooked is converting the apps to use Modern Auth via “Delegated Permission” instead of “Application Permission”, this is the easiest way to make the switch with minimal changes from both parties, as the permissions to mailboxes stay the same as they were, based on the user, instead of adding a service principal and app access rules to limit its mailbox access.
Hey Eric,
We also reviewed the delegated permission setup, but they dont seem to be adequate for M2M (machine to machine) setting, where a script periodically polls a mailbox in the background without user intervention. We have some python scripts on a server watching mailboxes.
The problem we have is that you cannot exempt the app from multi-factor-authentication (confirmed by MS support ticket) as the user having delegated permissions needs to MFA every now and then. So no unattended “automation” possible here as it seems.
One would also need to create service accounts, since the users that have delegated permissions might change roles, etc. So you want a constant user account that is not changing to have the delegated permissions.
Would you have an idea on how to overcome those challenges?
We also looked at “application permissions”, but they automatically grant access to ALL mailboxes in the tenant, which is not desired either.
Best Sebastian
Hi Tony, from a firewall perspective, does this method require 443 to be open to the Azure Ad endpoints? If using IMAP and OAuth, is 993 still required? I am not able to find sufficient Microsoft information as I am wrestling with out Security team to implement, but they want documentation to back it up.
The ports are still used. It’s only the authorization (OAuth) and authentication that changes. The security team should understand how OAuth works (it’s well documented by Microsoft and others) and the protocols (POP3 and IMAP4) dictate the well-known ports they use.
993 is still used or 993 and 443 are used? Thanks
Please can you tell me how can I run new-serviceprincipal? I’ve installed preview version of onlineexchangemanagement but it did not show up, I cannot run powershell from the azur eportal because there is no azure subscription tied to my office365 account, basically I tried everything but wasn’t able to find a way to run new-serviceprincipal
New-ServicePrincipal runs the way I describe it in the article. I am using V2.0.5 of the Exchange Online Management module.
Does it mean if I have few hundred IMAP mailboxes I need to create an app and Service Principal for every of them?
Not to mention various client applications almost impossible to recode and developers resist to make any change, I wonder how it ends
btw, Microsoft was preparing for 2 years yet 3 months before the big change giving something new, and how to work with developers and vendors? I still meet plenty of them that hear about the change for the first time from me…
This is for application access to mailboxes, not regular email clients
This means that to configure an email with POP/IMAP I will require an external application so that it can already use modern authentication
No, it means that the email client must support modern authentication. Thunderbird is a good example: https://office365itpros.com/2022/07/14/imap4-clients-to-exchange-online/
Thanks for the post. Got it working finally. Struggeled for while with two things:
1. I was posting access token requests to the older login endpoint. It returns a token but some stuff is missing in the jwt, `roles` e.g. So be sure to include `v2.0` in login url: https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token
2. Tried a lot of different scope values and the one that finally worked is `https://outlook.office365.com/.default`, which is mentioned here in the comments and stated also quite far down on https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#use-client-credentials-grant-flow-to-authenticate-imap-and-pop-connections
Hey all,
would you recommend IMAP.AccessAsApp as an alternative to “delegated permissions” or “application permissions”?
We have some python scripts accessing mailboxes to help with some automations. But seems either delegated or app permissions both have draw-backs in our scenario.
1. app permissions give access to ALL mailboxes, not a single one. So in case the python script missuses the permissions one has a security breach
2. The new New-ApplicationAccessPolicy does not seem to support a “default deny” policy and subsequent “allow app X to access mailbox Y” policies
3. delegated permissions need a named account, which could change role/leave/etc.
4. Service accounts are on extra cost and possibly hard to manage or dont scale to multiple hundreds of accounts
5. enterprise apps using delegated permissions to AzureAD or EXO cannot be excepted from MFA. Confirmed by MS support ticket as we tried to configure a conditional access policy which didnt work.
So my question to all of you would be:
Is IMAP.AccessAsApp a valid futureproof tech setup?
Can you share any other ideas on how to securely connect an app to a single inbox in an unattended machine-to-machine setup?
Love to hear your input!
Thanks much
Sebastian
I am not an expert in this space and hesitate to answer without doing some research. I suggest you attend the Julian Stephens sessions at next week’s NEC (https://mecairlift.event.microsoft.com/) and ask him these questions as he’s the leader for app permissions for Exchange Online.
I am sure that Microsoft Graph APIs will be just another piece of crap that Microsucks has produced. They probably started releasing security updates for it was even completed.