Sometimes it is necessary to find and delete duplicates within a Microsoft Exchange or Office 365 mailbox. For example, when a clean Migration has been run within Migration Monster without first deleting and recreating or emptying the destination mailbox.


The following PowerShell script and procedure can assist with this:-


Please check all links for the latest versions of scripts and procedures as from time to time they are updated.


We have also attached the  script it self as an attachment but again, please check for the latest versions.


https://gallery.technet.microsoft.com/office/Removing-Duplicate-Items-f706e1cc 


The script was written by Michel de Rooij  https://social.technet.microsoft.com/profile/michel%20de%20rooij/ 


Step 1: Have to have version 5 of PowerShell installed

You'll need to confirm you have Powershell version 5 running for this work.

To check your version of Powershell type in $psversiontable

Here's a link for what's needed to install the most current version -


https://docs.microsoft.com/en-us/powershell/scripting/setup/installing-windows-powershell?view=powershell-6


Step 2: Need to make sure to have Microsoft Exchange Web Services Managed API 2.2 installed

Download here
https://www.microsoft.com/en-us/download/details.aspx?id=42951


Step 3: Need to install Microsoft Online Services Sign-In Assistant for IT Professionals RTW

Download here
https://www.microsoft.com/en-us/download/details.aspx?id=28177

Step 4: Need to install PackageManagement PowerShell Modules Preview - March 2016

Download here
https://www.microsoft.com/en-us/download/details.aspx?id=51451

Step 5: You'll need to configure Powershell connection to Office 365 (if applicable)

https://docs.microsoft.com/en-us/office365/enterprise/powershell/connect-to-office-365-powershell

Step 6: Setup impersonation of the mailbox you will be administering i.e. removing the duplicates from

https://goo.gl/muLsjx


Syntax and examples below but please also see the latest updates at:- https://gallery.technet.microsoft.com/office/Removing-Duplicate-Items-f706e1cc


EXAMPLE

$cred= get-credential

.\Remove-DuplicateItems.ps1 -Mailbox <mailbox.org> -Credentials $Credentials -impersonation -includefolder '#inbox#\*','#calendar#\*' -MailboxWide -verbose -server outlook.office365.com


Syntax:
Remove-DuplicateItems.ps1 [[-Identity] <String>] [[-Type] <String>] [-Retain <String>] [-Server <String>] [-Impersonation] [-DeleteMode <String>] [-Credentials <PSCredential>] [-Mode <String>] [-MailboxOnly] [-ArchiveOnly] [-IncludeFolders <String[]>] [-ExcludeFolders <String[]>] [-PriorityFolders <String[]>] [-MailboxWide] [-NoSize] [-NoProgressBar] [Report] [-Force] [-WhatIf] [-Confirm] [<CommonParameters>]


More Examples

.\Remove-DuplicateItems.ps1 -Mailbox Francis -Type All -Impersonation -DeleteMode SoftDelete -Mode Quick -Verbose

Check Francis' mailbox for duplicate items in each folder, soft deleting duplicates, matching on PidTagSearchKey and using impersonation.

.\Remove-DuplicateItems.ps1 -Mailbox Philip -Retain Oldest -Type Mail -Impersonation -DeleteMode MoveToDeletedItems -Mode Full -Verbose

Check Philip's mailbox for duplicate task items in each folder and moves duplicates to the Deleted Items folder, using preset matching criteria and impersonation. When duplicates are found, the oldest is retained.

$Credentials= Get-Credential
.\Remove-DuplicateItems.ps1 -Mailbox olrik@office365tenant.com -Credentials $Credentials

Sets $Credentials variable. Then, check olrik@office365tenant.com's mailbox for duplicate items in each folder, using Credentials provided earlier.

$Credentials= Get-Credential
.\Remove-DuplicateItems.ps1 -Mailbox olrik@office365tenant.com -Server outlook.office365.com -Credentials $Credentials -IncludeFolders '#Inbox#\*','\Projects\*' -ExcludeFolders 'Keep Out' -PriorityFolders '*Important*' -MailboxWide

Remove duplicate items from specified mailbox in Office365 using fixed Server FQDN - bypassing AutoDiscover, limiting operation against the Inbox, and top Projects folder, and all of their subfolders, but excluding any folder named Keep Out. Duplicates are checked over all folders, but priority is given to folders containing the word Important, causing items in those folders to be kept over items in other folders when duplicates are found.