Make PowershellGet a trusted repository

After installing PowershellGet you can make us of this powerful repository to find modules and scripts you need for your personal happiness or current tasks you have to do with PowerShell.

Click here to learn how to install PowerShellGet on your computer

After you’ve installed PowershellGet and start installing modules, you always see warnings like this:

AzurePS4

As the message is already telling you (yep, sometimes it is really a good idea to read the messages on the screen šŸ˜‰ ), you can trust the repo to avoid that message in future. For those who still don’t like reading PowerShell messages and prefer reading this article, here is the command:

Set-PSRepository -Name PSGallery -InstallationPolicy Trusted

Install PowerShellGet on your computer

PowerShell is continuously evolving, that means that for some modules you don’t get an installer anymore. You have to use PowershellGet.

But how to get it? In the following table I’ve summarized the ways:

Scenario Install instructions
Windows 10
Windows Server 2016
Included in Windows Management Framework (WMF) 5.0. This is aready shipped and ready-to-go with the Windows OS
I want to upgrade to PowerShell 5
  1. Install the latest version of WMF
  2. Run the following command:
    Install-Module PowerShellGet -Force
I am running on a version of Windows with PowerShell 3 or PowerShell 4 Get the PackageManagement modules

  1. Run the following command:
    Install-Module PowerShellGet -Force

Read theĀ official articleĀ on docs.microsoft.com.

Now you can issue the following command to see how powerful this module is to find the module of your needs:

Get-Command -Module PowershellGet

Using a Mac or Linux? No problem. ReadĀ this articleĀ first to learn how to use PowerShell on your device.

Get useful output from Get-MigrationBatch

No matter if you are doing an on-premises migration or a move to Office 365 – when you issue a PowerShell command you expect useful output. So when you execute the following command:

Get-MigrationBatch

You see the following result:

Get-MigrationBatch1

What the…. useful? Where’s the information about the failure rate, etc etc etc? Especially because the ECP shows us this information? What did the developers of this cmdLet think (or not think) an administrator wants to know?

So let’s pimp this command by using simple PowerShell cmdLets.

Get-MigrationBatch | select identity,Status,TotalCount,SyncedCount,FinalizedCount,FailedCount,Notificationemails | Format-Table

And now the output is much more helpful than before:

Get-MigrationBatch2

If you are a PowerShell beginner and ask yourself now: “How can I find the exact names of the fields I am looking for?” Well here is the answer: execute this command and search for the parameters you need and start building your own output:

(Get-MigrationBatch)[0] | select *

This shows you all attributes of the first (and so also of all other) migration batches. If you have only one batch, you can simplify it a little bit:

Get-MigrationBatch | select *

If you plan to exercise more with Mirgation Batches a good point to start is the Technet help for Get-MigrationBatch

Splitting up a big CSV file to multiple smaller CSV files using PowerShell

Sometimes you have the problem that you get large Excel or CSV lists to work with but you want to split them up by a certain criteria, e.g. company. Your goal may be to be able to create multiple Exchange Hybrid migration batches.

To guide you through this task, my example data looks like this:

Firstname;LastName;Country
Max;Mustermann;Germany
Maria;Musterfrau;Germany
Igor;Smirnow;Russia
John;Dow;USA
Felix;Frisch;Germany

As you can see, it doesn’t matter how the data is sorted.

To get started you must specify some parameters to control the behaviour how PowerShell will split the file:

$GroupField = "Country"
$Delimiter = ";"
$csv = "C:\tmp\MyBigCSVWithTonsOfData.csv"
$outfolder = "C:\tmp\CSV-Files\"

What do these parameters do? It’s simple:

  • $GroupField specified which CSV column will taken as identifier for the split process.
  • $Delimiter specified the delimiter you are using in your CSV.
  • $csv is the full path to the CSV file you want to split up in multiple smaller files.
  • $outfolder is the folderpath where the generated CSV files will be stored.

After the preparation you must read the CSV and prepare the data to be splitted:

$all = Import-Csv $csv -Delimiter $Delimiter
$groupedCollection = $all | Group-Object $GroupField

You see I use the command “Group-Object” which acts like the Excel function “Filter”. With this command you can group structured data and the result are multiple groups which contain the single entries. So if your CSV input file contained 5 entries, myou group by country and 3 of the people have “Germany”, 1 has “USA” and one has “Russia” the result of these 2 lines will be a return value of 3 groups:

  • One group with 3 entries – the people with country “Germany”
  • One group with 1 entry – the person with country “USA”
  • One group with 1 entry – the person with country “Russia”

PSGroup

Having a deeper look at the return value, you can see that the variable $groupcollection is an array. You can check this by executing this command:

$groupCollection.GetType()

Working with an array is easy, you probably know it from other scripts you wrote with PowerShell. Just create a foreach-loop and iterate through the items:

foreach($group in $groupedCollection)
{
   Write-Host ("Group '" + $group.Name + "' // " + $group.Count.ToString() + " Members")
   $group.Group | ConvertTo-Csv -NoTypeInformation -Delimiter "," | Out-File ($outfolder + $group.Name + ".csv")
}

And voila, the result is that you have 3 CSV files – one for each country:

PSGroup2

Now you can continue your work with the new smaller CSV files. As you may have noticed, I’ve hard-coded the output files with delimiter “,”. Ā Of course, if you need another delimiter, feel free to adjust it to your needs.

So after all that you can copy/paste the whole tiny script at once here:

$GroupField = "Company"
$Delimiter = ";"
$csv = "S:\CSV-Files\MyBigCSVWithTonsOfData.csv"
$outfolder = "S:\CSV-Files\"

$all = Import-Csv $csv -Delimiter $Delimiter
$groupedCollection = $all | Group-Object $GroupField

foreach($group in $groupedCollection)
{
   Write-Host ("Group '" + $group.Name + "' // " + $group.Count.ToString() + " Members")
   $group.Group | ConvertTo-Csv -NoTypeInformation -Delimiter "," | Out-File ($outfolder + $group.Name + ".csv")
}

Create multiple hybrid migration batches with PowerShell

Sometimes in Exchange Online migration projects you have the demand to make not few big batches but many small ones. In the end this means you have many CSV files which you want to use for bulk creation of migration batches.

You can either create them by using the GUI (which is not really fun when you have more than 10 CSV files) or by using this tiny PowerShell command. You have to replace the highlighted values with your own ones:

Get-ChildItem *.csv  | Foreach-Object{ New-MigrationBatch -Name ($_.Name -replace ".csv","") -TargetDeliveryDomain "TENANTNAME.mail.onmicrosoft.com" -AutoStart -AllowUnknownColumnsInCsv $true -NotificationEmails "" -CSVData ([System.IO.File]::ReadAllBytes( $_.FullName)) -BadItemLimit 99999 -LargeItemLimit 99999 -AllowIncrementalSyncs $true -SourceEndpoint "NAME OF YOUR HYBRID ENDPOINT"}

 

This command searches for all CSV files in the current folder and creates a migration batch for each CSV file with the following attributes:

  • The name of the batch will be the file name of the CSV file without the file extension
  • The batch will start automatically but has to be completed manually
  • The notification emails will be sent to the email you provide here
  • The CSV files may have any column, but the column “EmailAddress” must be present
  • The batches will perform incremental syncs
  • the LargeItemLimit and BadItemLimit are very high to ensure the mailboxes will not skip because of items that can’t be migrated
  • The hybrid endpoint of your organization will be taken for the move

Of course you may adjust this command depending on your needs, e.g. Auto Completion or a lower BadItemLimit.

In my case there was a limit of 100 migration batches. The Exchange Online Service Description doesn’t mention this limit, but be aware of the fact that this may hit you as well.

Happy migrating šŸ™‚