One of the most valuable things I have picked up about PowerShell has been how to run parallel tasks pre-PowerShell 7. PowerShell 7 offers ForEach-Object Parallel which is fantastic, but when you are dealing with clients, you can’t always rely on them having or wanting to update to PowerShell 7.
This method still works with PowerShell 7, so you can’t go wrong here! This template will get you started in running your code in parallel. Keep in mind, that script blocks in PowerShell are not aware of variables in the script, you have to pass them to the script block as arguments & parameters. You can use this to run parallel processes, tasks, whole scripts, etc! I have set the value to 2 parallel jobs at most by default. As soon as one of the parallel processes is complete, it will start the next item in the queue, always keeping two jobs or “threads” (these are not actual threads, just a similar concept).
# Array of items to later add to queue
$array_queue_items = @("item-1", "item-2", "item-3", "item-4")
# How many parallel jobs to have running at once
$parallel = 2
# Function to run jobs
function RunJobFromQueue
{
if ( $queue.Count -gt 0)
{
$scriptBlock
{
#Code to run in parallel here
}
}
# The background jobs will be labeled with this string. Helpful for troubleshooting and general identification of this script
$Identifier = "parallel-script-$(New-Guid)"
# Assign the Start-Job command and all arguments to the $job variable
$job = Start-Job -Name $Identifier -ScriptBlock $scriptBlock1 # -ArgumentList $arg1, $arg2, $arg3 if you need arguments
# Magic. Resgisters and unregisters events to keep the queue progressing
Register-ObjectEvent -InputObject $job -EventName StateChanged -SourceIdentifier $Identifier -Action { RunJobFromQueue; Unregister-Event $eventsubscriber.SourceIdentifier; Remove-Job $eventsubscriber.SourceIdentifier } | Out-Null
}
}
# Create a synchronized queue
$queue = [System.Collections.Queue]::Synchronized( (New-Object System.Collections.Queue) )
# Add each item in the $array_queue_items to the queue
foreach ($item in $array_queue_items)
{
$queue.Enqueue($item)
}
# Controls how many parallel jobs to kick off. If $parallel is 2, it will process 2 queue items in succession. When one finishes, the next will automatically start
for ( $i = 0; $i -lt $Parallel; $i++ )
{
RunJobFromQueue
}Also, remember that each new job will start in the default PowerShell location C:\Users\<username>. You may have to pass the directory you want your jobs to run in (say for logging) as a parameter & argument.
