How to execute a PowerShell function several times in parallel?
No update necessary for this. Define a script block and use Start-Job
to run the script block as many times as necessary. Example:
$cmd = { param($a, $b) Write-Host $a $b}$foo = "foo"1..5 | ForEach-Object { Start-Job -ScriptBlock $cmd -ArgumentList $_, $foo}
The script block takes 2 parameters $a
and $b
which are passed by the -ArgumentList
option. In the example above, the assignments are $_
→ $a
and $foo
→ $b
. $foo
is just an example for a configurable, but static parameter.
Run Get-Job | Remove-Job
at some point to remove the finished jobs from the queue (or Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id }
if you want to retrieve the output).
Here's a quick bogus scriptblock for the purpose of testing:
$Code = { param ($init) $start = Get-Date (1..30) | % { Start-Sleep -Seconds 1; $init +=1 } $stop = Get-Date Write-Output "Counted from $($init - 30) until $init in $($stop - $start)."}
This scriptblock can then be passed on to Start-Job
, with for example 3 parameters (10, 15, 35)
$jobs = @()(10,15,35) | % { $jobs += Start-Job -ArgumentList $_ -ScriptBlock $Code }Wait-Job -Job $jobs | Out-NullReceive-Job -Job $jobs
This creates 3 jobs, assign them to the $jobs
variable, runs them in parallel and then waits for these 3 jobs to finish, and retrieves the results:
Counted from 10 until 40 in 00:00:30.0147167.Counted from 15 until 45 in 00:00:30.0057163.Counted from 35 until 65 in 00:00:30.0067163.
This did not take 90 seconds to execute, only 30.
One of the tricky parts is to provide -Argumentlist
to Start-Job
, and include a param()
block inside the ScriptBlock. Otherwise, your values are never seen by the scriptblock.
You can use an alternative which may be faster than invoking jobs if the function is not a long running one. Max thread is 25 and I'm only invoking this function 10 times, so I expect my total runtime to be 5 seconds. You can wrap Measure-Command around the 'results=' statement to view stats.
Example:
$ScriptBlock = { Param ( [int]$RunNumber ) Start-Sleep -Seconds 5 Return $RunNumber}$runNumbers = @(1..10)$MaxThreads = 25$runspacePool = [RunspaceFactory ]::CreateRunspacePool(1, $MaxThreads)$runspacePool.Open()$pipeLines = foreach($num in $runNumbers){ $pipeline = [powershell]::Create() $pipeline.RunspacePool = $runspacePool $pipeline.AddScript($ScriptBlock) | Out-Null $pipeline.AddArgument($num) | Out-Null $pipeline | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $pipeline.BeginInvoke() -PassThru }#obtain results as they come.$results = foreach($pipeline in $pipeLines){ $pipeline.EndInvoke($pipeline.AsyncResult )}#cleanup code.$pipeLines | % { $_.Dispose()}$pipeLines = $nullif ( $runspacePool ) { $runspacePool.Close()}#your results$results