How to execute a PowerShell function several times in parallel? How to execute a PowerShell function several times in parallel? powershell powershell

How to execute a PowerShell function several times in parallel?


No update necessary for this. Define a script block and use Start-Job to run the script block as many times as necessary. Example:

$cmd = {  param($a, $b)  Write-Host $a $b}$foo = "foo"1..5 | ForEach-Object {  Start-Job -ScriptBlock $cmd -ArgumentList $_, $foo}

The script block takes 2 parameters $a and $b which are passed by the -ArgumentList option. In the example above, the assignments are $_$a and $foo$b. $foo is just an example for a configurable, but static parameter.

Run Get-Job | Remove-Job at some point to remove the finished jobs from the queue (or Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id } if you want to retrieve the output).


Here's a quick bogus scriptblock for the purpose of testing:

$Code = {    param ($init)    $start = Get-Date    (1..30) | % { Start-Sleep -Seconds 1; $init +=1 }    $stop = Get-Date    Write-Output "Counted from $($init - 30) until $init in $($stop - $start)."}

This scriptblock can then be passed on to Start-Job, with for example 3 parameters (10, 15, 35)

$jobs = @()(10,15,35) | % { $jobs += Start-Job -ArgumentList $_ -ScriptBlock $Code }Wait-Job -Job $jobs | Out-NullReceive-Job -Job $jobs

This creates 3 jobs, assign them to the $jobs variable, runs them in parallel and then waits for these 3 jobs to finish, and retrieves the results:

Counted from 10 until 40 in 00:00:30.0147167.Counted from 15 until 45 in 00:00:30.0057163.Counted from 35 until 65 in 00:00:30.0067163.

This did not take 90 seconds to execute, only 30.

One of the tricky parts is to provide -Argumentlist to Start-Job, and include a param() block inside the ScriptBlock. Otherwise, your values are never seen by the scriptblock.


You can use an alternative which may be faster than invoking jobs if the function is not a long running one. Max thread is 25 and I'm only invoking this function 10 times, so I expect my total runtime to be 5 seconds. You can wrap Measure-Command around the 'results=' statement to view stats.

Example:

$ScriptBlock = {    Param ( [int]$RunNumber )    Start-Sleep -Seconds 5       Return $RunNumber}$runNumbers = @(1..10)$MaxThreads = 25$runspacePool = [RunspaceFactory ]::CreateRunspacePool(1, $MaxThreads)$runspacePool.Open()$pipeLines = foreach($num in $runNumbers){    $pipeline = [powershell]::Create()    $pipeline.RunspacePool = $runspacePool    $pipeline.AddScript($ScriptBlock)    | Out-Null       $pipeline.AddArgument($num)  | Out-Null    $pipeline | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $pipeline.BeginInvoke() -PassThru }#obtain results as they come.$results =  foreach($pipeline in $pipeLines){    $pipeline.EndInvoke($pipeline.AsyncResult )}#cleanup code.$pipeLines | % { $_.Dispose()}$pipeLines = $nullif ( $runspacePool ) { $runspacePool.Close()}#your results$results