gnu parallel: output each job to a different file
It turns out I needed to quote out the redirect, because it was being processed outside of parallel
:
seq 10 | parallel awk \''{...}'\' file{}.txt ">" file{}.out
Another way is to introduce the entire parallel command inside double quotes:
seq 10 | parallel " awk command > file{}.out "
Although, sometimes is useful redirect the output to file and also to stdout. You can achieve that using tee
. In this case, the command to be used could be:
seq 10 | parallel " awk command | tee file{}.out "
As I was parallelizing a script that has no pipes support, the solutions here were not valid for me, so I opened a new thread here that solves the issue.
I hope it will help some others arriving here.