How to use parallel execution in a shell script? How to use parallel execution in a shell script? unix unix

How to use parallel execution in a shell script?


Convert this into a Makefile with proper dependencies. Then you can use make -j to have Make run everything possible in parallel.

Note that all the indents in a Makefile must be TABs. TAB shows Make where the commands to run are.

Also note that this Makefile is now using GNU Make extensions (the wildcard and subst functions).

It might look like this:

export PATH := .:${PATH}FILES=$(wildcard file*)RFILES=$(subst file,r,${FILES})final: combine ${RFILES}    combine ${RFILES} final    rm ${RFILES}ex: example.ccombine: combine.cr%: file% ex    ex $< $@


In bash I would do;

ex file1 r1  &ex file2 r2  &ex file3 r3  &wait... continue with script...

and spawn them out to run in parallel. You can check out this SO thread for another example.


#!/bin/bashgcc example.c -o exgcc combine.c -o combine# Call 'ex' 3 times in "parallel"for i in {1..3}; do  ex file${i} r${i} &done#Wait for all background processes to finishwait# Combine & removecombine r1 r2 r3 finalrm r1 r2 r3

I slightly altered the code to use brace expansion {1..3} rather than hard code the numbers since I just realized you said there are many more files than just 3. Brace expansion makes scaling to larger numbers trivial by replacing the '3' inside the braces to whatever number you need.