How do I execute multiple commands in parallel on an array of parameters with bash, and fail if at least one of them failed How do I execute multiple commands in parallel on an array of parameters with bash, and fail if at least one of them failed bash bash

How do I execute multiple commands in parallel on an array of parameters with bash, and fail if at least one of them failed


You are so close to getting the syntax of GNU Parallel correct:

COMMAND=echoPARAMETERS=(first-parameter second-parameter third-parameter)parallel -j 4 -k --retries 2 "$COMMAND" {} ::: "${PARAMETERS[@]}" ||  echo $? commands failed. More than 99 if $? = 100

Or if you really insist on doing the retrying yourself:

PARAMETERS=(first-parameter second-parameter third-parameter)export -f run_and_retryparallel -j 4 -k run_and_retry {} ::: "${PARAMETERS[@]}" ||  echo One or more commands failed


I need to know if at least one of the executions failed (returned non-zero)

From posix xargs:

EXIT STATUS

1-125
A command line meeting the specified requirements could not be assembled, one or more of the invocations of utility returned a non-zero exit status, or some other error occurred.

The man xargs seems a bit different:

EXIT STATUS

123 if any invocation of the command exited with status 1-125

But I would check the return status of the command and return a predefined number (ex. 1) from the function to handle that.

parameters=(1 2 3 fail)func() {     COMMAND=sleep    # I guess OP intends to try running COMMAND twice    if ! "$COMMAND" 0."$1" && ! "$COMMAND" 0."$1"; then        return 1    fi}export -f funcif printf "%s\0" "${parameters[@]}" | xargs -0 -P4 -n1 -t -- bash -c 'func $1' -- ; then   echo "Success!"else   echo "Error!"fi

Live version available at tutorialspoint.

Well, we can even count the number of childs manually, and it get's pretty simple with wait -n. From stackoverflow - WAIT for “1 of many process” to finish:

bash 4.3 added a -n flag to the built-in wait command, which causes the script to wait for the next child to complete.

So we can just:

cnt=0failed=falsefor i in "${parameters[@]}"; do    ( func "$i" ) &    if (( cnt < 4 )); then        cnt=$((cnt+1))    else        # handle more then 4 processes        if ! wait -n; then           failed=true        fi    fidone# handle still running processes after all have been forkedfor i in $(seq $cnt); do    if ! wait -n; then        failed=true    fidoneif "$failed"; then    echo "One of the jobs failed!"fi