Wget with input-file and output-document Wget with input-file and output-document bash bash

Wget with input-file and output-document


Don't use cat. You can have xargs read from a file. From the man page:

       --arg-file=file       -a file              Read items from file instead of standard input.  If you use this              option, stdin remains unchanged when commands are  run.   Other‐              wise, stdin is redirected from /dev/null.


how about using a loop?

while read -r linedo   md5=$(echo "$line"|md5sum)   wget ... $line ... --output-document $md5 ......done < url-list.txt


In your question you use -P 4 which suggests you want your solution to run in parallel. GNU Parallel http://www.gnu.org/software/parallel/ may help you:

cat url-list.txt | parallel 'wget {} --output-document "`echo {}|md5sum`"'