Read list of files on unix and run command Read list of files on unix and run command unix unix

Read list of files on unix and run command


There are a couple of ways to do it. Since the names are 'one per line' in the data file, we can assume there are no newlines in the file names.

for loop

for file in $(<list.txt)do    sickle pe -f "${file}_file1.fastq" -r "${file}_file2.fastq" -t sangerdone

while loop with read

while read filedo    sickle pe -f "${file}_file1.fastq" -r "${file}_file2.fastq" -t sangerdone < list.txt

The for loop only works if there are no blanks in the names (nor other white-space characters such as tabs). The while loop is clean as long as you don't have newlines in the names, though using while read -r file would give you even better protection against the unexpected. The double quotes around the file name in the for loop are decorative (but harmless) because the file names cannot contain blanks, but those in the while loop prevent file names containing blanks from being split when they should not be split. It's often a good idea to quote variables every time you use them, though it strictly only matters when the variable might contain blanks but you don't want the value split up.

I've had to guess what names should be passed to the sickle command since your question is not clear about it — I'm 99% sure I've guessed wrong, but it matches the different suffixes in your sample command assuming the base name of file is input. I've omitted the trailing backslash; it is the 'escape' character and it is not clear what you really want there.


Use a Bash For-Loop

Bash has a very reasonable for-loop as one of its looping constructs. You can replace the echo command below with whatever custom command you want. For example:

for file in name1 name2 name3; do  echo "${file}_R1" "${file}_R2"done

The idea is that the loop assigns each filename to the file variable, then you append the _R1 and _R2 suffixes to them. Note that quoting may be important, and does no harm if it isn't needed, so you ought to use it as a defensive programming measure.

Use xargs for Argument Lists

If you want to read from a file instead of using the for-loop directly, you can use Bash's read builtin, but xargs is often more portable across shells. For example, the following uses flags available in the version of xargs from GNU findutils to read in arguments from a file and then append a suffix to each of them:

$ xargs --arg-file=list.txt --max-args=1 -I{} /bin/echo "{}_R1" "{}_R2"name1_R1 name1_R2name2_R1 name2_R2name3_R1 name3_R2

Again, you can replace "echo" with the command line of your choice.


Use a while loop with read:

while read fn; do    <command> "${fn}_R1" "${fn}_R2"done < list.txt