How to loop through file names returned by find?
TL;DR: If you're just here for the most correct answer, you probably want my personal preference (see the bottom of this post):
# execute `process` once for each filefind . -name '*.txt' -exec process {} \;
If you have time, read through the rest to see several different ways and the problems with most of them.
The full answer:
The best way depends on what you want to do, but here are a few options. As long as no file or folder in the subtree has whitespace in its name, you can just loop over the files:
for i in $x; do # Not recommended, will break on whitespace process "$i"done
Marginally better, cut out the temporary variable x
:
for i in $(find -name \*.txt); do # Not recommended, will break on whitespace process "$i"done
It is much better to glob when you can. White-space safe, for files in the current directory:
for i in *.txt; do # Whitespace-safe but not recursive. process "$i"done
By enabling the globstar
option, you can glob all matching files in this directory and all subdirectories:
# Make sure globstar is enabledshopt -s globstarfor i in **/*.txt; do # Whitespace-safe and recursive process "$i"done
In some cases, e.g. if the file names are already in a file, you may need to use read
:
# IFS= makes sure it doesn't trim leading and trailing whitespace# -r prevents interpretation of \ escapes.while IFS= read -r line; do # Whitespace-safe EXCEPT newlines process "$line"done < filename
read
can be used safely in combination with find
by setting the delimiter appropriately:
find . -name '*.txt' -print0 | while IFS= read -r -d '' line; do process "$line" done
For more complex searches, you will probably want to use find
, either with its -exec
option or with -print0 | xargs -0
:
# execute `process` once for each filefind . -name \*.txt -exec process {} \;# execute `process` once with all the files as arguments*:find . -name \*.txt -exec process {} +# using xargs*find . -name \*.txt -print0 | xargs -0 process# using xargs with arguments after each filename (implies one run per filename)find . -name \*.txt -print0 | xargs -0 -I{} process {} argument
find
can also cd into each file's directory before running a command by using -execdir
instead of -exec
, and can be made interactive (prompt before running the command for each file) using -ok
instead of -exec
(or -okdir
instead of -execdir
).
*: Technically, both find
and xargs
(by default) will run the command with as many arguments as they can fit on the command line, as many times as it takes to get through all the files. In practice, unless you have a very large number of files it won't matter, and if you exceed the length but need them all on the same command line, you're SOL find a different way.
What ever you do, don't use a for
loop:
# Don't do thisfor file in $(find . -name "*.txt")do …code using "$file"done
Three reasons:
- For the for loop to even start, the
find
must run to completion. - If a file name has any whitespace (including space, tab or newline) in it, it will be treated as two separate names.
- Although now unlikely, you can overrun your command line buffer. Imagine if your command line buffer holds 32KB, and your
for
loop returns 40KB of text. That last 8KB will be dropped right off yourfor
loop and you'll never know it.
Always use a while read
construct:
find . -name "*.txt" -print0 | while read -d $'\0' filedo …code using "$file"done
The loop will execute while the find
command is executing. Plus, this command will work even if a file name is returned with whitespace in it. And, you won't overflow your command line buffer.
The -print0
will use the NULL as a file separator instead of a newline and the -d $'\0'
will use NULL as the separator while reading.
find . -name "*.txt"|while read fname; do echo "$fname"done
Note: this method and the (second) method shown by bmargulies are safe to use with white space in the file/folder names.
In order to also have the - somewhat exotic - case of newlines in the file/folder names covered, you will have to resort to the -exec
predicate of find
like this:
find . -name '*.txt' -exec echo "{}" \;
The {}
is the placeholder for the found item and the \;
is used to terminate the -exec
predicate.
And for the sake of completeness let me add another variant - you gotta love the *nix ways for their versatility:
find . -name '*.txt' -print0|xargs -0 -n 1 echo
This would separate the printed items with a \0
character that isn't allowed in any of the file systems in file or folder names, to my knowledge, and therefore should cover all bases. xargs
picks them up one by one then ...