Is it OK to use the same input file as output of a piped command? Is it OK to use the same input file as output of a piped command? unix unix

Is it OK to use the same input file as output of a piped command?


linux bash unix shell pipe

No, it is not ok. All commands in a pipeline execute at the same time, and the shell prepares redirections before executing the commands. So, it is likely that the command will overwrite the file before cat reads it.

You need sponge(1) from moreutils.

linux bash unix shell pipe

You can also use something like this (not recommended, use explicit temp files in production code):

{ rm file && your_command > file; } < file

linux bash unix shell pipe

Not only should you NOT write your output to your input, but also you should avoid looping your output back to your input.

When dealing with big files, I tried 

    cat *allfastq30 > Sample_All_allfastq30

and it generated error messages:

    cat: Sample_All_allfastq30: input file is output file

Recent Posts
How can I color dots in a xy scatterplot according to column value?
How to update a claim in ASP.NET Identity?
What does {0} mean when initializing an object?
Accessing members of items in a JSONArray with Java
How to log SQL statements in Spring Boot?
Powershell Get-WebSite name parameter is ignored
How to detect scroll to bottom of html element
Java synchronized method
How to test controllers with CodeIgniter?
Detect Visual Composer
Matplotlib: Specify format of floats for tick labels
Rails join a list of strings with commas and "and" before the last