Append to the top of a large file: bash

Try

cat file_with_new_lines file > newfile

I did some benchmarking to compare using sed with in-place edit (as suggested here) to cat (as suggested here).

~3GB bigfile filled with dots:

$ head -n3 bigfile................................................................................................................................................................................................................................................$ du -b bigfile3025635308      bigfile

File newlines with two lines to insert on top of bigfile:

$ cat newlinessome datasome other data$ du -b newlines26      newlines

Benchmark results using dumbbench v0.08:

cat:

$ dumbbench -- sh -c "cat newlines bigfile > bigfile.new"cmd: Ran 21 iterations (0 outliers).cmd: Rounded run time per iteration: 2.2107e+01 +/- 5.9e-02 (0.3%)

sed with redirection:

$ dumbbench -- sh -c "sed '1i some data\nsome other data' bigfile > bigfile.new"cmd: Ran 23 iterations (3 outliers).cmd: Rounded run time per iteration: 2.4714e+01 +/- 5.3e-02 (0.2%)

sed with in-place edit:

$ dumbbench -- sh -c "sed -i '1i some data\nsome other data' bigfile"cmd: Ran 27 iterations (7 outliers).cmd: Rounded run time per iteration: 4.464e+01 +/- 1.9e-01 (0.4%)

So sed seems to be way slower (80.6%) when doing in-place edit on large files, probably due to moving the intermediary temp file to the location of the original file afterwards. Using I/O redirection sed is only 11.8% slower than cat.

Based on these results I would use cat as suggested in this answer.

Try doing this :

using sed :

sed -i '1i NewLine' file

Or using ed :

ed -s file <<EOF1iNewLine.wqEOF