Move top 1000 lines from text file to a new file using Unix shell commands
Out of curiosity, I found a box with a GNU version of sed
(v4.1.5) and tested the (uncached) performance of two approaches suggested so far, using an 11M line text file:
$ wc -l input11771722 input$ time head -1000 input > output; time tail -n +1000 input > input.tmp; time cp input.tmp input; time rm input.tmpreal 0m1.165suser 0m0.030ssys 0m1.130sreal 0m1.256suser 0m0.062ssys 0m1.162sreal 0m4.433suser 0m0.033ssys 0m1.282sreal 0m6.897suser 0m0.000ssys 0m0.159s$ time head -1000 input > output && time sed -i '1,+999d' inputreal 0m0.121suser 0m0.000ssys 0m0.121sreal 0m26.944suser 0m0.227ssys 0m26.624s
This is the Linux I was working with:
$ uname -aLinux hostname 2.6.18-128.1.1.el5 #1 SMP Mon Jan 26 13:58:24 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
For this test, at least, it looks like sed
is slower than the tail
approach (27 sec vs ~14 sec).