Unix:merge multiple CSV files with same header by keeping the header of the first file Unix:merge multiple CSV files with same header by keeping the header of the first file unix unix

Unix:merge multiple CSV files with same header by keeping the header of the first file


awk 'FNR==1 && NR!=1{next;}{print}' *.csv

tested on solaris unix:

> cat file1.csvId,city,name ,location1,NA,JACK,CA>> cat file2.csvID,city,name,location2,NY,JERRY,NY>> nawk 'FNR==1 && NR!=1{next;}{print}' *.csvId,city,name ,location1,NA,JACK,CA2,NY,JERRY,NY> 

Explanation given by kevin-d:

FNR is the number of lines (records) read so far in the current file. NR is the number of lines read overall. So the condition 'FNR==1 && NR!=1{next;}' says, "Skip this line if it's the first line of the current file, and at least 1 line has been read overall." This has the effect of printing the CSV header of the first file while skipping it in the rest.

Link for the difference between and


If Perl is an option:

perl -ne 'print if $. > 1 or ! $h; $h=1; close ARGV if eof' *.csv > master.csv

$. is the line number.
It is NOT reset automatically between files, so close ARGV if eof is needed.
$h records if the header has already been printed.


<?phpini_set('auto_detect_line_endings', true);$dir = "include/*.csv";$returnVal = array();foreach (glob($dir) as $file) {    $header = null;    $file = fopen($file, 'r') or die('Unable to open file!');    while(($row = fgetcsv($file)) !== false){        if($header === null){            $header = $row;            continue;        }        $newRow = array();        for($i = 0; $i<count($row); $i++){            $newRow[] = $row[$i];           }        if($newRow[0] == null)        break;        else        $returnVal[] = $newRow;    }    fclose($file);}//var_dump($returnVal);$output = fopen("file.csv",'w') or die("Can't open output");fputcsv($output, array('Date','close','open'));foreach($returnVal as $product) {    fputcsv($output, $product);}

fclose($output) or die("Can't close php://output"); ?>