Python generator to read large CSV file
You can have a generator, that reads lines from two different csv readers and yield their lines as pairs of arrays. The code for that is:
import csvimport numpy as npdef getData(filename1, filename2): with open(filename1, "rb") as csv1, open(filename2, "rb") as csv2: reader1 = csv.reader(csv1) reader2 = csv.reader(csv2) for row1, row2 in zip(reader1, reader2): yield (np.array(row1, dtype=np.float), np.array(row2, dtype=np.float)) # This will give arrays of floats, for other types change dtypefor tup in getData("file1", "file2"): print(tup)