Why Python does not see all the rows in a file?
You most likely have a file with one or more DOS EOF (CTRL-Z) characters in it, ASCII codepoint 0x1A. When Windows opens a file in text mode, it'll still honour the old DOS semantics and end a file whenever it reads that character. See Line reading chokes on 0x1A.
Only by opening a file in binary mode can you bypass this behaviour. To do so and still count lines, you have two options:
read in chunks, then count the number of line separators in each chunk:
def bufcount(filename, linesep=os.linesep, buf_size=2 ** 15): lines = 0 with open(filename, 'rb') as f: last = '' for buf in iter(f.read, ''): lines += buf.count(linesep) if last and last + buf[0] == linesep: # count line separators straddling a boundary lines += 1 if len(linesep) > 1: last = buf[-1] return lines
Take into account that on Windows
os.linesep
is set to\r\n
, adjust as needed for your file; in binary mode line separators are not translated to\n
.Use
io.open()
; theio
set of file objects open the file in binary mode always, then do the translations themselves:import iowith io.open(filename) as f: lines = sum(1 for line in f)