Performance drop with fputs after writing more than 2,5GB. Why? Performance drop with fputs after writing more than 2,5GB. Why? windows windows

Performance drop with fputs after writing more than 2,5GB. Why?


The main reason for all this is a Windows disk cache. Then your program eats all RAM for it, then swapping begins, and thus, slowdowns. To fight these you need to:

1) Open file in commit mode using c flag:

FILE* fp1 = fopen("D:\\largeTestFile.txt", "wc");

2) Periodically write buffer to disk using flush function:

if(i%1000000 == 0){    // write content to disk    fflush(fp1);    clock_t ms = clock()-last;    printf("Writting of %i Lines took %i ms\n", i, ms);    last = clock();}

This way you will use reasonable amount of disk cache. Speed will be basically limited by the speed of your hard drive.