Allocating largest buffer without using swap Allocating largest buffer without using swap linux linux

Allocating largest buffer without using swap


Well, under linux you can use mlock()/mlockall() to keep an adress range in physical memory and prevent it from being swapped out. The process using mlock needs a couple of privileges to do so, "man mlock" has the details. I am not sure about the maximum mlock'able block (it might differ from what seems to be "free"), so probably a binary search could help (lock a range, if that fails reduce the size of the area etc..)

On the other hand, 110MB/s is not really a problem for a Solid-State-Drive. A 60GB SSD with 280MB/s write speed costs about $200 on the corner. Just copy the sensor data into a small write buffer and stream that to the SSD.


If the computer system is dedicated to receiving data from your sensor, you can simply disable swap. Then allocate as big buffer as you can, leaving enough memory in the system only for essential tools.


If you malloc the needed amount of memory and write to it at that speed, you'll still get a performance hit due to all the page faults (i.e. mapping each page of virtual memory to physical memory, which also may include swapping out memory of other processes).

In order to avoid that, you could memset the entire allocated buffer to 0 before you start reading from the sensor, so that all the needed virtual memory is mapped to physical memory.

If you only use the available physical memory, you should suffer no swapping at all. Using more would cause memory of other processes to be swapped to the disk - if these processes are idle, it shouldn't pose any problem. If they're active (i.e. using their memory once in a while), some swapping would occur - probably in a much lower rate than the hard-drive bandwidth. The more memory you use, more active processes' memory would be swapped out, and more HD activity would occur - at this point the maximal amount of memory you could use with decent performance is pretty much a result of trial and error.

By using more than the physical memory available, you'll definitely cause swapping at the rate of memory writes, and there's no way to avoid that.