Limit RAM usage to python program
Ok so I've made some research and found a function to get the memory from Linux systems here: Determine free RAM in Python and I modified it a bit to get just the free memory available and set the maximum memory available as its half.
Code:
def memory_limit(): soft, hard = resource.getrlimit(resource.RLIMIT_AS) resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard))def get_memory(): with open('/proc/meminfo', 'r') as mem: free_memory = 0 for i in mem: sline = i.split() if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'): free_memory += int(sline[1]) return free_memoryif __name__ == '__main__': memory_limit() # Limitates maximun memory usage to half try: main() except MemoryError: sys.stderr.write('\n\nERROR: Memory Exception\n') sys.exit(1)
The line to set it to the half is resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard))
where get_memory() * 1024 / 2
sets it to the half (it's in bytes).
Hope this can help others in future with the same matter! =)
I modify the answer of @Ulises CT. Because I think to change too much original function is not so good, so I turn it to a decorator. I hope it helps.
import resourceimport platformimport sysdef memory_limit(percentage: float): """ 只在linux操作系统起作用 """ if platform.system() != "Linux": print('Only works on linux!') return soft, hard = resource.getrlimit(resource.RLIMIT_AS) resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 * percentage, hard))def get_memory(): with open('/proc/meminfo', 'r') as mem: free_memory = 0 for i in mem: sline = i.split() if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'): free_memory += int(sline[1]) return free_memorydef memory(percentage=0.8): def decorator(function): def wrapper(*args, **kwargs): memory_limit(percentage) try: function(*args, **kwargs) except MemoryError: mem = get_memory() / 1024 /1024 print('Remain: %.2f GB' % mem) sys.stderr.write('\n\nERROR: Memory Exception\n') sys.exit(1) return wrapper return decorator@memory(percentage=0.8)def main(): print('My memory is limited to 80%.')
it is better to use :
if str(sline[0]) == 'MemAvailable:': free_memory = int(sline[1]) break
the answer code provided me with 8 GB of available mem on a machine with 1TB of RAM