Limit RAM usage to python program Limit RAM usage to python program python python

Limit RAM usage to python program


Ok so I've made some research and found a function to get the memory from Linux systems here: Determine free RAM in Python and I modified it a bit to get just the free memory available and set the maximum memory available as its half.

Code:

def memory_limit():    soft, hard = resource.getrlimit(resource.RLIMIT_AS)    resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard))def get_memory():    with open('/proc/meminfo', 'r') as mem:        free_memory = 0        for i in mem:            sline = i.split()            if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):                free_memory += int(sline[1])    return free_memoryif __name__ == '__main__':    memory_limit() # Limitates maximun memory usage to half    try:        main()    except MemoryError:        sys.stderr.write('\n\nERROR: Memory Exception\n')        sys.exit(1)

The line to set it to the half is resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard)) where get_memory() * 1024 / 2 sets it to the half (it's in bytes).

Hope this can help others in future with the same matter! =)


I modify the answer of @Ulises CT. Because I think to change too much original function is not so good, so I turn it to a decorator. I hope it helps.

import resourceimport platformimport sysdef memory_limit(percentage: float):    """    只在linux操作系统起作用    """    if platform.system() != "Linux":        print('Only works on linux!')        return    soft, hard = resource.getrlimit(resource.RLIMIT_AS)    resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 * percentage, hard))def get_memory():    with open('/proc/meminfo', 'r') as mem:        free_memory = 0        for i in mem:            sline = i.split()            if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):                free_memory += int(sline[1])    return free_memorydef memory(percentage=0.8):    def decorator(function):        def wrapper(*args, **kwargs):            memory_limit(percentage)            try:                function(*args, **kwargs)            except MemoryError:                mem = get_memory() / 1024 /1024                print('Remain: %.2f GB' % mem)                sys.stderr.write('\n\nERROR: Memory Exception\n')                sys.exit(1)        return wrapper    return decorator@memory(percentage=0.8)def main():    print('My memory is limited to 80%.')


due to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773

it is better to use :

if str(sline[0]) == 'MemAvailable:':            free_memory = int(sline[1])            break

the answer code provided me with 8 GB of available mem on a machine with 1TB of RAM