Shared variable in python's multiprocessing Shared variable in python's multiprocessing python python

Shared variable in python's multiprocessing


When you use Value you get a ctypes object in shared memory that by default is synchronized using RLock. When you use Manager you get a SynManager object that controls a server process which allows object values to be manipulated by other processes. You can create multiple proxies using the same manager; there is no need to create a new manager in your loop:

manager = Manager()for i in range(5):    new_value = manager.Value('i', 0)

The Manager can be shared across computers, while Value is limited to one computer. Value will be faster (run the below code to see), so I think you should use that unless you need to support arbitrary objects or access them over a network.

import timefrom multiprocessing import Process, Manager, Valuedef foo(data, name=''):    print type(data), data.value, name    data.value += 1if __name__ == "__main__":    manager = Manager()    x = manager.Value('i', 0)    y = Value('i', 0)    for i in range(5):        Process(target=foo, args=(x, 'x')).start()        Process(target=foo, args=(y, 'y')).start()    print 'Before waiting: '    print 'x = {0}'.format(x.value)    print 'y = {0}'.format(y.value)    time.sleep(5.0)    print 'After waiting: '    print 'x = {0}'.format(x.value)    print 'y = {0}'.format(y.value)

To summarize:

  1. Use Manager to create multiple shared objects, including dicts andlists. Use Manager to share data across computers on a network.
  2. Use Value or Array when it is not necessary to share informationacross a network and the types in ctypes are sufficient for yourneeds.
  3. Value is faster than Manager.

Warning

By the way, sharing data across processes/threads should be avoided if possible. The code above will probably run as expected, but increase the time it takes to execute foo and things will get weird. Compare the above with:

def foo(data, name=''):    print type(data), data.value, name    for j in range(1000):        data.value += 1

You'll need a Lock to make this work correctly.

I am not especially knowledgable about all of this, so maybe someone else will come along and offer more insight. I figured I would contribute an answer since the question was not getting attention. Hope that helps a little.