Python Multiprocessing Lib Error (AttributeError: __exit__)
In Python 2.x and 3.0, 3.1 and 3.2, multiprocessing.Pool()
objects are not context managers. You cannot use them in a with
statement. Only in Python 3.3 and up can you use them as such. From the Python 3 multiprocessing.Pool()
documentation:
New in version 3.3: Pool objects now support the context management protocol – see Context Manager Types.
__enter__()
returns the pool object, and__exit__()
calls terminate().
For earlier Python versions, you could use contextlib.closing()
, but take into account this'll call pool.close()
, not pool.terminate()
. Terminate manually in that case:
from contextlib import closingwith closing(Pool(processes=2)) as pool: pool.map(myFunction, mylist) pool.map(myfunction2, mylist2) pool.terminate()
or create your own terminating()
context manager:
from contextlib import contextmanager@contextmanagerdef terminating(thing): try: yield thing finally: thing.terminate()with terminating(Pool(processes=2)) as pool: pool.map(myFunction, mylist) pool.map(myfunction2, mylist2)
with
statement is for object that have __enter__
and __exit__
functions, i.e. Context Manager Typesmultiprocessing.Pool
is not Context Manager Type.try do the following:
pool = Pool(processes=2)pool.map(myFunction, mylist)pool.map(myfunction2, mylist2)