Threading pool similar to the multiprocessing Pool?
I just found out that there actually is a thread-based Pool interface in the
multiprocessing module, however it is hidden somewhat and not properly documented.
It can be imported via
from multiprocessing.pool import ThreadPool
It is implemented using a dummy Process class wrapping a python thread. This thread-based Process class can be found in
multiprocessing.dummy which is mentioned briefly in the docs. This dummy module supposedly provides the whole multiprocessing interface based on threads.
Yes, and it seems to have (more or less) the same API.
import multiprocessingdef worker(lnk): .... def start_process(): .........if(PROCESS): pool = multiprocessing.Pool(processes=POOL_SIZE, initializer=start_process)else: pool = multiprocessing.pool.ThreadPool(processes=POOL_SIZE, initializer=start_process)pool.map(worker, inputs)....