Threading pool similar to the multiprocessing Pool? Threading pool similar to the multiprocessing Pool? python python

Threading pool similar to the multiprocessing Pool?


I just found out that there actually is a thread-based Pool interface in the multiprocessing module, however it is hidden somewhat and not properly documented.

It can be imported via

from multiprocessing.pool import ThreadPool

It is implemented using a dummy Process class wrapping a python thread. This thread-based Process class can be found in multiprocessing.dummy which is mentioned briefly in the docs. This dummy module supposedly provides the whole multiprocessing interface based on threads.


In Python 3 you can use concurrent.futures.ThreadPoolExecutor, i.e.:

executor = ThreadPoolExecutor(max_workers=10)a = executor.submit(my_function)

See the docs for more info and examples.


Yes, and it seems to have (more or less) the same API.

import multiprocessingdef worker(lnk):    ....    def start_process():    .........if(PROCESS):    pool = multiprocessing.Pool(processes=POOL_SIZE, initializer=start_process)else:    pool = multiprocessing.pool.ThreadPool(processes=POOL_SIZE,                                            initializer=start_process)pool.map(worker, inputs)....


matomo