Python: Something like `map` that works on threads [closed]

There is a map method in multiprocessing.Pool. That does multiple processes.

And if multiple processes aren't your dish, you can use multiprocessing.dummy which uses threads.

import urllibimport multiprocessing.dummyp = multiprocessing.dummy.Pool(5)def f(post):    return urllib.urlopen('http://stackoverflow.com/questions/%u' % post)print p.map(f, range(3329361, 3329361 + 5))

Someone recommended I use the futures package for this. I tried it and it seems to be working.

http://pypi.python.org/pypi/futures

Here's an example:

"Download many URLs in parallel."import functoolsimport urllib.requestimport futuresURLS = ['http://www.foxnews.com/',        'http://www.cnn.com/',        'http://europe.wsj.com/',        'http://www.bbc.co.uk/',        'http://some-made-up-domain.com/']def load_url(url, timeout):    return urllib.request.urlopen(url, timeout=timeout).read()with futures.ThreadPoolExecutor(50) as executor:   future_list = executor.run_to_futures(           [functools.partial(load_url, url, 30) for url in URLS])

Here is my implementation of threaded map:

from threading import Threadfrom queue import Queuedef thread_map(f, iterable, pool=None):    """    Just like [f(x) for x in iterable] but each f(x) in a separate thread.    :param f: f    :param iterable: iterable    :param pool: thread pool, infinite by default    :return: list if results    """    res = {}    if pool is None:        def target(arg, num):            try:                res[num] = f(arg)            except:                res[num] = sys.exc_info()        threads = [Thread(target=target, args=[arg, i]) for i, arg in enumerate(iterable)]    else:        class WorkerThread(Thread):            def run(self):                while True:                    try:                        num, arg = queue.get(block=False)                        try:                            res[num] = f(arg)                        except:                            res[num] = sys.exc_info()                    except Empty:                        break        queue = Queue()        for i, arg in enumerate(iterable):            queue.put((i, arg))        threads = [WorkerThread() for _ in range(pool)]    [t.start() for t in threads]    [t.join() for t in threads]    return [res[i] for i in range(len(res))]