Mulitprocess Pools with different functions

To pass different functions, you can simply call map_async multiple times.

Here is an example to illustrate that,

from multiprocessing import Poolfrom time import sleepdef square(x):    return x * xdef cube(y):    return y * y * ypool = Pool(processes=20)result_squares = pool.map_async(f, range(10))result_cubes = pool.map_async(g, range(10))

The result will be:

>>> print result_squares.get(timeout=1)[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]>>> print result_cubes.get(timeout=1)[0, 1, 8, 27, 64, 125, 216, 343, 512, 729]

They will not run in parallel.See following code:

def updater1(q,i):        print "UPDATER 1:", i    returndef updater2(q,i):        print "UPDATER2:", i    returnif __name__=='__main__':    a = range(10)    b=["abc","def","ghi","jkl","mno","pqr","vas","dqfq","grea","qfwqa","qwfsa","qdqs"]    pool = multiprocessing.Pool()    func1 = partial(updater1,q)    func2 = partial(updater2,q)    pool.map_async(func1, a)    pool.map_async(func2, b)    pool.close()    pool.join()

The above code yields the following printout:

UPDATER 1: 1UPDATER 1: 0UPDATER 1: 2UPDATER 1: 3UPDATER 1: 4UPDATER 1: 5UPDATER 1: 6UPDATER 1: 7UPDATER 1: 8UPDATER 1: 9UPDATER2: abcUPDATER2: defUPDATER2: ghiUPDATER2: jklUPDATER2: mnoUPDATER2: pqrUPDATER2: vasUPDATER2: dqfqUPDATER2: greaUPDATER2: qfwqaUPDATER2: qwfsaUPDATER2: qdqs

You can use map or some lambda function (edit: actually you can't use a lambda function). You can use a simple map function:

def smap(f, *args):    return f(*args)pool = multiprocessing.Pool(processes=30)res=pool.map(smap, function_list, args_list1, args_list2,...)

The normal map function takes iterables as inputs, which is inconvenient.