Parfor for Python
The one built-in to python would be multiprocessing
docs are here. I always use multiprocessing.Pool
with as many workers as processors. Then whenever I need to do a for-loop like structure I use Pool.imap
As long as the body of your function does not depend on any previous iteration then you should have near linear speed-up. This also requires that your inputs and outputs are pickle
-able but this is pretty easy to ensure for standard types.
UPDATE:Some code for your updated function just to show how easy it is:
from multiprocessing import Poolfrom itertools import productoutput = np.zeros((N,N))pool = Pool() #defaults to number of available CPU'schunksize = 20 #this may take some guessing ... take a look at the docs to decidefor ind, res in enumerate(pool.imap(Fun, product(xrange(N), xrange(N))), chunksize): output.flat[ind] = res
There are many Python frameworks for parallel computing. The one I happen to like most is IPython, but I don't know too much about any of the others. In IPython, one analogue to parfor would be client.MultiEngineClient.map()
or some of the other constructs in the documentation on quick and easy parallelism.
Jupyter Notebook
To see an example consider you want to write the equivalence of this Matlab code on in Python
matlabpool open 4parfor n=0:9 for i=1:10000 for j=1:10000 s=j*i end end nenddisp('done')
The way one may write this in python particularly in jupyter notebook. You have to create a function in the working directory (I called it FunForParFor.py) which has the following
def func(n): for i in range(10000): for j in range(10000): s=j*i print(n)
Then I go to my Jupyter notebook and write the following code
import multiprocessing import FunForParForif __name__ == '__main__': pool = multiprocessing.Pool(processes=4) pool.map(FunForParFor.func, range(10)) pool.close() pool.join() print('done')
This has worked for me! I just wanted to share it here to give you a particular example.