Multiple processes sharing a single Joblib cache
Specify a common, fixed cachedir
and decorate the function that you want to cache using
from joblib import Memorymem = Memory(cachedir=cachedir)@mem.cachedef f(arguments): """do things""" pass
or simply
def g(arguments): passcached_g = mem.cache(g)
Then, even if you are working across processes, across machines, if all instances of your program have access to cachedir
, then common function calls can be cached there transparently.