Pass multiple parameters to concurrent.futures.Executor.map?
One argument that is repeated, one argument in c
from itertools import repeatfor result in executor.map(f, repeat(a), c): pass
Need to unpack items of c
, and can unpack c
from itertools import izipfor result in executor.map(f, *izip(*c)): pass
Need to unpack items of c
, can't unpack c
- Change
f
to take a single argument and unpack the argument in the function. If each item in
c
has a variable number of members, or you're callingf
only a few times:executor.map(lambda args, f=f: f(*args), c)
It defines a new function that unpacks each item from
c
and callsf
. Using a default argument forf
in thelambda
makesf
local inside thelambda
and so reduces lookup time.If you've got a fixed number of arguments, and you need to call
f
a lot of times:from collections import dequedef itemtee(iterable, n=2): def gen(it = iter(iterable), items = deque(), next = next): popleft = items.popleft extend = items.extend while True: if not items: extend(next(it)) yield popleft() return [gen()] * nexecutor.map(f, *itemtee(c, n))
Where n
is the number of arguments to f
. This is adapted from itertools.tee
.
You need to remove the *
on the map
call:
args = ((a, b) for b in c)for result in executor.map(f, args): pass
This will call f
, len(args)
times, where f
should accept one parameter.
If you want f
to accept two parameters you can use a lambda call like:
args = ((a, b) for b in c)for result in executor.map(lambda p: f(*p), args): # (*p) does the unpacking part pass
You can use currying to create new function via partial method in Python
from concurrent.futures import ThreadPoolExecutorfrom functools import partialdef some_func(param1, param2): # some code# currying some_func with 'a' argument is repeatedfunc = partial(some_func, a)with ThreadPoolExecutor() as executor: executor.map(func, list_of_args): ...
If you need to pass more than one the same parameters you can pass them to partial method
func = partial(some_func, a, b, c)