multiprocessing - Passing multiple parameters to pool.map() function in Python -
this question has answer here:
i need way use function within pool.map() accepts more 1 parameter. per understanding, target function of pool.map() can have 1 iterable parameter there way can pass other parameters in well? in case, need pass in few configuration variables, lock() , logging information target function.
i have tried research , think may able use partial functions work? don't understand how these work. appreciated! here simple example of want do:
def target(items, lock): item in items: # cool stuff if (... condition here ...): lock.acquire() # write stdout or logfile, etc. lock.release() def main(): iterable = [1, 2, 3, 4, 5] pool = multiprocessing.pool() pool.map(target(pass params here), iterable) pool.close() pool.join()
you can use functools.partial (as suspected):
from functools import partial def target(lock, iterable_item): item in iterable_item: # cool stuff if (... condition here ...): lock.acquire() # write stdout or logfile, etc. lock.release() def main(): iterable = [1, 2, 3, 4, 5] pool = multiprocessing.pool() l = multiprocessing.lock() func = partial(target, l) pool.map(func, iterable) pool.close() pool.join() example:
def f(a, b, c): print("{} {} {}".format(a, b, c)) def main(): iterable = [1, 2, 3, 4, 5] pool = multiprocessing.pool() = "hi" b = "there" func = partial(f, a, b) pool.map(func, iterable) pool.close() pool.join() if __name__ == "__main__": main() output:
hi there 1 hi there 2 hi there 3 hi there 4 hi there 5
Comments
Post a Comment