Python multiprocessing.Pool的四种方法比较：: map, apply, map_async, apply_async

最新推荐文章于 2023-11-27 21:50:54 发布

原创最新推荐文章于 2023-11-27 21:50:54 发布 · 4.9k 阅读

7 ·

CC 4.0 BY-SA版权

python进阶专栏收录该内容

8 篇文章

订阅专栏

本文深入探讨了Python中multiprocessing模块的Pool类下map和apply_async方法的区别与应用场景。通过对比不同方法的特性，如是否支持多参数、是否阻塞、是否保持任务执行顺序以及是否并行执行等，为读者提供了在不同场景下选择合适方法的指导。同时，通过具体代码示例展示了如何使用这些方法来提高程序的并发处理能力。

There are four choices to mapping jobs to process. Here are the differences:

             Multi-args   Concurrence    Blocking     Ordered-results
map          no           yes            yes          yes
apply        yes          no             yes          no
map_async    no           yes            no           yes
apply_async  yes          yes            no           no

In Python 3, a new function starmap can accept multiple arguments.

Note that map and map_async are called for a list of jobs in one time, but apply and apply_async can only called for one job. However,apply_async execute a job in background therefore in parallel. See examples:

# map
results = pool.map(worker, [1, 2, 3])
 
# apply
for x, y in [[1, 1], [2, 2]]:
    results.append(pool.apply(worker, (x, y)))
 
def collect_result(result):
    results.append(result)
 
# map_async
pool.map_async(worker, jobs, callback=collect_result)
 
# apply_async
for x, y in [[1, 1], [2, 2]]:
    pool.apply_async(worker, (x, y), callback=collect_result)

记录python multiprocessing Pool的map和apply_async方法 _pool.map_async