10.6.8 进程池
ProcessPoolExecutor的工作与ThreadPoolExecutor类似,不过使用进程而不是线程。这种方法允许CPU密集的操作使用一个单独的CPU,而不会因为Cpython解释器的全局解释器锁而被阻塞。
from concurrent import futures
import os
def task(n):
return (n,os.getpid())
ex = futures.ProcessPoolExecutor(max_workers=2)
results = ex.map(task,range(5,0,-1))
for n,pid in results:
print('ran task {} in process {}'.format(n,pid))
与线程池一样,要重用各个工作进程以执行多个任务。
运行结果:
如果某个工作进程出了问题,导致它意外退出,则认为ProcessPoolExecutor"中断",不会再调度任务。
from concurrent import futures
import os
import signal
with futures.ProcessPoolExecutor(max_workers=2) as ex:
print('getting the pid for one worker')
f1 = ex.submit(os.getpid)
pid1 = f1.result()
print('killing process {}'.format(pid1))
os.kill(pid1,signal.SIGHUP)
print('submitting another task')
f2 = ex.submit(os.getpid)
try:
pid2 = f2.result()
except futures.process.BrokenProcessPool as e:
print('could not start new tasks: {}'.format(e))
BrokenProcessPool异常实际上是在处理结果时抛出的,而不是在提交新任务时抛出。
运行结果: