Python-4.26 进程池线程池_python 进程池中调用线程池-优快云博客

本文通过示例介绍了Python中concurrent.futures模块的使用，包括ThreadPoolExecutor和ProcessPoolExecutor，用于实现线程池和进程池的异步任务提交。示例展示了如何提交任务、设置回调函数以及关闭执行器。同时，还演示了如何结合requests库进行异步HTTP请求处理。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

concurrent.futures模块提供了高度封装的异步调用接口
ThreadPoolExecutor：线程池，提供异步调用
ProcessPoolExecutor: 进程池，提供异步调用

submit(fn, *args, **kwargs)异步提交任务
shutdown(wait=True) 相当于进程池的pool.close()+pool.join()操作

import os
import random
import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def task(name):
    print('name:%s pid:%s run' % (name, os.getpid()))
    time.sleep(random.randint(1, 3))


if __name__ == '__main__':
    pool = ProcessPoolExecutor(4)
    # pool = ThreadPoolExecutor(5)

    for i in range(10):
        pool.submit(task, 'winnie%s' % i)

    pool.shutdown(wait=True)

    print('主')

import os
import random
import time
from concurrent.futures import ThreadPoolExecutor
from threading import currentThread


def task():
    print('name:%s pid:%s run' % (currentThread().getName(), os.getpid()))
    time.sleep(random.randint(1, 3))


if __name__ == '__main__':
    pool = ThreadPoolExecutor(5)

    for i in range(10):
        pool.submit(task, )

    pool.shutdown(wait=True)

    print('主')

使用异步调用机制：提交完任务后，不用等待任务执行完毕

import time
from concurrent.futures import ThreadPoolExecutor

import requests


def get(url):
    print('GET %s' % url)
    response = requests.get(url)
    time.sleep(3)
    return {'url': url, 'content': response.text}


def parse(res):
    res = res.result()
    print('%s parse res is %s' % (res['url'], len(res['content'])))


if __name__ == '__main__':
    urls = [
        'http://www.cnblogs.com/linhaifeng',
        'https://www.python.org',
        'https://www.openstack.org',
    ]

    pool = ThreadPoolExecutor(2)

    for url in urls:
        pool.submit(get, url).add_done_callback(parse)