Python协程详解

原创已于 2025-05-20 22:57:33 修改 · 616 阅读

8 ·

CC 4.0 BY-SA版权

文章标签：

#python #php #开发语言

于 2025-05-20 22:53:57 首次发布

Python 专栏收录该内容

5 篇文章

订阅专栏

部署运行你感兴趣的模型镜像

Python协程详解

在这里插入图片描述

引言

协程（Coroutine）是Python中实现并发编程的重要方式之一。与传统的多线程和多进程相比，协程具有更轻量级、更高效的特点。本文将详细介绍Python协程的概念、实现方式以及实际应用场景。

1. 协程基础

1.1 什么是协程？

协程是一种用户态的轻量级线程，由用户控制调度。与线程不同，协程的切换不需要操作系统参与，因此开销更小。在Python中，协程主要用于处理I/O密集型任务。

1.2 协程的特点

轻量级：比线程占用更少的系统资源
高效：切换开销小
可控：由用户控制调度
适合I/O密集型任务

2. 协程的实现方式

2.1 生成器实现（Python 3.4之前）

def coroutine_example():
    while True:
        x = yield
        print('Received:', x)

# 使用示例
co = coroutine_example()
next(co)  # 启动协程
co.send('Hello')  # 输出: Received: Hello
co.send(42)  # 输出: Received: 42

2.2 async/await语法（Python 3.5+）

import asyncio

async def hello():
    print('Hello')
    await asyncio.sleep(1)
    print('World')

# 运行协程
asyncio.run(hello())

3. 异步编程基础

3.1 基本概念

import asyncio

async def main():
    # 创建任务
    task1 = asyncio.create_task(say_after(1, 'hello'))
    task2 = asyncio.create_task(say_after(2, 'world'))
    
    # 等待任务完成
    await task1
    await task2

async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

# 运行主函数
asyncio.run(main())

3.2 并发执行

import asyncio

async def fetch_data(url):
    # 模拟网络请求
    await asyncio.sleep(1)
    return f"Data from {url}"

async def main():
    # 并发执行多个任务
    tasks = [
        fetch_data("http://example.com/1"),
        fetch_data("http://example.com/2"),
        fetch_data("http://example.com/3")
    ]
    
    # 等待所有任务完成
    results = await asyncio.gather(*tasks)
    print(results)

asyncio.run(main())

4. 实际应用场景

4.1 网络请求

import aiohttp
import asyncio

async def fetch_url(session, url):
    async with session.get(url) as response:
        return await response.text()

async def main():
    async with aiohttp.ClientSession() as session:
        urls = [
            'http://example.com/1',
            'http://example.com/2',
            'http://example.com/3'
        ]
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
        return results

# 运行示例
results = asyncio.run(main())

4.2 文件操作

import asyncio
import aiofiles

async def read_file(filename):
    async with aiofiles.open(filename, mode='r') as f:
        return await f.read()

async def write_file(filename, content):
    async with aiofiles.open(filename, mode='w') as f:
        await f.write(content)

async def main():
    # 并发读写文件
    content = await read_file('input.txt')
    await write_file('output.txt', content.upper())

asyncio.run(main())

5. 高级特性

5.1 超时控制

import asyncio

async def long_running_task():
    await asyncio.sleep(10)
    return "Task completed"

async def main():
    try:
        # 设置超时时间
        result = await asyncio.wait_for(long_running_task(), timeout=5.0)
        print(result)
    except asyncio.TimeoutError:
        print("Task timed out")

asyncio.run(main())

5.2 信号处理

import asyncio
import signal

async def main():
    loop = asyncio.get_running_loop()
    
    # 注册信号处理器
    for sig in (signal.SIGINT, signal.SIGTERM):
        loop.add_signal_handler(
            sig,
            lambda: asyncio.create_task(shutdown())
        )
    
    # 主程序逻辑
    while True:
        await asyncio.sleep(1)

async def shutdown():
    print("Shutting down...")
    tasks = [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
    [task.cancel() for task in tasks]
    await asyncio.gather(*tasks, return_exceptions=True)
    loop.stop()

asyncio.run(main())

6. 最佳实践

6.1 错误处理

import asyncio

async def risky_operation():
    if random.random() < 0.5:
        raise ValueError("Random error")
    return "Success"

async def main():
    try:
        result = await risky_operation()
        print(result)
    except ValueError as e:
        print(f"Error occurred: {e}")
    except Exception as e:
        print(f"Unexpected error: {e}")

asyncio.run(main())

6.2 资源管理

import asyncio
from contextlib import asynccontextmanager

@asynccontextmanager
async def managed_resource():
    print("Acquiring resource")
    try:
        yield "resource"
    finally:
        print("Releasing resource")

async def main():
    async with managed_resource() as resource:
        print(f"Using {resource}")
        await asyncio.sleep(1)

asyncio.run(main())

7. 性能优化

7.1 并发限制

import asyncio
import aiohttp

async def fetch_with_semaphore(sem, session, url):
    async with sem:
        async with session.get(url) as response:
            return await response.text()

async def main():
    # 限制并发数为3
    sem = asyncio.Semaphore(3)
    async with aiohttp.ClientSession() as session:
        tasks = [
            fetch_with_semaphore(sem, session, url)
            for url in ['http://example.com'] * 10
        ]
        results = await asyncio.gather(*tasks)
        return results

asyncio.run(main())

7.2 批量处理

import asyncio

async def process_batch(items, batch_size=5):
    results = []
    for i in range(0, len(items), batch_size):
        batch = items[i:i + batch_size]
        # 处理批次
        batch_results = await asyncio.gather(
            *[process_item(item) for item in batch]
        )
        results.extend(batch_results)
    return results

async def process_item(item):
    await asyncio.sleep(0.1)  # 模拟处理时间
    return f"Processed {item}"

# 使用示例
items = list(range(20))
results = asyncio.run(process_batch(items))

结语

Python协程是一种强大的并发编程工具，特别适合处理I/O密集型任务。通过合理使用协程，我们可以：

提高程序的并发性能
减少资源消耗
简化异步代码的编写
提高代码的可维护性

记住，协程不是万能的，它最适合处理I/O密集型任务。对于CPU密集型任务，还是应该考虑使用多进程。

练习

使用协程实现一个简单的聊天服务器

import asyncio
import json
from datetime import datetime

class ChatServer:
    def __init__(self, host='localhost', port=8888):
        self.host = host
        self.port = port
        self.clients = {}  # 存储客户端连接
        self.messages = []  # 存储聊天记录

    async def handle_client(self, reader, writer):
        # 获取客户端地址
        addr = writer.get_extra_info('peername')
        print(f"New connection from {addr}")
        
        # 生成客户端ID
        client_id = f"client_{len(self.clients)}"
        self.clients[client_id] = writer
        
        try:
            # 发送欢迎消息
            welcome_msg = {
                'type': 'system',
                'content': f'Welcome! Your ID is {client_id}',
                'timestamp': datetime.now().strftime('%H:%M:%S')
            }
            writer.write(json.dumps(welcome_msg).encode() + b'\n')
            await writer.drain()
            
            # 广播新用户加入
            await self.broadcast({
                'type': 'system',
                'content': f'User {client_id} joined the chat',
                'timestamp': datetime.now().strftime('%H:%M:%S')
            }, exclude=client_id)
            
            # 处理客户端消息
            while True:
                data = await reader.readline()
                if not data:
                    break
                    
                message = json.loads(data.decode())
                message['timestamp'] = datetime.now().strftime('%H:%M:%S')
                message['sender'] = client_id
                
                # 存储消息
                self.messages.append(message)
                
                # 广播消息
                await self.broadcast(message)
                
        except Exception as e:
            print(f"Error handling client {client_id}: {e}")
        finally:
            # 清理客户端连接
            del self.clients[client_id]
            writer.close()
            await writer.wait_closed()
            
            # 广播用户离开
            await self.broadcast({
                'type': 'system',
                'content': f'User {client_id} left the chat',
                'timestamp': datetime.now().strftime('%H:%M:%S')
            })
            print(f"Connection from {addr} closed")

    async def broadcast(self, message, exclude=None):
        """广播消息给所有客户端"""
        message_str = json.dumps(message) + '\n'
        for client_id, writer in self.clients.items():
            if client_id != exclude:
                try:
                    writer.write(message_str.encode())
                    await writer.drain()
                except Exception as e:
                    print(f"Error broadcasting to {client_id}: {e}")

    async def start(self):
        """启动服务器"""
        server = await asyncio.start_server(
            self.handle_client, self.host, self.port
        )
        
        print(f"Chat server running on {self.host}:{self.port}")
        async with server:
            await server.serve_forever()

# 客户端代码
class ChatClient:
    def __init__(self, host='localhost', port=8888):
        self.host = host
        self.port = port
        self.reader = None
        self.writer = None

    async def connect(self):
        """连接到服务器"""
        self.reader, self.writer = await asyncio.open_connection(
            self.host, self.port
        )
        print("Connected to chat server")

    async def receive_messages(self):
        """接收消息"""
        while True:
            try:
                data = await self.reader.readline()
                if not data:
                    break
                message = json.loads(data.decode())
                self.display_message(message)
            except Exception as e:
                print(f"Error receiving message: {e}")
                break

    def display_message(self, message):
        """显示消息"""
        timestamp = message['timestamp']
        if message['type'] == 'system':
            print(f"[{timestamp}] System: {message['content']}")
        else:
            sender = message['sender']
            content = message['content']
            print(f"[{timestamp}] {sender}: {content}")

    async def send_message(self, content):
        """发送消息"""
        if self.writer:
            message = {
                'type': 'message',
                'content': content
            }
            self.writer.write(json.dumps(message).encode() + b'\n')
            await self.writer.drain()

    async def run(self):
        """运行客户端"""
        await self.connect()
        
        # 启动接收消息的任务
        receive_task = asyncio.create_task(self.receive_messages())
        
        try:
            while True:
                message = input("> ")
                if message.lower() == 'quit':
                    break
                await self.send_message(message)
        except KeyboardInterrupt:
            pass
        finally:
            if self.writer:
                self.writer.close()
                await self.writer.wait_closed()
            receive_task.cancel()

# 使用示例
async def main():
    # 启动服务器
    server = ChatServer()
    server_task = asyncio.create_task(server.start())
    
    # 等待服务器启动
    await asyncio.sleep(1)
    
    # 启动客户端
    client = ChatClient()
    await client.run()
    
    # 清理
    server_task.cancel()

if __name__ == "__main__":
    asyncio.run(main())

练习解析

这个聊天服务器实现了以下功能：

服务器端功能：
- 支持多客户端同时连接
- 广播消息给所有客户端
- 处理客户端加入和离开
- 保存聊天记录
- 系统消息通知
客户端功能：
- 连接到服务器
- 发送消息
- 接收消息
- 优雅退出
技术特点：
- 使用asyncio实现异步通信
- 使用JSON格式传输消息
- 实现了基本的错误处理
- 支持优雅关闭
使用方法：
- 运行服务器：python chat_server.py
- 运行客户端：python chat_client.py
- 输入消息并按回车发送
- 输入’quit’退出