Python标准库queue模块原理浅析

最新推荐文章于 2025-06-25 11:43:37 发布

小屋子大侠

最新推荐文章于 2025-06-25 11:43:37 发布

阅读量1.7k

点赞数 6

CC 4.0 BY-SA版权

分类专栏： python 文章标签： python python标准库 queue

本文链接：https://blog.youkuaiyun.com/qq_33339479/article/details/86706626

Python标准库queue模块原理浅析

本文环境python3.5.2

queue模块的实现思路

作为一个线程安全的队列模块，该模块提供了线程安全的一个队列，该队列底层的实现基于Python线程threading中的Condition原理来实现的队列（对该原理的讲解可参考以前博文）。本文就先概述一下queue队列的使用示例；

import queue
import threading

q = queue.Queue()

def worker():
    while True:
        item = q.get()
        if item is None:
            break
        print('item ', item)
        q.task_done()

threads = []
for i in range(10):
    t = threading.Thread(target=worker)
    t.start()
    threads.append(t)


for item in range(20):
    q.put(item)

q.join()

for i in range(10):
    q.put(None)

for t in threads:
    t.join()

输出结果如下；

item  0
item  1
item  2
item  3
item  4
item  5
item  6
item  7
item  8
item  9
item  10
item  11
item  12
item  13
item  14
item  15
item  16
item  17
item  18
item  19

由输出结果可知，队列保证了线程之间数据的安全性，通过初始化一个Queue的实例，然后向该实例中put数据，让线程get等待，此时就保证了各个线程之间互斥的取数据并进行相关的业务处理。本文来简述一下Queue类的大概流程。

Queue类的实现原理

首先查看Queue类的定义；

class Queue:
    '''Create a queue object with a given maximum size.

    If maxsize is <= 0, the queue size is infinite.
    '''

    def __init__(self, maxsize=0):
        self.maxsize = maxsize                                                  # 初始化队列的大小，
        self._init(maxsize)                                                     # 初始化一个队列deque实例

        # mutex must be held whenever the queue is mutating.  All methods
        # that acquire mutex must release it before returning.  mutex
        # is shared between the three conditions, so acquiring and
        # releasing the conditions also acquires and releases mutex.
        self.mutex = threading.Lock()                                           # 获取线程锁

        # Notify not_empty whenever an item is added to the queue; a
        # thread waiting to get is notified then.
        self.not_empty = threading.Condition(self.mutex)                        # 获取为空的条件变量

        # Notify not_full whenever an item is removed from the queue;
        # a thread waiting to put is notified then.
        self.not_full = threading.Condition(self.mutex)                         # 获取未满的条件变量

        # Notify all_tasks_done whenever the number of unfinished tasks
        # drops to zero; thread waiting to join() is notified to resume
        self.all_tasks_done = threading.Condition(self.mutex)                   # 获取全部任务完成的