Python有界任务队列完整实现：固定worker、背压与12项离线测试

当批量处理一组AI任务时，常见写法是为每条输入创建Task，再在内部套Semaphore。它限制同时工作，却仍可能创建大量等待任务。本文换成固定worker和有界Queue：生产者在队列满时等待，结果按原始输入顺序归位。

示例完全离线，工作函数只做字符串变换。它不连接模型、不测试429，也不把本地队列当作服务端额度。

环境、文件和输入输出

Python 3.11及以上，全部标准库。建立queue-lab目录，保存bounded_queue.py、window_probe.py、test_bounded_queue.py，进入目录运行python bounded_queue.py、python window_probe.py和python -m unittest -v。

run_jobs接收普通list，最多100个字符串，每项最多128个Python字符；workers为1—16的整数，capacity为1—100的整数，布尔值不当整数接受。这些是教学保护值，不是厂商参数。

work必须是可等待的本地工作函数，返回字符串；普通Exception使该项failed，其他项继续，不自动重试、不输出异常原文。返回Outcome元组，保留输入下标、ok/failed状态和结果；另外返回观测到的峰值指标。开始第一个await前复制输入快照，但输入列表本来就在内存，不能据此声称流式处理了无限数据。

完整实现：bounded_queue.py

"""Finite offline batch, one event loop; not an API rate limiter."""
import asyncio
from dataclasses import dataclass


@dataclass(frozen=True)
class Outcome:
    index: int
    status: str
    value: str


async def run_jobs(items, work, *, workers=2, capacity=2):
    if type(workers) is not int or not 1 <= workers <= 16:
        raise ValueError("invalid_workers")
    if type(capacity) is not int or not 1 <= capacity <= 100:
        raise ValueError("invalid_capacity")
    if type(items) is not list or len(items) > 100:
        raise ValueError("invalid_items")
    if any(type(x) is not str or len(x) > 128 for x in items):
        raise ValueError("invalid_item")
    # Snapshot caller's finite input before the first suspension.
    items = tuple(items)
    owner = asyncio.current_task()
    queue = asyncio.Queue(maxsize=capacity)
    results = [None] * len(items)
    active = peak_active = peak_waiting = 0

    async def produce():
        nonlocal peak_waiting
        for index, text in enumerate(items):
            await queue.put((index, text))
            peak_waiting = max(peak_waiting, queue.qsize())
        for _ in range(workers):
            await queue.put(None)

    async def consume():
        nonlocal active, peak_active
        while True:
            entry = await queue.get()
            try:
                if entry is None:
                    return
                index, text = entry
                active += 1
                peak_active = max(peak_active, active)
                try:
                    value = await work(text)
                    if type(value) is not str:
                        raise TypeError("invalid_result")
                    results[index] = Outcome(index, "ok", value)
                except asyncio.CancelledError:
                    if not owner.cancelling():
                        # A child cannot silently disappear and strand the producer.
                        raise RuntimeError("worker_cancelled") from None
                    raise
                except Exception:
                    # No automatic retries and no exception-body disclosure.
                    results[index] = Outcome(index, "failed", "")
                finally:
                    active -= 1
            finally:
                queue.task_done()  # Includes sentinel and cancelled jobs.

    async with asyncio.TaskGroup() as group:
        for _ in range(workers):
            group.create_task(consume())
        group.create_task(produce())
    await queue.join()
    assert all(x is not None for x in results)
    return tuple(results), {"peak_active": peak_active,
                            "peak_waiting_jobs_sampled": peak_waiting}


async def demo():
    async def local_work(text):
        await asyncio.sleep(0)  # Yield locally, not a latency measurement.
        if text == "bad":
            raise RuntimeError("synthetic failure")
        return text.upper()

    results, stats = await run_jobs(["a", "bad", "c", "d"], local_work,
                                    workers=2, capacity=1)
    print([(x.index, x.status, x.value) for x in results])
    print(stats)


if __name__ == "__main__":
    asyncio.run(demo())

生产者只创建一个协程，固定创建W个消费者。Queue最多容纳Q个条目；生产者还可能在put处持有一个待入队条目。退出用None哨兵，每个消费者取到哨兵后返回；task_done放finally里，与每次成功get配对，包括哨兵和被取消的作业。

TaskGroup等待本批生产者、消费者结束。整批被取消时，正在执行的工作有机会在finally清理，取消继续传播。若工作函数自行抛取消而外层批次没有被取消，将它升级为worker_cancelled错误，由任务组取消其余子任务，避免消费者静默消失后生产者永远堵在put。工作函数应配合取消，不吞掉CancelledError或执行长时间同步阻塞。

普通作业失败采用“保留失败位置，其他作业继续”的批处理政策；这不适合要求全有或全无的事务。实际业务需要按副作用、持久化和幂等要求另设计，不要把示例包装成生产任务系统。

对照模型：window_probe.py

以下模型说明同样两个工作位置，窗口内启动次数仍会随假想耗时改变。它不读真实时钟，不产生HTTP请求，也不是速率限制器。

"""Deterministic hypothetical schedule; no clock or network measurement."""
import heapq
import math


def planned_starts(durations, slots=2):
    if type(slots) is not int or not 1 <= slots <= 16:
        raise ValueError("invalid_slots")
    if type(durations) is not list or len(durations) > 100:
        raise ValueError("invalid_durations")
    for duration in durations:
        if type(duration) not in (int, float) or not 0 < duration <= 60:
            raise ValueError("invalid_duration")
        if not math.isfinite(duration):
            raise ValueError("invalid_duration")
    available = [(0.0, slot) for slot in range(slots)]
    heapq.heapify(available)
    starts = []
    for duration in durations:
        start, slot = heapq.heappop(available)
        starts.append(start)
        heapq.heappush(available, (start + duration, slot))
    return starts


if __name__ == "__main__":
    for label, duration in (("slow", 1.0), ("fast", 0.01)):
        starts = planned_starts([duration] * 6)
        print(label, starts, "starts_in_[0,1)=", sum(0 <= t < 1 for t in starts))

完整回归：test_bounded_queue.py

import asyncio
import unittest
from bounded_queue import run_jobs
from window_probe import planned_starts


class QueueTests(unittest.IsolatedAsyncioTestCase):
    async def test_empty(self):
        async def work(x): return x
        results, stats = await run_jobs([], work)
        self.assertEqual(results, ())
        self.assertEqual(stats["peak_active"], 0)

    async def test_concurrency_and_queue_bound(self):
        active = peak = 0
        both_running = asyncio.Event()
        async def work(x):
            nonlocal active, peak
            active += 1; peak = max(peak, active)
            try:
                if active == 2: both_running.set()
                await both_running.wait()
                await asyncio.sleep(0)
                return x
            finally:
                active -= 1
        results, stats = await run_jobs([str(i) for i in range(8)], work,
                                        workers=2, capacity=1)
        self.assertEqual(peak, 2)
        self.assertEqual(stats["peak_active"], peak)
        self.assertLessEqual(stats["peak_waiting_jobs_sampled"], 1)
        self.assertTrue(all(x.status == "ok" for x in results))

    async def test_input_order_survives_completion_order(self):
        released = asyncio.Event()
        async def work(x):
            if x == "first": await released.wait()
            else: released.set()
            return x
        results, _ = await run_jobs(["first", "second"], work)
        self.assertEqual([x.value for x in results], ["first", "second"])

    async def test_failure_isolated_without_retry(self):
        calls = []
        async def work(x):
            calls.append(x)
            if x == "bad": raise RuntimeError("private synthetic detail")
            return x
        results, _ = await run_jobs(["a", "bad", "c"], work)
        self.assertEqual([x.status for x in results], ["ok", "failed", "ok"])
        self.assertEqual(results[1].value, "")
        self.assertCountEqual(calls, ["a", "bad", "c"])

    async def test_invalid_result(self):
        async def work(x): return 123
        results, _ = await run_jobs(["x"], work)
        self.assertEqual(results[0].status, "failed")

    async def test_cancel_joins_active_workers(self):
        started, blocker = asyncio.Event(), asyncio.Event()
        active = 0
        async def work(x):
            nonlocal active
            active += 1; started.set()
            try:
                await blocker.wait()
                return x
            finally:
                active -= 1
        batch = asyncio.create_task(run_jobs([str(i) for i in range(10)], work,
                                              workers=2, capacity=1))
        await started.wait()
        batch.cancel()
        with self.assertRaises(asyncio.CancelledError): await batch
        self.assertEqual(active, 0)

    async def test_bad_limits_fail_before_work(self):
        async def work(x): self.fail("must not execute")
        for name in ("workers", "capacity"):
            for value in (True, 0, -1, "2", 101):
                with self.assertRaises(ValueError):
                    await run_jobs(["x"], work, **{name:value})

    async def test_worker_cannot_silently_cancel_and_strand_producer(self):
        async def work(x): raise asyncio.CancelledError()
        with self.assertRaises(ExceptionGroup) as caught:
            await asyncio.wait_for(run_jobs(["x"] * 10, work,
                                             workers=1, capacity=1), timeout=2)
        self.assertTrue(any(str(x) == "worker_cancelled" for x in caught.exception.exceptions))

    async def test_bad_inputs(self):
        async def work(x): self.fail("must not execute")
        for items in (("x",), ["x"] * 101, [3], ["x" * 129]):
            with self.assertRaises(ValueError): await run_jobs(items, work)


class WindowTests(unittest.TestCase):
    def test_same_concurrency_different_window_count(self):
        slow, fast = planned_starts([1.] * 6), planned_starts([.01] * 6)
        self.assertEqual(sum(t < 1 for t in slow), 2)
        self.assertEqual(sum(t < 1 for t in fast), 6)

    def test_half_open_window_and_slots(self):
        self.assertEqual(planned_starts([1., 1.], slots=1), [0., 1.])
        self.assertEqual(sum(t < 1 for t in planned_starts([1., 1.], 1)), 1)

    def test_bad_schedule_inputs(self):
        for value in (True, 0, -1, float("nan"), float("inf"), "1"):
            with self.assertRaises(ValueError): planned_starts([value])
        for slots in (True, 0, 17):
            with self.assertRaises(ValueError): planned_starts([1.], slots)


if __name__ == "__main__":
    unittest.main()

实际运行与指标解释

本次在Python 3.11.9运行12项测试通过。bounded_queue.py得到：

[(0, 'ok', 'A'), (1, 'failed', ''), (2, 'ok', 'C'), (3, 'ok', 'D')]
{'peak_active': 1, 'peak_waiting_jobs_sampled': 1}

workers=2是上限，不保证两个位置始终占满。这个短作业演示实际峰值为1；测试另用Event确保两个作业确实重叠，再验证峰值不超过配置。不能把演示的调度顺序当成所有机器上的性能保证。

window_probe.py的slow与fast样本在[0,1)窗口里分别启动2项和6项。两者同为2个工作位置，仍没有得到相同的窗口请求数。

peak_waiting_jobs_sampled只在生产者成功放入业务项后采样，不计哨兵，不是连续监控仪表。Queue本身控制条目容量；结果槽、输入快照、单个结果大小、其他批次和其他进程均不在这个指标的证明范围内。结果字符串没有大小上限，下一课再单独实现任务预算；不要以本例宣称总内存恒定。

常见问题与故障定位

队列满了为什么没报错？本例用await put，满时等待；这是一种背压政策。交互入口还要决定可等待多久，不能让用户无限等。

join为什么不结束？先核对每次get是否恰好一次task_done，再检查工作函数是否卡住。示例没有单项/整批业务超时，实际接入可由调用方用asyncio.timeout设置适当的整体期限，并验证取消清理。

为什么取消测试还要等待任务？只调用cancel而不await，不能证明清理完成。测试等待外层批次结束，确认活跃工作计数归零；这只证明本地示例控制流，不证明远端取消或账单回滚。

为什么遇到failed没有自动重试？重试是另一层政策，需要总尝试预算、幂等和远端错误分类。不能把无限重试塞进消费者，抵消本地排队带来的保护。

为什么仍可能429？worker数没有表达时间窗口内请求量、Token量或其他应用共享消耗。应查实际服务规则与账号限制，不能把SDK连接数当账户配额。

worker_cancelled的ExceptionGroup怎么处理？说明工作函数发生了批次外取消；整批已停止，先调查取消来源，不自动重建任务。不要捕获后无条件宣称全部完成。

练习与下一篇

把capacity分别改为1和3，保持workers=2，比较结果顺序与采样指标；不要据此推断真实接口吞吐。再加一个本地工作函数永久等待Event，用调用方的asyncio.timeout结束整批，并检查finally执行。不要为测试队列而接入真实密钥。

下一篇实现单个用户任务的尝试次数和结果大小预算。欢迎在评论里给出脱敏后的输入数量、可接受等待时间和失败处理要求，一起判断哪一层应拒绝或等待。

来源和验证边界

Python 3.11队列文档说明有界Queue的阻塞、get与task_done配对；同步原语文档说明Semaphore管理同时进入临界区的数量。这些原语供同一事件循环使用，不能直接当跨进程、跨机器的共享额度。

OpenAI限流指南列出请求量、Token量等不同维度。这里仅用它说明限流维度不止并发，未照搬任何账户的额度、价格或模型支持范围；其他服务应核对自己的当前协议。

今天检索并打开近期官方公告，没有发现适合本课的直接限流能力上新，因此采用常青教学。内容由AI辅助整理，代码只做本地离线验证，没有模型调用、真实429实验、吞吐或费用测量。