Python使用管道、队列、zeromq进行IPC速度对比测试

最新推荐文章于 2025-06-12 14:34:55 发布

原创最新推荐文章于 2025-06-12 14:34:55 发布 · 1.4k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#python #管道 #队列 #zeromq #ipc

Python 同时被 2 个专栏收录

180 篇文章

订阅专栏

测试

9 篇文章

订阅专栏

博客主要围绕Python的管道、队列和ZeroMQ展开，作者在MacBook Pro 16 2.6 i7上对这三者进行了测试，但文档未详细给出测试结果。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

管道：

import sys
from  multiprocessing import Process, Pipe
import time

import result

msg_list = []
for i in range(100):
    msg_list.append(result.Test_data)

def worker(receiver): 
    for _ in range(1000000):
        receiver.recv()
 
    sys.exit(1)
 
def main():
    receiver, sender = Pipe(duplex=False)
    p = Process(target=worker, args=(receiver,))
    p.start()
    for _ in range(1000000):
        sender.send(msg_list)
    sender.close()
    p.join()

if __name__ == "__main__":
    start_time = time.time()
    main()
    end_time = time.time()
    duration = end_time - start_time
    msg_per_sec = 1000000 / duration
 
    print(f"Duration: {duration}")
    print(f"Messages Per Second: {msg_per_sec}")

队列：

import sys
from  multiprocessing import Process, Queue
import time

import result

msg_list = []
for i in range(100):
    msg_list.append(result.Test_data)

def worker(q): 
    for _ in range(1000000):
        q.get()
 
    sys.exit(1)
 
def main():
    q = Queue()
    p = Process(target=worker, args=(q,))
    p.start()
    for _ in range(1000000):
        q.put(msg_list)
    p.join()

if __name__ == "__main__":
    start_time = time.time()
    main()
    end_time = time.time()
    duration = end_time - start_time
    msg_per_sec = 1000000 / duration
 
    print(f"Duration: {duration}")
    print(f"Messages Per Second: {msg_per_sec}")

zeromq：

import sys
import zmq
from multiprocessing import Process
import time
 
import result

msg_list = []
for i in range(100):
    msg_list.append(result.Test_data)

def worker():
    context = zmq.Context()
    work_receiver = context.socket(zmq.PULL)
    work_receiver.connect("tcp://127.0.0.1:5557")
 
    for _ in range(1000000):
        work_receiver.recv_pyobj()
 
def main():
    p = Process(target=worker, args=())
    p.start()
    context = zmq.Context()
    ventilator_send = context.socket(zmq.PUSH)
    ventilator_send.bind("tcp://127.0.0.1:5557")
    for _ in range(1000000):
        ventilator_send.send_pyobj(msg_list)
    p.join()
 
if __name__ == "__main__":
    start_time = time.time()
    main()
    end_time = time.time()
    duration = end_time - start_time
    msg_per_sec = 1000000 / duration
 
    print(f"Duration: {duration}")
    print(f"Messages Per Second: {msg_per_sec}")

在我的MacBook Pro 16 2.6 i7上的测试结果
管道：

Duration: 31.029146909713745
Messages Per Second: 32227.763235313043

队列：

Duration: 35.74612784385681
Messages Per Second: 27975.05800818804

zeromq:

Duration: 27.434931993484497
Messages Per Second: 36449.880766516544