测试Python多线程与多进程

最新推荐文章于 2025-06-04 15:45:47 发布

原创最新推荐文章于 2025-06-04 15:45:47 发布 · 653 阅读

2 ·

CC 4.0 BY-SA版权

Python 专栏收录该内容

23 篇文章

订阅专栏

本文通过实际代码演示了单进程、多线程及多进程在CPU密集型任务和IO密集型任务上的表现差异，并分析了Python全局锁(GIL)的影响。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >


#!/usr/bin/python
# -*- coding: UTF-8 -*-
import time
import os 
from threading import Thread
from multiprocessing import Process

def count(x, y):
    # 使程序完成大量计算
    for i in xrange(100000):
        x += x
        y += y
        
def write(filename):
    f = open(filename, "w")
    for i in xrange(100000):
        f.write("testwrite\n")
    f.close()
 
def read(filename):
    f = open(filename, "r")
    lines = f.readlines()
    f.close()
    
def io(filename):
    write(filename)
    read(filename)

def testSingleProcess():
    # CPU密集操作
    t = time.time()
    for i in xrange(10):
        count(1, 1)
    print("SingleProcess cpu", time.time() - t)
     
    # IO密集操作
    t = time.time()
    for i in xrange(10):
        io("test%d.txt"%i)
    print("SingleProcess IO", time.time() - t)
    
def testMultithreading():
    counts = []
    t = time.time()
    for i in xrange(10):
        thread = Thread(target=count, args=(1,1))
        counts.append(thread)
        thread.start()
     
    e = counts.__len__()
    while True:
        for th in counts:
            if not th.is_alive():
                e -= 1
        if e <= 0:
            break
    print("Multithreading cpu",time.time() - t)
    
    ios = []
    t = time.time()
    for i in xrange(10):
        thread = Thread(target=io, args=("test%d.txt"%i,))
        ios.append(thread)
        thread.start()
     
    e = ios.__len__()
    while True:
        for th in ios:
            if not th.is_alive():
                e -= 1
        if e <= 0:
            break
    print("Multithreading IO",time.time() - t)
    
def testMultiprocess():
    counts = []
    t = time.time()
    for i in xrange(10):
        process = Process(target=count, args=(1,1))
        counts.append(process)
        process.start()
    e = counts.__len__()
    while True:
        for th in counts:
            if not th.is_alive():
                e -= 1
        if e <= 0:
            break
    print("Multiprocess cpu", time.time() - t)
    
    ios = []
    t = time.time()
    for i in xrange(10):
        process = Process(target=io, args=("test%d.txt"%i,))
        ios.append(process)
        process.start()
     
    e = ios.__len__()
    while True:
        for th in ios:
            if not th.is_alive():
                e -= 1
        if e <= 0:
            break
    print("Multiprocess IO", time.time() - t)
    
def clearTmpFile():
    for i in xrange(10):
        os.remove("test%d.txt"%i) 
if __name__ == "__main__":   

    testSingleProcess()
    time.sleep(2)
    clearTmpFile()
    time.sleep(2)
    
    testMultithreading()
    time.sleep(2)
    clearTmpFile()
    time.sleep(2)
    
    testMultiprocess()
    time.sleep(2)
    clearTmpFile()

测试时的cpu运行情况如下：

运行结果：

('SingleProcess cpu', 41.20599985122681)
('SingleProcess IO', 0.3340001106262207)
('Multithreading cpu', 41.062999963760376)
('Multithreading IO', 4.700000047683716)
('Multiprocess cpu', 11.5239999294281)
('Multiprocess IO', 0.29200005531311035)

可以看出，只有多进程下，才能利用多核的优势，将大量计算快速执行完毕

原因是:

Python是运行在解释器中的语言，有一个全局锁（GIL），在使用多线程(Thread)的情况下，不能发挥多核的优势。

而使用多进程(Multiprocess)，则可以发挥多核的优势真正地提高效率。

另外，Python还有基于协程的网络库gevent（github）

协程不同于线程的地方在于协程不是操作系统进行切换，而是由编码进行切换，即由程序员控制的，这样就没有了线程的所谓安全问题。