Ubuntu18环境下 python 多进程与多线程测试

测试说明

测试的对象为多次求平方的程序。

def task_powers(limit: int) -> float:
    """return powers of arrays from 1 to limit

    Args:
        limit (int): max num

    Returns:
        float: list of powers
    """
    return [x**2 for x in range(limit)]

单一线程单一进程测试函数

def test_normal(test_func: FunctionType) -> None:
    """ test TEST_TIMES times test_func with args = (TASK_PARAM)

    Args:
        test_func (FunctionType): test function
    """
    # 测试多次运行时间
    time_s = time.time()
    for i in range(TEST_TIMES):
        task_powers(TASK_PARAM)
    time_e = time.time()
    time_used = time_e - time_s
    print("Total time used with normal type: \t", time_used)

多线程下测试函数

def test_multiThreading(test_func: FunctionType) -> None:
    """ test TEST_TIMES times test_func with args = TASK_PARAM by multithrading

    Args:
        test_func (FunctionType): _description_
    """
    # 测试多次运行时间
    time_s = time.time()
    ts = []
    for i in range(TEST_TIMES):
        ts.append(Thread(target=test_func, args=(TASK_PARAM,)))
        ts[i].start()
    for i in range(TEST_TIMES):
        ts[i].join()
    time_e = time.time()
    time_used = time_e - time_s
    print("Total time used with multithrading type: \t", time_used)

多进程下测试函数

def test_multiProposing(test_func: FunctionType) -> None:
    """ test TEST_TIMES times test_func with args = TASK_PARAM by multiprocessing

    Args:
        test_func (FunctionType): test function
    """
    # 测试多次运行时间
    time_s = time.time()
    ts = []
    for i in range(TEST_TIMES):
        ts.append(Process(target=test_func, args=(TASK_PARAM,)))
        ts[i].start()
    for i in range(TEST_TIMES):
        ts[i].join()
    time_e = time.time()
    time_used = time_e - time_s
    print("Total time used with multiprocessing type: \t", time_used)

主函数与测试结果

TASK_FUNC = task_powers
TASK_PARAM = 5000000   # 测试用函数的参数
TEST_TIMES = 8
if __name__ == '__main__':
    test_normal(TASK_FUNC)            # test normal
    test_multiThreading(TASK_FUNC)    # test multithrading
    test_multiProposing(TASK_FUNC)    # test multiprocessing

查看程序输出：

Total time used with normal type:                          9.709944009780884
Total time used with multithrading type:            9.602959871292114
Total time used with multiprocessing type:       1.7592921257019043

打开Ubuntu的系统监视器看各个CPU与内存的使用情况：

可以发现，单一进程单一线程的情况下会把CPU单核落满；多线程情况下会把多个核心都调度起来，但每个核心的利用率不高，平均每个核心利用率都在12.5%（本次测试开了8个线程，也就是说多个核心加起来的利用率约等于拉满一个核心的效果）；最后多进程程序运行时几乎所有核心都拉满了利用率，程序运行时间大幅下降，但并不是只有单进程的1/8，推测多进程设置与调度花费了一定时间。