windows系统下python多进程创建子进程的代码必须在if name == ‘__main__’:代码块内的通俗解释
在python多进程的学习过程中,发现windows系统下创建子进程的代码必须在main代码块内,否则就会抛出RuntimeError,在网上找了找大佬们的专业回答后,终于有了些自己的理解,也有一些新的问题。
文章目录
个人理解
与windows创建进程的方式有关,由于windows创建子进程方式为spawn,即创建的子进程会分配一个全新的python解释器。
而进程与线程不同,进程间其内部的变量不能共享,且由于target目标函数可能需要全局代码中的某些变量的值,
因此新创建的子进程的python解释器需要重新执行一遍全局代码来初始化其内部的全局变量,然后再传参执行target函数。
多进程的简单代码样例
代码如下:
import time
from multiprocessing import Process
# 子进程的target函数
def func(name):
time.sleep(2)
# 输出子进程的__name__
print(f"this is child process: {name}, name = {__name__}")
# 全局代码
print("some words.")
if __name__ == '__main__':
# 输出主进程的__name__
print(f'this is {__name__}')
pl = []
for i in range(5):
name = f'python_process_{i}'
# 创建子进程
p = Process(target=func, args=(name,))
p.start()
pl.append(p)
for p in pl:
p.join()
print('main process end.')
执行结果如下:
PS D:\123\VSCode_codes\spider> python -u "d:\123\VSCode_codes\spider\2.py"
some words.
this is __main__
some words.
some words.
some words.
some words.
some words.
this is child process: python_process_0, name = __mp_main__
this is child process: python_process_1, name = __mp_main__
this is child process: python_process_2, name = __mp_main__
this is child process: python_process_3, name = __mp_main__
this is child process: python_process_4, name = __mp_main__
main process end.
可知,全局代码不论主进程还是子进程都会执行,且主进程的__name__ == ‘__main__’,创建的子进程__name__ == ‘__mp_main__’
针对于不同情形的流程
无外乎就是创建子进程的代码是否在main函数内,情形与流程如下:
-
若创建子进程的代码在main函数内,执行py文件时:
主进程正常执行全局代码,由于主进程的__name__ == ‘__main__’, 因此会执行main函数代码块,创建了子进程。
由于spawn,会分配子进程一个新的python解释器,子进程重新执行一遍全局代码。
此时因为子进程的__name__ == ‘__mp_main__’,也就是说子进程的__name__不是’__main__',不会执行main函数代码块,也就不会再执行创建子进程的代码了。
在子进程执行完全局代码后,传参执行target函数。 -
反之,若创建子进程的代码不在main函数内,而在全局代码内,执行py文件时:
主进程执行全局代码,此时创建了子进程
由于spawn,会分配子进程一个新的python解释器,子进程会执行全局代码,包括创建子进程的语句
现在子进程又创建了一个子进程的子进程,在它的子进程中又会执行创建子进程的语句,循环往复,最终崩溃
抛出RuntimeError的源码位置
# spawn.py
def _check_not_importing_main():
if getattr(process.current_process(), '_inheriting', False):
raise RuntimeError('''
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.''')
一些疑问
测试代码如下:
import time
from multiprocessing import Process
# 子进程的子进程target函数
def f(name):
time.sleep(2)
print(f"child-child process created by :{name}")
# 子进程的target函数
def func(name):
time.sleep(2)
p = Process(target=f, args=(name,))
p.start()
print(f"child process: {name}")
# 全局代码
print(f"global words from {__name__}.")
if __name__ == '__main__':
pl = []
for i in range(3):
name = f'python_process_{i}'
# 创建子进程
p = Process(target=func, args=(name,))
p.start()
pl.append(p)
for p in pl:
p.join()
print('main process end.')
输出:
PS D:\123\VSCode_codes\spider> python -u "d:\123\VSCode_codes\spider\3.py"
global words from __main__.
global words from __mp_main__.
global words from __mp_main__.
global words from __mp_main__.
child process: python_process_0
child process: python_process_1
child process: python_process_2
global words from __mp_main__.
global words from __mp_main__.
global words from __mp_main__.
child-child process created by :python_process_0
child-child process created by :python_process_1
child-child process created by :python_process_2
main process end.
若在子进程的中创建子进程并不会报错,而此异常似乎仅在全局代码创建运行子进程时才会抛出,原因是什么?
源码中_check_not_importing_main函数中的’_inheriting’字段含义是什么?
若是继承的意思,为何仅在子进程在全局代码创建时才为True,而在子进程函数中创建却为False?
欢迎大佬们前来解答。