报错信息
D:\mydatapro\venv_net\Lib\site-packages\threadpoolctl.py:1214: RuntimeWarning:
Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at
the same time. Both libraries are known to be incompatible and this
can cause random crashes or deadlocks on Linux when loaded in the
same Python program.
Using threadpoolctl may cause crashes or deadlocks. For more
information and possible workarounds, please see
https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md
warnings.warn(msg, RuntimeWarning)
错误原因
这个警告表明当前的Python 环境中同时加载了 Intel OpenMP (libiomp) 和 LLVM OpenMP (libomp),这可能导致随机崩溃或死锁,尤其是在 Linux 系统上。这种冲突通常发生在使用了不同编译器编译的库时(例如,NumPy 使用了 Intel MKL,而其他库使用了 GCC 或 Clang 编译)。
我试过的方法
- 在运行 Python 脚本之前,设置环境变量,使用的是 Windows,所以在命令行中设置:【没有用】
set MKL_THREADING_LAYER=GNU
- 重新安装 NumPy 和 SciPy【没有用】
conda install -c conda-forge numpy scipy
再次查找原因
import threadpoolctl
# 获取当前加载的线程池信息
threadpool_info = threadpoolctl.threadpool_info()
# 打印完整的 threadpool_info 返回内容
print("完整的 threadpool_info 返回内容:")
print(threadpool_info)
# 打印加载的 OpenMP 库
for info in threadpool_info:
print(f"Library: {info.get('user_api', '未知')}")
print(f" - Type: {info.get('type', '未知')}")
print(f" - Name: {info.get('name', '未知')}")
print(f" - Version: {info.get('version', '未知')}")
print(f" - Number of threads: {info.get('num_threads', '未知')}")
print(f" - User API: {info.get('user_api', '未知')}")
print(f" - Internal API: {info.get('internal_api', '未知')}")
print(f" - Prefix: {info.get('prefix', '未知')}")
print(f" - File: {info.get('file', '未知')}")
print("-" * 40)
D:\mydatapro\venv_net\Lib\site-packages\threadpoolctl.py:1214: RuntimeWarning:
Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at
the same time. Both libraries are known to be incompatible and this
can cause random crashes or deadlocks on Linux when loaded in the
same Python program.
Using threadpoolctl may cause crashes or deadlocks. For more
information and possible workarounds, please see
https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md
warnings.warn(msg, RuntimeWarning)
完整的 threadpool_info 返回内容:
[{'user_api': 'openmp', 'internal_api': 'openmp', 'num_threads': 1, 'prefix': 'libomp', 'filepath': 'C:\\Windows\\System32\\libomp140.x86_64.dll', 'version': None}, {'user_api': 'openmp', 'internal_api': 'openmp', 'num_threads': 16, 'prefix': 'vcomp', 'filepath': 'E:\\Anaconda\\vcomp140.dll', 'version': None}, {'user_api': 'openmp', 'internal_api': 'openmp', 'num_threads': 16, 'prefix': 'libiomp', 'filepath': 'D:\\mydatapro\\venv_net\\Lib\\site-packages\\torch\\lib\\libiomp5md.dll', 'version': None}, {'user_api': 'openmp', 'internal_api': 'openmp', 'num_threads': 1, 'prefix': 'libiomp', 'filepath': 'D:\\mydatapro\\venv_net\\Lib\\site-packages\\torch\\lib\\libiompstubs5md.dll', 'version': None}, {'user_api': 'blas', 'internal_api': 'openblas', 'num_threads': 16, 'prefix': 'libopenblas', 'filepath': 'D:\\mydatapro\\venv_net\\Lib\\site-packages\\numpy.libs\\libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-2bde3a66a51006b2b53eb373ff767a3f.dll', 'version': '0.3.23.dev', 'threading_layer': 'pthreads', 'architecture': 'Haswell'}, {'user_api': 'openmp', 'internal_api': 'openmp', 'num_threads': 16, 'prefix': 'vcomp', 'filepath': 'D:\\mydatapro\\venv_net\\Lib\\site-packages\\sklearn\\.libs\\vcomp140.dll', 'version': None}, {'user_api': 'blas', 'internal_api': 'openblas', 'num_threads': 16, 'prefix': 'libopenblas', 'filepath': 'D:\\mydatapro\\venv_net\\Lib\\site-packages\\scipy.libs\\libopenblas_v0.3.27--3aa239bc726cfb0bd8e5330d8d4c15c6.dll', 'version': '0.3.27', 'threading_layer': 'pthreads', 'architecture': 'Haswell'}]
Library: openmp
- Type: 未知
- Name: 未知
- Version: None
- Number of threads: 1
- User API: openmp
- Internal API: openmp
- Prefix: libomp
- File: 未知
----------------------------------------
Library: openmp
- Type: 未知
- Name: 未知
- Version: None
- Number of threads: 16
- User API: openmp
- Internal API: openmp
- Prefix: vcomp
- File: 未知
----------------------------------------
Library: openmp
- Type: 未知
- Name: 未知
- Version: None
- Number of threads: 16
- User API: openmp
- Internal API: openmp
- Prefix: libiomp
- File: 未知
----------------------------------------
Library: openmp
- Type: 未知
- Name: 未知
- Version: None
- Number of threads: 1
- User API: openmp
- Internal API: openmp
- Prefix: libiomp
- File: 未知
----------------------------------------
Library: blas
- Type: 未知
- Name: 未知
- Version: 0.3.23.dev
- Number of threads: 16
- User API: blas
- Internal API: openblas
- Prefix: libopenblas
- File: 未知
----------------------------------------
Library: openmp
- Type: 未知
- Name: 未知
- Version: None
- Number of threads: 16
- User API: openmp
- Internal API: openmp
- Prefix: vcomp
- File: 未知
----------------------------------------
Library: blas
- Type: 未知
- Name: 未知
- Version: 0.3.27
- Number of threads: 16
- User API: blas
- Internal API: openblas
- Prefix: libopenblas
- File: 未知
----------------------------------------
D:\mydatapro\venv_net\Lib\site-packages\threadpoolctl.py:1214: RuntimeWarning:
Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at
the same time. Both libraries are known to be incompatible and this
can cause random crashes or deadlocks on Linux when loaded in the
D:\mydatapro\venv_net\Lib\site-packages\threadpoolctl.py:1214: RuntimeWarning:
Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at
the same time. Both libraries are known to be incompatible and this
can cause random crashes or deadlocks on Linux when loaded in the
same Python program.
Using threadpoolctl may cause crashes or deadlocks. For more
information and possible workarounds, please see
https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md
warnings.warn(msg, RuntimeWarning)
可以看到,环境中确实同时加载了多个 OpenMP 运行时库,包括 libomp(LLVM/Clang 的 OpenMP)和 libiomp(Intel 的 OpenMP)。这正是导致警告的原因。此外,还有一些其他库(如 vcomp 和 libopenblas)也使用了 OpenMP。但是我不知道怎么解决这个问题了。