Apex安装踩坑

解决Apex安装与配置问题
最新推荐文章于 2024-09-28 23:47:07 发布
原创 最新推荐文章于 2024-09-28 23:47:07 发布 · 2.8k 阅读 · 3 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#pytorch #centos #yum

在CentOS系统中安装Apex时遇到了一系列问题，包括pytorch与cuda版本对应问题，需要确保两者匹配。接着是gcc版本升级，由于默认版本为4.8.5，需通过yum安装指定版本的gcc。在升级gcc过程中，遭遇了nvidia-docker的GPG验证错误，通过关闭GPG验证机制解决。最后，apex安装成功后，在使用时发现EncdecMultiheadAttn模块找不到，通过修改安装命令并指定具体路径解决了该问题。

apex centos下安装踩坑

apex

1.0 pytorch版本、cuda版本的对应问题

安装apex需要机器上的pytorch版本和cuda是对应的版本，否则报错。
cuda版本查看 cat /usr/local/cuda/version.txt
https://pytorch.org/get-started/previous-versions/

2.0 gcc版本问题

centos默认gcc4.8.5，不满足apex需要的gcc版本大于5.0。可以使用yum安装并指定gcc版本
笔者参考 https://www.cnblogs.com/jixiaohua/p/11732225.html
若scl enable devtoolset-8 bash 命令失败，gcc 版本没变，那么直接去/opt/rh/devtoolset-8 下面enable 即可

2.1 yum更新gcc过程中遇到nvidia-docker的GPG验证问题

yum install centos-release-scl
命令报错：
repomd.xml signature could not be verified for nvidia-docker
参考
https://blog.youkuaiyun.com/u012560213/article/details/101430549
关闭GPG key的验证机制解决

3.0 apex安装后，发现EncdecMultiheadAttn module找不到的问题

quickstart上的安装命令不得行，需要uninstall，然后具体指定，使用连接上的命令
https://github.com/NVIDIA/apex/tree/

最低0.47元/天 解锁文章
apex——安装方式和避免踩坑
qq_37844044的博客
01-21 8119
背景：这个库的安装不是像其他的一样的直接使用 pip install XXX的形式，而是使用原始的Git方式 1、apex 这是NVIDIA开发的基于PyTorch的混合精度训练加速神器，能够增加运算速度，并且减少显存的占用 2、安装步骤 1、git clone https://github.com/NVIDIA/apex.git 2、cd apex 3、pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--
华为昇腾pytorch安装踩坑记
shallwei的博客
10-15 1095
华为昇腾pytorch安装踩坑记
tensorflow-1.12支持cuda10.0
01-11
自编译tensorflow： 1.python3.5,tensorflow1.12; 2.支持cuda10.0,cudnn7.3.1,TensorRT-5.0.2.6-cuda10.0-cudnn7.3; 3.支持mkl,无MPI; 软硬件硬件环境：Ubuntu16.04,GeForce GTX 1080 配置信息： hp@dla:~/work/ts_compile/tensorflow$ ./configure WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". You have bazel 0.19.1 installed. Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3 Found possible Python library paths: /usr/local/lib/python3.5/dist-packages /usr/lib/python3/dist-packages Please input the desired Python library path to use. Default is [/usr/local/lib/python3.5/dist-packages] Do you wish to build TensorFlow with XLA JIT support? [Y/n]: XLA JIT support will be enabled for TensorFlow. Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: No OpenCL SYCL support will be enabled for TensorFlow. Do you wish to build TensorFlow with ROCm support? [y/N]: No ROCm support will be enabled for TensorFlow. Do you wish to build TensorFlow with CUDA support? [y/N]: y CUDA support will be enabled for TensorFlow. Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 10.0]: Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-10.0 Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]: 7.3.1 Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-10.0]: Do you wish to build TensorFlow with TensorRT support? [y/N]: y TensorRT support will be enabled for TensorFlow. Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]:/home/hp/bin/TensorRT-5.0.2.6-cuda10.0-cudnn7.3/targets/x86_64-linux-gnu Please specify the locally installed NCCL version you want to use. [Default is to use https://github.com/nvidia/nccl]: Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 6.1,6.1,6.1]: Do you want to use clang as CUDA compiler? [y/N]: nvcc will be used as CUDA compiler. Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Do you wish to build TensorFlow with MPI support? [y/N]: No MPI support will be enabled for TensorFlow. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: Not configuring the WORKSPACE for Android builds. Preconfigured Bazel build configs. You can use any of the below by adding "--config=" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=monolithic # Config for mostly static monolithic build. --config=gdr # Build with GDR support. --config=verbs # Build with libverbs support. --config=ngraph # Build with Intel nGraph support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. Preconfigured Bazel build configs to DISABLE default on features: --config=noaws # Disable AWS S3 filesystem support. --config=nogcp # Disable GCP support. --config=nohdfs # Disable HDFS support. --config=noignite # Disable Apacha Ignite support. --config=nokafka # Disable Apache Kafka support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished 编译： hp@dla:~/work/ts_compile/tensorflow$ bazel build --config=opt --config=mkl --verbose_failures //tensorflow/tools/pip_package:build_pip_package 卸载已有tensorflow： hp@dla:~/temp$ sudo pip3 uninstall tensorflow 安装自己编译的成果： hp@dla:~/temp$ sudo pip3 install tensorflow-1.12.0-cp35-cp35m-linux_x86_64.whl
docker安装
eudemonia
09-09 382
1.版本校验： 目前，CentOS 仅发行版本中的内核支持 Docker。Docker 运行在 CentOS 7 上，要求系统为64位、系统内核版本为 3.10 以上。Docker 运行在 CentOS-6.5 或更高的版本的 CentOS 上，要求系统为64位、系统内核版本为 2.6.32-431 或者更高版本。 命令：uname -r 校验Linux内核版本（3.10以上版本） 从 2017 年 3 月开始 docker 在原来的基础上分为两个分支版本: Docker CE 和 Docker EE。
CentOS 7.6 安装Docker以及安装MySQL，Redis的操作步骤
qq_44723773的博客
08-24 387
安装Docker yum install -y yum-utils device-mapper-persistent-data lvm2 //安装必要工具 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo //设置yum源 yum install -y docker-ce //下载docker systemctl start docker //启动docker
Tensorflow与Python、CUDA、cuDNN的版本对应表
热门推荐
发布工作学习过程中遇到的问题和解决方案
05-02 25万+
TensorFlow​经过测试的构建配置官网说明
【深度学习】TensorFlow（cuda版）安装以及对应第三方库安装流程
qq_42513876的博客
09-28 7904
本教程将指导您如何在Anaconda环境中安装TensorFlow（CUDA版本），并配置所需的三方组件，包括CUDA Toolkit、cuDNN、protobuf等。（参考b站视频：BV1rd4y187nM）
windows apex 踩坑
07-22
确保安装了对应版本的VSCode，并配置好Apex支持。 2. **SDK路径错误**：安装并配置Java Development Kit (JDK) 和 Salesforce CLI时，确保它们的路径正确无误。尤其是如果你在Windows上使用cygwin，文件路径的转换...
MMDetection安装踩坑教程
最新发布
03-18
2. 是否安装`apex`（部分模型需要） 3. 环境变量设置是否正确 ### 五、典型错误案例 **案例**：`ImportError: cannot import name 'sigmoid_focal_loss'` - **原因**：mmcv版本不匹配 - **解决**： ```bash pip ...
配置Tensorflow使用CUDA进行GPU加速(超详细教程)
m0_51302496的博客
04-01 4万+
对于刚使用Tensorflow的友友来说配置环境并使用GPU进行加速也是件令人头疼的事情，纯自己折腾会遇到比较多的坑，所以这里详细介绍一下Tensorflow的环境配置
tensorflow1.12支持cuda10
01-11
自编译tensorflow： 1.python3.5,tensorflow1.12; 2.支持cuda10.0,cudnn7.3.1,TensorRT-5.0.2.6-cuda10.0-cudnn7.3; 3.无mkl支持; 软硬件硬件环境：Ubuntu16.04,GeForce GTX 1080 TI 配置信息： hp@dla:~/work/ts_compile/tensorflow$ ./configure WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". You have bazel 0.19.1 installed. Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3 Found possible Python library paths: /usr/local/lib/python3.5/dist-packages /usr/lib/python3/dist-packages Please input the desired Python library path to use. Default is [/usr/local/lib/python3.5/dist-packages] Do you wish to build TensorFlow with XLA JIT support? [Y/n]: XLA JIT support will be enabled for TensorFlow. Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: No OpenCL SYCL support will be enabled for TensorFlow. Do you wish to build TensorFlow with ROCm support? [y/N]: No ROCm support will be enabled for TensorFlow. Do you wish to build TensorFlow with CUDA support? [y/N]: y CUDA support will be enabled for TensorFlow. Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 10.0]: Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-10.0 Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]: 7.3.1 Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-10.0]: Do you wish to build TensorFlow with TensorRT support? [y/N]: y TensorRT support will be enabled for TensorFlow. Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]://home/hp/bin/TensorRT-5.0.2.6-cuda10.0-cudnn7.3/targets/x86_64-linux-gnu Please specify the locally installed NCCL version you want to use. [Default is to use https://github.com/nvidia/nccl]: Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 6.1,6.1,6.1]: Do you want to use clang as CUDA compiler? [y/N]: nvcc will be used as CUDA compiler. Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Do you wish to build TensorFlow with MPI support? [y/N]: No MPI support will be enabled for TensorFlow. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: Not configuring the WORKSPACE for Android builds. Preconfigured Bazel build configs. You can use any of the below by adding "--config=" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=monolithic # Config for mostly static monolithic build. --config=gdr # Build with GDR support. --config=verbs # Build with libverbs support. --config=ngraph # Build with Intel nGraph support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. Preconfigured Bazel build configs to DISABLE default on features: --config=noaws # Disable AWS S3 filesystem support. --config=nogcp # Disable GCP support. --config=nohdfs # Disable HDFS support. --config=noignite # Disable Apacha Ignite support. --config=nokafka # Disable Apache Kafka support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished 编译： bazel build --config=opt --verbose_failures //tensorflow/tools/pip_package:build_pip_package 卸载已有tensorflow： hp@dla:~/temp$ sudo pip3 uninstall tensorflow 安装自己编译的成果： hp@dla:~/temp$ sudo pip3 install tensorflow-1.12.0-cp35-cp35m-linux_x86_64.whl
Tensorflow与cuda版本关系（附加多个cuda版本安装）
weixin_42206075的博客
02-18 1万+
Tensorflow与cuda版本关系（附加多个cuda版本安装） 多说一句 如果，在网上down的代码用的tf的版本与你本机cuda不一致的话，可以在本机安装多个cuda版本，到时候再根据你自己的项目来选择使用哪个版本的cuda（就我本机win10为栗子） 1.安装cuda，这一步在网上太多教程，随便找个就行（cuda安装教程（windows）） 2.一般来说，如果cuda默认安装，路径应该与我上面贴出来的图中的路径一致：C:\Program Files\NVIDIA GPU Computing To
安装tensorflow的GPU版本（详细图文教程）--CUDA11.6的安装
Aaron的博客
06-14 4万+
TensorFlow是深度学习领域使用最为广泛的一个Google的开源软件库（最初由Google brain team进行开发的内部库，由于它的易用性Google决定把它开源出来）.采取数据流图，用于数值计算.节点——处理数据线——节点间的输入输出关系数据流图中的数据叫做tensor, 表示张量, 即N维数据, tensor在数据流图中流动表示计算的过程, 这也是tensorflow名字的由来.支持多种平台，GPU、CPU、移动设备tensorflow特性:在官网可以下载python3.9并安装好。这里我就
cuda tensorflow
qiqzhang的博客
05-02 386
tensorflow-gpu==1.3 cuda8 tensorflow-gpu==1.5 cuda9
Linux 16.04+Caffe+TensorFlow+CUDA9
宇宙骑士欧老师
12-20 7029
特别说明： 0. Caffe 官网地址：http://caffe.berkeleyvision.org/ 1. TensorFlow官网地址：http://tensorflow.google.cn 1. 本文为作者亲自实验完成，但仅限用于学术交流使用，使用本指南造成的任何不良后果由使用者自行承担，与本文作者无关，谢谢！为保证及时更新，转载请标明出处，谢谢！ 2. 本文使用2017年12月2
cuda tensorflow版本对应_Tensorflow-gpu环境配置
weixin_39836860的博客
12-11 181
查看本机cuda驱动版本：cd C:Program FilesNVIDIA GPU Computing ToolkitCUDA显卡类型是NVIDIACUDA GPUs​developer.nvidia.com驱动下载：NVIDIA 驱动程序下载​www.nvidia.cnCuDA的下载网址：CUDA Toolkit 11.1 Update 1 Downloads​developer.nvidia...
tensorflow gpu对应的cuda版本
qq_18256855的博客
08-03 2568
无
Docker化运行和部署环境
没有网络安全就没有国家安全
08-27 720
本篇文章主要讲解如何Docker化运行和部署环境
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值