[环境配置]ubuntu18.04+python3.6+caffe-gpu配置记录

Ubuntu18自带python版本为3.6.8，本文流程为

• 系统环境配置caffe（python3.6）
• miniconda3的3.6虚拟环境下运行caffe不报错

1.cuda+miniconda+cudnn+nvidia driver

首先，安装cuda等（之前写过，点这里）

2.系统环境配置caffe

2.1 安装caffe

根据caffe官网，版本高于17.04的ubuntu可以直接安装预编译的caffe，不用分别安装依赖了。
在命令行输入(不是conda里)

sudo apt install caffe-cuda #Gpu版本
sudo apt install caffe-cpu #Cpu版本

2.2 下载源码

cd到准备好的文件夹中，从github上下载caffe的源码

cd ***
git clone https://github.com/BVLC/caffe.git

2.3修改MakeFile.config

cd caffe
sudo cp Makefile.config.example Makefile.config
sudo vim Makefile.config #没有图形界面
sudo gedit Makefile.config #有图形界面

有图形界面直接修改，没有的话使用vim（按i之后修改，按esc退出修改模式，按：wq！保存退出）
修改位置如下

• 第4行

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

• 第22行

# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3

• 第25行

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
CUSTOM_CXX := g++

• 第35行

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH :=-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

• 第47行

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS:=open

• 第68行

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include

• 第79行

一定要检测是否有对应的文件夹。按照自己电脑python的路径来写

# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := /usr/include/python3.6m \
                 /usr/lib/python3/dist-packages/numpy/core/include

• 第95行

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

• 第103行

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
USE_NCCL := 1

2.4 修改Makefile

还是用vim或者gedit打开Makefile（sudo）,第425行

NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

2.5 安装caffe需要的包

cd python
for req in $(cat requirements.txt); do pip install $req; done

2.6 编译caffe

cd ..
make all -j48 #j后面的数字代表处理器的核数

此时可能报错nccl.h

compilation terminated.
Makefile:592: recipe for target '.build_release/examples/cpp_classification/classification.o' failed
make: *** [.build_release/examples/cpp_classification/classification.o] Error 1
In file included from ./include/caffe/parallel.hpp:19:0,
                 from ./include/caffe/caffe.hpp:13,
                 from tools/upgrade_net_proto_text.cpp:10:
./include/caffe/util/nccl.hpp:5:18: fatal error: nccl.h: 没有那个文件或目录
 #include <nccl.h>

解决方法：

cd **** #准备好存放nccl的文件夹
git clone https://github.com/NVIDIA/nccl.git
sudo make install -j48
sudo ldconfig

最后一步创建动态链接非常重要

没有报错的话就继续，如果有报错需要重新make一定要先执行make clean

2.7 编译pycaffe

make pycaffe -j48

没有报错即成功

2.8 检查是否编译成功

***@*** : ****/caffe/python$ python3
Python 3.6.8 (default, Oct  7 2019, 12:59:55) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe

没有报错即成功

错误1：

ImportError: dynamic module does not define init function (init_caffe)

如果在caffe/python下直接进入python，python版本为2.7，就会出现这个错误，进入python3就解决了

错误2：

ImportError: libnccl.so.2: cannot open shared object file: No such file or directory

2.6安装nccl时没有建立动态链接，cd到nccl目录建立动态链接，cd回caffe目录make clean之后重新make all -j48 ，cd python ，make pycaffe -j48，然后重新检查

3. 在3.6虚拟环境中运行caffe工程

3.1 conda 中建立python==3.6.8的虚拟环境

conda create -n py36 python=3.6.8
y
source activate py36

3.2

cd ***/caffe/python
(py36) ****@**** : ****/caffe/python$ python 
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe

错误1：

no mudule named…
pip 安装一下

错误2：

ModuleNotFoundError: No module named ‘skimage’

pip install scikit-image

错误3：

from google.protobuf.internal import enum_type_wrapper
ModuleNotFoundError: No module named ‘google’

pip install google
pip install protobuf

3.3修改项目的caffe路径，使用