【环境搭建】linux上编译安装caffe框架+Makefile.config文件详解

linux上编译安装caffe框架+Makefile.config文件详解

编译安装caffe之前需要按【环境搭建】glog和gflags的安装和【环境搭建】在ubuntu16.04上编译安装ffmpeg和opencv中的内容安装glog，gflags，ffmpeg和openc。caffe源码在GitHub上，下载解压过程不再赘述

1. 下面是安装caffe所需要的部分依赖包

sudo apt-get install libleveldb-dev
sudo apt-get install libsnappy-dev
sudo apt-get install libhdf5-serial-dev
# "-no-install-recommends"参数：只安装主要依赖项（Depends字段中的包），避免安装非必须的文件，从而减小镜像的体积
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install liblmdb-dev

# sudo apt-get remove 是移除apt安装的包

# protobuf-compiler不需要用apt安装，这里选择通过编译安装protobuf
# libgoogle-glog-dev libgflags-dev不需要用apt安装，这里选择通过编译安装glog和gflags
# libopencv-dev不需要用apt安装，这里选择通过编译安装OpenCV
# libopenblas-dev不需要用apt安装，安装OpenCV时通过apt安装了atlas计算库

2. 编译安装protobuf(环境变量和路径的配置内容这里不再赘述)

git clone https://github.com/protocolbuffers/protobuf
./autogen.sh
./configure
make -j4
make install
# 环境变量需要配置PATH，类似ffmpeg

需要注意，如果编译caffe时报很长的关于protobuf的错，极有可能是protobuf版本太高，目前protobuf版本要求小于3.5.1



安装protobuf-3.5.1，执行./autogen.sh时可能会有下面这样的错误

解决方法如下

git clone https://github.com/paulsapps/gmock-1.7.0

将克隆下来的文件夹改名为gmock，并移动到protobuf-3.5.1源码目录下

# 将autogen.sh文件中的如下内容注释掉
if test ! -e gmock/gtest; then
  curl $curlopts -L -O https://github.com/google/googletest/archive/release-1.7.0.zip
  unzip -q release-1.7.0.zip
  rm release-1.7.0.zip
  mv googletest-release-1.7.0 gmock/gtest
fi
# ./autogen.sh可以正常执行

3. 修改Makefile.config文件

cp Makefile.config.example Makefile.config
vim Makefile.config

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1
# 有GPU并安装了cuDNN的情况下去掉注释。仅有CPU的时候，此项保持被注释掉的状态不做更改

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# 仅有CPU的情况下去掉注释。有GPU并安装了cuDNN的时候，此项保持被注释掉的状态不做更改

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# 因为会用到OpenCV所以此项一般去掉注释，并将值改为1
# USE_LEVELDB := 0
# USE_LMDB := 0
# leveldb和lmdb是caffe可以用的两种数据库，这两项一般保持被注释掉的状态不做更改

# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0
#

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# 当需要读取LMDB文件时可以取消注释，一般情况下保持被注释掉的状态不做更改

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3
# OpenCV的版本，现在一般都是OpenCV3，所以此项一般去掉注释即可

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# 

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# CUDA的安装目录，里面需要包含CUDA的bin和lib（lib64）文件夹，没有GPU的情况此项需要被被注释掉

# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61
# 和GPU计算能力有关，没有GPU需要注释掉

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# 如果用的是ATLAS计算库则赋值atlas，MKL计算库则赋值mkl，OpenBlas则赋值open

# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# 所用计算库的头文件和库文件所在文件夹，前者需要包含所用计算库的头文件（一般在/usr/include/atlas(openblas)）中，后者需要包含libatlas.so(libopenbalas.so）这类文件（一般在/usr/lib(lib64)中）

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# 计算库如果不是安装在标准路径则要指明，此项一般保持被注释掉的状态不做更改

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# Matlab相关路径，不配置Matlab接口则此项保持被注释掉的状态不做更改

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
		/usr/lib/python2.7/dist-packages/numpy/core/include
# python安装路径，前者包含许多头文件，后者包含一个numpy文件夹，里面也是许多头文件

# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# python库文件的位置，里面应该包含libpythonX.X.so这类文件
# 对于python3，如果路径里只有libpython3.5m.so是不行的。一定要找即包括3.5m.so，也包括3.5.so的文件夹，否在make报关于protobuf的错

# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1
# 此项一般取消注释

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# caffe用得到的头文件和库文件，因为在编译之前执行了.bash_prefile，里面有相关的路径，所以这里不需要额外添加OpenCV，ffmpeg，glog，gflags这类的头文件和库文件路径。
# Ubuntu下需要添加/usr/include/hdf5/serial和/usr/lib/x86_64-linux-gnu/hdf5/serial（具体路径按自己的情况修改）

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# 没有GPU则此项注释掉

# enable pretty build (comment to see full commands)
Q ?= @
# 不做更改

样例：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the lines after *_35 for compatibility.
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
             -gencode arch=compute_35,code=sm_35 \
             -gencode arch=compute_50,code=sm_50 \
             -gencode arch=compute_52,code=sm_52 \
             -gencode arch=compute_60,code=sm_60 \
             -gencode arch=compute_61,code=sm_61 \
             -gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
# BLAS := atlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/include/atlas
BLAS_LIB := /usr/lib

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python3.5 \
		/usr/lib/python3/dist-packages/numpy/core/include/numpy
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda2
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		$(ANACONDA_HOME)/include/python2.7 \
		$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib/python3.5/config-3.5m-x86_64-linux-gnu
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

4. 编译安装

make -j4
# 如果不执行会导致python import caffe 不成功
make pycaffe
make test -j4
make runtest -j4

caffe编译安装完成

# 在.bash_profile文件中添加caffe的python路径
export PYTHONPATH="/caffe源码所在路径/python":$PYTHONPATH

部分编译时遇到的错误的解决方法

1. 错误一
   解决方法如下：

This error arrises because gflags 2.1 changed the name of the namespace from google to gflags. There are attempts by members of the caffe community to fix this error albeit they are not finalized. You should reassign the namespace from google to gflags as follows.

In files
caffe/include/caffe/common.hpp
caffe/examples/mnist/convert_mnist_data.cpp
Comment out the ifndef
// #ifndef GFLAGS_GFLAGS_H_
namespace gflags = google;
// #endif // GFLAGS_GFLAGS_H_

This should work temporarily. You should fork and occasionally sync your caffe repo with the BVLC/caffe repo on github so that you get the latest updates of the code.

解决方法出处：stackoverflow.

2. 错误二
   
   解决方法如下：

you can fix it by adding “opencv_videoio” to LIBRARIES in the Makefile.

解决方法出处：C3D_github_issues.

3. 错误三

# import caffe报错：No module named skimage.io
sudo apt-get install python-skimage

4. 错误四

# import caffe报错：ImportError: /home/zhangchen/thirdparty-source/C3D/C3D-v1.1/python/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

# 如果要把caffe装在python3上，需要改Makefile中的boost_python，将其改为对应的python3版本

# 我的/usr/lib/x86_64-linux-gnu中关于boost内容如上图
# 修改Makefile文件
# 原：
PYTHON_LIBRARIES ?= boost_python python2.7
# 修改后：
PYTHON_LIBRARIES ?= boost_python-py35 python3.5

结语

如果您有修改意见或问题，欢迎留言或者通过邮箱和我联系。
如果我的文章对您有帮助，转载请注明出处。