Ubuntu18.04+caffe+python3.6下编译Faster-RCNN的各种坑

我是在已经编译好Caffe环境下再次编译Faster-RCNN,相当于从新编译了一遍caffe，
首先大体上按照这篇文章作为编译Faster-RCNN的主线：https://blog.youkuaiyun.com/weixin_40368377/article/details/81985023

首先最坑的就是模型下载不下来的问题，需要翻墙，我这边直接给百度云的链接了：

遇到的坑

1编译caffe

/usr/local/cuda/include/cudnn.h:803:27:note: declared here
make: ***[.build_release/src/caffe/solvers/adam_solver.o] Error 1

问题出现的原因：

随着cudnn的更新，原来fasr-rcnn的cu代码已经不再适用，需要将py-faster-rcnn中的cudnn代码进行更新。（caffe最新版本的下载：https://github.com/BVLC/caffe.git）

解决办法

（1）将./include/caffe/util/cudnn.hpp换成最新版的caffe里的cudnn的实现，即相应的cudnn.hpp.
（2）将./include/caffe/layers里的，所有以cudnn开头的文件，例如cudnn_conv_layer.hpp。 都替换成最新版的caffe里的相应的同名文件。
（3）将./src/caffe/layer里的，所有以cudnn开头的文件，例如cudnn_lrn_layer.cu，cudnn_pooling_layer.cpp，cudnn_sigmoid_layer.cu。都替换成最新版的caffe里的相应的同名文件。

参考博客：https://blog.youkuaiyun.com/qq_33618366/article/details/80542759

2 配置文件的修改

我是直接把编译caffe的配置文件拷贝了过来，并且把WITH_PYTHON_LAYER := 1前面的注释去掉，重新编译caffe
如果需要重新配置编译文件，参考博客https://blog.youkuaiyun.com/cc13949459188/article/details/82468987

3 编译出现的问题

编译test

nn/src/caffe/test/test_smooth_L1_loss_layer.cpp

解决办法：

找到test_smooth_L1_loss_layer.cpp文件
屏蔽  //#include "caffe/vision_layers.hpp"

测试python接口是否可用时出现了错误：

>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jcole/Git/caffe/python/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "/home/jcole/Git/caffe/python/caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: /home/jcole/Git/caffe/python/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

原因：

libboost_python.so的版本不匹配，默认python版本为3.5，而boost_python为2.7。

解决方法：

修改make.config文件， 将PYTHON_LIBRARIES := boost_python3 python3.5m注释去掉，并修改为PYTHON_LIBRARIES := boost_python-py35 python3.5m. 重新编译即可。

注意：

python3.5m一定要带上，否则会出现如下错误

CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyNumber_InPlaceRemainder'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyLong_FromLong'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyUnicode_AsUTF8String'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyUnicode_FromFormat'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyExc_ValueError'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyModule_Create2'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyNumber_And'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PySlice_New'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyList_Sort'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyImport_ImportModule'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyExc_OverflowError'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyType_IsSubtype'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyLong_Type'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyObject_SetAttrString'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyList_Insert'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyRun_FileExFlags'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyExc_RuntimeError'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyComplex_Type'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyDict_Type'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyDict_Copy'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyObject_CallFunction'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyUnicode_Type'
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so: undefined reference to `PyDict_Size'

参考博客：https://blog.youkuaiyun.com/zjchenchujie/article/details/89976153

4 源代码采用Python2编写的，转换成Python3需要改变的地方

1 AttributeError: ‘dict’ object has no attribute ‘iteritems’

之所以会出现上述错误是因为python3中已经没有这个属性，直接改为items即可：

result = sorted(classCount.items(), key=operator.itemgetter(1), reverse=True)

参考博客：https://blog.youkuaiyun.com/sinat_35512245/article/details/78639317

2 print函数的问题

因为原项目是python2的，但我的环境是python3，所以需要把所有的print改为print().

3 Cpickle的问题同样是由python版本引起的

import cPickle# python2

import pickle# python3

4 ModuleNotFoundError: No module named ‘generate_anchors’

from generate_anchors import generate_anchaors

改为：

from lib.nrp.generate_anchors import generate_anchaors

5 NameError: name ‘xrange’ is not defined

同样是由python版本引起的，xrange()更换为range()

6 AttributeError: ‘NoneType’ object has no attribute ‘format’

注意“.format”是str变量的属性。其实是自己手抖在修改print()的括号的时候造成的，小问题。

参考博客：https://www.jianshu.com/p/db77100cdd5d

5 “error while loading shared libraries: xxx.so.x” 错误的原因和解决办法

我采用了Anaconda里面的lib,在终端输入：

export LD_LIBRARY_PATH=/usr/local/mysql/lib:$LD_LIBRARY_PATH 

其他方法参考博客：https://blog.youkuaiyun.com/sahusoft/article/details/7388617