一.简述:
AirVO是一个基于点和线特征的光照稳健和准确的立体视觉测距(VO)系统。它是一个混合VO系统,结合了传统优化技术的效率和基于学习方法的鲁棒性。为了对光照变化具有鲁棒性,将基于学习的特征提取(SuperPoint)和匹配(SuperGlue)方法引入该系统。此外,还为VO提出了一个新的线条处理管道,该管道将二维线条与图像上基于学习的二维点联系起来,从而导致更强大的特征匹配和三角计算。这种新方法提高了VO的准确性和可靠性,特别是在有光照挑战的环境中。通过使用Nvidia TensorRT Toolkit加速CNN和GNN部分,点特征检测和匹配实现了比原始代码快5倍以上。该系统在Nvidia Jetson AGX Xavier(一种低功耗的嵌入式设备)上的运行速度约为15Hz,在笔记本电脑上的运行速度为40Hz。
二.依赖环境:
AirVO 环境依赖的组件非常之多,各组件也都有一些子依赖:
- OpenCV 4.2
- Eigen 3
- Ceres 2.0.0
- G2O (tag:20230223_git)
- TensorRT 8.6.1.6
- CUDA 12.1
- python
- ROS noetic
- Boost
三. 部署安装:
1.1 opencv安装:
1.安装依赖:
apt-get install build-essential
apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
2. 安装opencv:
mkdir ~/opencv && cd ~/opencv
wget "https://github.com/opencv/opencv/archive/4.2.0.zip"
wget "https://github.com/opencv/opencv_contrib/archive/4.2.0.zip"
unzip 4.2.0.zip
unzip 4.2.0.zip.1
mv opencv_contrib-4.2.0 opencv-4.2.0/
cd ~/opencv
mkdir build
cd build
//opencv install path is /usr/local/opencv-4.2.0/
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local/opencv-4.2.0/ ../opencv-4.2.0
make -j7 # runs 7 jobs in parallel(启动线程数)
make install
1.2 测试安装:
cd ~ && git clone https://github.com/opencv/opencv_extra.git
将OPENCV_TEST_DATA_PATH环境变量设置为<路径到 opencv_extra/testdata>。
export OPENCV_TEST_DATA_PATH=~/root/opencv_extra/testdata
执行<cmake_build_dir>/bin/opencv_test_core
cd ~/opencv/build/
bin/bin/opencv_test_core
2.1 Eigen 3 安装:(部分系统默认已安装)
apt-get install libeigen3-dev
3.1 Ceres 2.0.0安装:
1. 安装基础依赖(多数已安装):
apt-get install cmake
apt-get install libgoogle-glog-dev libgflags-dev
apt-get install libatlas-base-dev
apt-get install libeigen3-dev
apt-get install libsuitesparse-dev
2. 安装ceres
wget http://ceres-solver.org/ceres-solver-2.0.0.tar.gz
tar fxvz ceres-solver-2.0.0.tar.gz
mkdir ceres-bin
cd ceres-bin
cmake ../ceres-solver-2.2.0
make -j3
make test
make install
3.2 测试验证:
#bin/simple_bundle_adjuster ../ceres-solver-2.2.0/data/problem-16-22106-pre.txt
wget https://github.com/RainerKuemmerle/g2o/archive/refs/tags/20230223_git.tar.gz
tar fxvz 20230223_git.tar.gz
cd g2o-20230223_git
mkdir build
cd build
cmake ../
make
make install
5. TensorRT 8.4安装:
找个地方解压出来就可以,不需要编译,这里选择:
https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.4.3/tars/tensorrt-8.4.3.1.linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz
解压后,移动到/usr/local/TensorRT-8.4.3.1
tar xzvf TensorRT-8.4.3.1.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz
mv TensorRT-8.4.3.1 /usr/local
添加环境变量到~/.bashrc:
export LD_LIBRARY_PATH=/usr/local/TensorRT-8.4.3.1/lib:$LD_LIBRARY_PATH
export LIBRARY_PATH=/usr/local/TensorRT-8.4.3.1/lib:$LIBRARY_PATH
为了避免其它软件找不到 TensorRT 的库,建议把 TensorRT 的库和头文件添加到系统路径下:
ln -s /usr/local/TensorRT-8.4.3.1/lib/* /usr/local/lib/
ln -s /usr/local/TensorRT-8.4.3.1/include/* /usr/local/include/
6.1 cuda安装:
1. 需要先禁用nouveau驱动,可通过lsmod | grep nouveau查看,如果没有输出,表示禁用。
如果没有禁用,需要禁用:
vim /etc/modprobe.d/nvidia-installer-disable-nouveau.conf (新建)
内容:
blacklist nouveau
options nouveau modeset=0 (需要重启系统?)
通过ubuntu-drivers devices 查看可安装的驱动, 通常安装recommended的。
sudo apt install nvidia-driver-535
安装后,重启系统,查看是否生效:
nvidia-smi
6.2 安装cuda:
通过https://developer.nvidia.com/cuda-toolkit-archive , 选择对应的版本:
这里生成的信息是:
wget https://developer.download.nvidia.com/compute/cuda/11.6.2/local_installers/cuda_11.6.2_510.47.03_linux.run
sudo sh cuda_11.6.2_510.47.03_linux.run
选择:Continue --->accept--->因为之前已经安装了驱动,不选择(去掉510驱动前的[X])--->Choose components to upgrade
安装后,会在/usr/local下生成两个文件夹: cuda(软链)、cuda-11.6
查看版本: /usr/local/cuda/bin/nvcc --version
配置环境变量:vim .bashrc
export CUDA_HOME=/usr/local/cuda-11.6
export LD_LIBRARY_PATH=/usr/local/TensorRT-8.4.3.1/lib:$LD_LIBRARY_PATH:$CUDA_HOME/lib64
export PATH=$PATH:$CUDA_HOME/bin
source .bashrc
6.3 安装cudnn :
通过https://developer.nvidia.com/rdp/cudnn-archive 下载:
选择对应的版本:

下载后,解压:
tar -xvf cudnn-linux-x86_64-8.4.1.50_cuda11.6-archive.tar.xz
mv cudnn-linux-x86_64-8.4.1.50_cuda11.6-archive /usr/local
cd cudnn-linux-x86_64-8.4.1.50_cuda11.6-archive
cp lib/libcudnn* /usr/local/cuda-11.6/lib64
cp include/cudnn* /usr/local/cuda-11.6/include/
7. onnx安装:
pip(3) install onnx
8. 安装配置ROS Noetic:
配置ros源:
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
添加key:
sudo apt install curl # if you haven't already installed curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
安装:
sudo apt update # 更新apt软件源
sudo apt install ros-noetic-desktop-full # 安装ROS
环境配置:
echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc # 设置环境变量
source ~/.bashrc # 使设置生效
安装其他依赖:
sudo apt install python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential
初始化rosdep:
sudo apt install python3-rosdep
sudo rosdep init
rosdep update
执行roscore:
apt-get install libboost-all-dev
10. 安装Glog:
git clone https://github.com/google/glog.git
cd glog
mkdir build
cd build
cmake ../ (依赖cmake 3.22+ ,可能需要升级)
make
make install
11.安装AirVO
mkdir ~/catkin_ws/src -p
cd ~/catkin_ws/src
git clone https://github.com/xukuanHIT/AirVO.git
cd ../
catkin_make
source ~/catkin_ws/devel/setup.bash