【深度学习MVS系列论文】MVSNet: Depth Inference for Unstructured Multi-view Stereo

MVSNet: Depth Inference for Unstructured Multi-view Stereo

ECCV 2018

核心思路

1. extract deep visual image features
2. build 3D cost column upon the reference camera frustum via the differential equations homography warping
3. apply 3D convolution to regularize and regress the initial deep map
4. refine with the reference image

input: one reference image + several source images

output: depth for the reference image

key sight: differential equations homography warping operation, encode camera geometries in the network to build the 3D cost volumes from 2D image features and enables the end-to-end training

contribution:

• encode the camera parameters as the differential equations homography to build the 3D cost volume upon the camera frustum
• bridge the 2D feature extraction and 3D cost regularization networks
• decouple the MVS reconstruction to smaller problems of per-view depth map estimation
• variance-based metric that maps multiple features into one cost feature to adopt arbitrary number of views
• 3D cost volumn is built upon the camera frustum instead of the regular Euclidean space
• decouple MVS reconstruction to per-view depth map estimation

相关工作

• point cloud reconstruction: propagation strategy, gradually density the reconstruction

  • 无法并行化，耗费时间长

• volumetric reconstruction: 将3D空间划分为网格，判断点是否附着在surface

  • 离散化误差，高内存消耗

• depth map reconstruction

传统方法：</