解读Towards Unified Depth and Semantic Prediction from a Single Image（1）

解读Towards Unified Depth and Semantic Prediction from a Single Image（1）

很久以前，我分析了最早的运用深度学习估计单目深度的论文，并且简单看了代码

这篇论文把深度估计和语义分割结合起来，是CVPR2015的一篇论文

源代码似乎没有公布，暂且分析论文吧

那么正式开始，

从Abstract可以粗率了解到：

1, Depth estimation and semantic segmentation are two fundamental problems in image understanding

2, The two tasks are strongly correlated and mutually beneficial

 Abstract同时讲述方法的思想：

1, We first use a trained CNN to jointly predict a global layout composed of pixel-wise depth values and   semantic labels

2, To further obtain fine-level details, the image is decomposed into local segments for region-level depth and semantic prediction under the guidance of global layout

3, Utilizing the pixel-wise global prediction and region-wise local prediction, we formulate the inference  problem in a two-layer Hierarchical Conditional Random Field (HCRF) to produce the final depth and  semantic map

接着来看 Introduction，

首先讲了深度估计以及语义分割的联系，

1, While they address different aspects in scene understanding, there exist strong consistencies among the  semantic and geometric properties of image regions.

2, When the information from one task is available, it would provide valuable prior knowledge to guide the other one. 

然后讲了很多学者运用语义分割解决深度估计问题，但是存在一些问题

1, However, these approaches either assume the semantic labels are known, or perform semantic  segmentation to generate the semantic labels

同时，语义分割以及深度估计自身也有一些问题，

1, Since the two tasks are performed sequentially, the errors in the predicted semantic labels are inevitably propagated to the depth results

2, On the other hand, in semantic segmentation, with the increasing availability of RGBD data from additional depth sensors, many methods uses depth as another channel to regularize the segmentation and have  achieved much better performance than using RGB images alone.

下次再来分析！