[Paper note] FlowNet: Learning Optical Flow with Convolutional Networks

最新推荐文章于 2023-10-17 12:30:36 发布

原创最新推荐文章于 2023-10-17 12:30:36 发布 · 1.5k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#cnn #光流 #深度学习

paper-note 专栏收录该内容

20 篇文章

订阅专栏

提出FlowNet，首个使用训练好的卷积神经网络进行光学流估算的方法。引入相关层计算两帧之间的相似性，并通过上采样细化网络输出。实验在多个数据集上验证，包括Middlebury、KITTI、Sintel及自动生成的FlyingChairs数据集。结果显示，FlowNet虽然精度略低于传统算法，但速度明显更快。

                    
                        
                    
                    paper
code
 
Highlight 
First paper to use trained CNN for optical flow estimation
Introduce novel correlation layer
Refine network by upsampling
 
Model 
FlowNetSimple: concatenate two consecutive images.
FlowNetCorr: use correlation layer
Correlation layer 
 
  Calculated between two feature maps

     
      c(x1,x2)=∑o∈[−k,k]×[−k,k]<f1(x1+o),f2(x2+o)>
     
See model picture for an illustration
Refinement 
 
  
Concatenate the upsampled flow prediction and conv feature map
 
Experiment 
Datasets: 
 
  Middlebury
KITTI
Sintel
Flying Chairs (proposed, auto generated)
Loss function: endpoint error – Euclidean distance between the predicted flow vector and GT.
Conclusion 
 
  FlowNet performs a little worse than other OF algorithm, but obviously faster.
Network trained on Flying Chairs (auto generated) data has good generalization ability on natural scenes.

                