论文阅读 [TPAMI-2022] Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild

论文阅读 [TPAMI-2022] Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild

论文搜索(studyai.com)

搜索论文: Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild

搜索论文: http://www.studyai.com/search/whole-site/?q=Spatiotemporal+Bundle+Adjustment+for+Dynamic+3D+Human+Reconstruction+in+the+Wild

关键字(Keywords)

Cameras; Trajectory; Spatiotemporal phenomena; Bundle adjustment; Videos; Dynamics; Spatiotemporal bundle adjustment; motion prior; temporal alignment; dynamic 3D reconstruction; human model fitting

机器视觉

运动捕捉; 三维人体; 室外场景; 三维重建

摘要(Abstract)

Bundle adjustment jointly optimizes camera intrinsics and extrinsics and 3D point triangulation to reconstruct a static scene.

束调整联合优化相机内部和外部以及3D点三角剖分，以重建静态场景。.

The triangulation constraint, however, is invalid for moving points captured in multiple unsynchronized videos and bundle adjustment is not designed to estimate the temporal alignment between cameras.

然而，三角剖分约束对于在多个非同步视频中捕获的移动点无效，并且束调整不是为了估计摄像机之间的时间对准而设计的。.

We present a spatiotemporal bundle adjustment framework that jointly optimizes four coupled sub-problems: estimating camera intrinsics and extrinsics, triangulating static 3D points, as well as sub-frame temporal alignment between cameras and computing 3D trajectories of dynamic points.

我们提出了一个时空束调整框架，该框架联合优化了四个耦合子问题：估计相机内部和外部，三角化静态3D点，以及相机之间的子帧时间对齐和计算动态点的3D轨迹。.

Key to our joint optimization is the careful integration of physics-based motion priors within the reconstruction pipeline, validated on a large motion capture corpus of human subjects.

我们联合优化的关键是在重建管道中仔细整合基于物理的运动先验，并在大型人体运动捕捉语料库上进行验证。.

We devise an incremental reconstruction and alignment algorithm to strictly enforce the motion prior during the spatiotemporal bundle adjustment.

我们设计了一种增量重建和对齐算法，以严格执行时空束调整过程中的运动先验。.

This algorithm is further made more efficient by a divide and conquer scheme while still maintaining high accuracy.

该算法通过分治方案进一步提高了效率，同时保持了较高的精度。.

We apply this algorithm to reconstruct 3D motion trajectories of human bodies in dynamic events captured by multiple uncalibrated and unsynchronized video cameras in the wild.

我们将该算法应用于在野外由多个未校准和不同步的摄像机捕获的动态事件中重建人体的三维运动轨迹。.

To make the reconstruction visually more interpretable, we fit a statistical 3D human body model to the asynchronous video streams.

为了使重建在视觉上更具可解释性，我们将统计的3D人体模型与异步视频流相匹配。.

Compared to the baseline, the fitting significantly benefits from the proposed spatiotemporal bundle adjustment procedure.

与基线相比，拟和显著受益于拟议的时空束调整程序。.

Because the videos are aligned with sub-frame precision, we reconstruct 3D motion at much higher temporal resolution than the input videos.

由于视频与子帧精度对齐，我们以比输入视频更高的时间分辨率重建三维运动。.

Website: http://www.cs.cmu.edu/~ILIM/projects/IM/STBA…

网站：http://www.cs.cmu.edu/~ILIM/projects/IM/STBA。。.

作者(Authors)

[‘Minh Vo’, ‘Yaser Sheikh’, ‘Srinivasa G. Narasimhan’]