【读论文】AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

1. What

For scene construction and novel view synthesis, this paper imposes geometric constraints on Gaussian representing the road and sky regions, leverages 3D templates to initialize the foreground points, and introduces a reflected Gaussian consistency to supervise the unseen sides of the foreground objects. Moreover, it uses residual spherical harmonics for foreground objects. Finally, it achieves sota on Pandaset and KITTI datasets in both construction tasks and novel view synthesis with lateral ego-vehicle trajectory adjustments.

2. Why

Question of PVG: this method does not tackle the simulation of novel scenarios, such as ego-vehicle lane changes and adjusting object trajectories(It can’t edit scenes).

3. How

3.1 Input

A series of N images ($I_i$) taken by a camera with its corresponding intrinsic ($K_i$) and extrinsic ($E_i$) matrices, along with the 3D LiDAR point clouds $L_i$ **and corresponding dynamic objects trajectories $T_i$.

3.2 Background Reconstruction

1. The road and sky regions are decomposed from the rest of the background using semantic masks.

2. By projecting LiDAR points to the image plane at each time step $i$, each Gaussian is assigned to one of the road, sky, and other class.

3. When splatting road and sky Gaussians, these Gaussians are constrained to be flat by minimizing their roll and pitch angles as well as their vertical scale.

4. Finally, the loss was defined as:

   $${\mathcal{L}}_{BG}=(1-\lambda ){\mathcal{L}}_1(I_g,{\hat{I}}_g)+\lambda \mathcal{L}$$