【读论文】Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle

1. What

A novel point-based approach with a novel Dual-Domain Deformation Model for dynamic scene reconstruction.

Contribution:

1. Gaussian-Flow, which is a novel point-based differentiable rendering approach for dynamic 3D scene reconstruction, setting a new sota for training speed, rendering FPS, and novel view synthesis quality for 4D scene reconstruction.
2. Propose a Dual-Domain Deformation Model for efficient 4D scene training and rendering, which preserves a running speed on par with the original 3DGS with minimum overhead.
3. Can be used for downstream tasks

2. Why

2.1 Introduction

1. NeRF still remains a challenge for high-fidelity real-time rendering.
2. 3DGS has been used on 4D tasks but it significantly lowers the rendering speed of the original 3DGS.

2.2 Related work

Remarkable work

1. Dynamic Neural Radiance Field: dynamic neural scene flow methods have been proposed [27, 30],
2. Accelerated Neural Radiance Field
3. Differentiable Point-based Rendering: PointRF [41], DSS [39], and 3D Gaussians splatting(3DGS) [13].

3. How

3.1 Dual-Domain Deformation Model

Assume that only the rotation $q$, radiance $c$, and position $\mu$ of a 3D Gaussian particle change over time, while the scaling $s$ and opacity $\alpha$ remain constant.

Then, we use a time-dependent attribute residual $D(t)$ to adjust the error between the base attribute S 0 ∈ { μ 0 , c 0 , q 0 } S_{0}\in\{\mu_{0},c_{0},q_{0}\} S0​∈{ μ0​,c0​,q0​} and the attribute at time $t$. This is:

$$S(t)=S_0+D(t),$$

where $D(t)=P_N(t)+$