Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction
Authors: Diwen Wan, Ruijie Lu, Gang Zeng
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficiency and effectiveness of our proposed approach with experiments on three datasets: the synthetic dataset D-Ne RF (Pumarola et al., 2021) with 8 scenes, the real-world dataset Hyper Ne RF (Park et al., 2021a) and Ne RF-DS (Yan et al., 2023). For all experiments, we report the following metrics: PSNR, SSIM (Wang et al., 2004), MS-SSIM, LPIPS (Li et al., 2021a), size (rendering resolution), and FPS (rendering speed). |
| Researcher Affiliation | Academia | Diwen Wan 1 Ruijie Lu 1 Gang Zeng 1 1National Key Laboratory of General Artificial Intelligence, School of IST, Peking University, China. Correspondence to: Gang Zeng <zeng@pku.edu.cn>. |
| Pseudocode | No | The paper contains diagrams of the pipeline and network architecture (Figure 1 and Figure 7) but no explicit pseudocode or algorithm blocks. |
| Open Source Code | No | Please see our project page at https://dnvtmf.github.io/SP_GS.github.io. The paper provides a link to a project page, not a direct link to a source-code repository or an explicit statement of code release for the methodology. |
| Open Datasets | Yes | We demonstrate the efficiency and effectiveness of our proposed approach with experiments on three datasets: the synthetic dataset D-Ne RF (Pumarola et al., 2021) with 8 scenes, the real-world dataset Hyper Ne RF (Park et al., 2021a) and Ne RF-DS (Yan et al., 2023). |
| Dataset Splits | No | The paper mentions 'training data' and 'test views' but does not explicitly provide details about training/validation/test splits with percentages, sample counts, or citations to predefined splits for reproducibility. |
| Hardware Specification | Yes | All experiments are conducted on one NVIDIA V100 GPU with 32GB memory. |
| Software Dependencies | No | The paper mentions 'PyTorch' and 'Adam optimizer' but does not specify their version numbers or any other software dependencies with specific versions. |
| Experiment Setup | Yes | The network is trained for a total of 40k iterations, with the initial 3k iterations training without the deformation network F as a warm-up process to achieve relatively stable positions and shapes. 3D Gaussians in the canonical space will be initialized after the warm-up training, and for the initialization of superpoints, M Gaussians are sampled using the farthest point sampling algorithm, and the canonical positions pc of superpoints are equal to the centers of the sampled Gaussians. Moreover, the Aij of the learnable association matrix A will be initialized as 0.9 if the j-th superpoint is initialized with the i-th 3D Gaussian. Otherwise, Aij will be initialized as 0.1. Before each iteration, we calculate the canonical position of superpoints with Eq. 10. The Adam optimizer (Kingma & Ba, 2015) is employed to optimize our models. For 3D Gaussians, the training strategies are the same as those of 3D-GS unless stated otherwise. For the learnable parameters of F, the learning rate undergoes exponential decay, ranging from 1e-3 to 1e-5. The values for Adam s β are set to (0.9, 0.999). In our experiments, we set L = 10 for γ(pc j) and L = 6 for γ(t). |