ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field
Authors: Zhangkai Ni, Peiqi Yang, Wenhan Yang, Hanli Wang, Lin Ma, Sam Kwong
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive experimental results demonstrate that Col Ne RF outperforms state-of-the-art sparse input generalizable Ne RF methods. Furthermore, our approach exhibits superiority in finetuning towards adapting to new scenes, achieving competitive performance compared to per-scene optimized Ne RF-based methods while significantly reducing computational costs. Our code is available at: https://github.com/eezkni/Col Ne RF. |
| Researcher Affiliation | Collaboration | Zhangkai Ni1, Peiqi Yang1, Wenhan Yang2, Hanli Wang1*, Lin Ma3, Sam Kwong4 1 Department of Computer Science and Technology, Tongji University, China 2 Peng Cheng Laboratory, China 3 Meituan, China 4 Department of Computer Science, City University of Hong Kong, Hong Kong {zkni, 2233007, hanliwang}@tongji.edu.cn, yangwh@pcl.ac.cn, forest.linma@gmail.com, cssamk@cityu.edu.hk |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at: https://github.com/eezkni/Col Ne RF. |
| Open Datasets | Yes | We evaluate our method on two datasets: DTU (Jensen et al. 2014) and LLFF (Mildenhall et al. 2019). |
| Dataset Splits | No | The paper mentions following evaluation protocols of other papers for DTU and LLFF datasets, and discusses ray sampling during training ('sample 128 training rays per iteration'), but it does not explicitly provide the training/validation/test dataset splits with percentages or sample counts for reproducibility within its text. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU model, CPU, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using a 'pre-trained encoder Res Net34 (He et al. 2016)' but does not list specific software dependencies with version numbers (e.g., Python version, PyTorch/TensorFlow version, CUDA version). |
| Experiment Setup | Yes | In line with Pixel Ne RF, we sample 128 training rays per iteration. To boost controllability, we randomly emit 112 rays and designate the final 16 of them as reference rays. The remaining 16 rays of all 128 rays are sampled from regions adjacent to reference rays. These freshly sampled rays share the same camera parameters and origin with the reference rays, but exhibit an offset of up to 7 pixels on the pixel plane. These last 32 rays are used as paired adjacent rays for geometry ray regularization. For the training of 3-view and 6-view, we set the batch size (BS) to 3, and for the 9-view training, BS is set to 2. We maintain a fixed learning rate of 1e-4 throughout our training process. The loss weights λ1 and λ2 are set to 1e-4 and 2e-4 respectively throughout our experiments. We set τ = 0.1 in our experiments. |