Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation
Authors: WANG Jiaxu, Ziyi Zhang, Renjing Xu
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that our model can deliver better geometries, view consistencies, and rendering quality than all counterparts and benchmarks on three datasets in both generalization and finetuning settings, preliminarily proving the potential of the new paradigm for generalizable Ne RF.Extensive experiments were conducted on the Ne RF synthetic dataset Mildenhall et al. (2021), the DTU dataset Jensen et al. (2014), and the Blended MVS dataset Yao et al. (2020). |
| Researcher Affiliation | Academia | Jiaxu Wang1, Ziyi Zhang1, Renjing Xu1 1Hong Kong University of Science and Technology (Guangzhou) jwang457@connect.hkust-gz.edu.cn,{ziyizhang,renjingxu}@hkust-gz.edu.cn |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about open-source code availability or links to a code repository for their proposed method. |
| Open Datasets | Yes | Datasets. We pretrain our model on the train set of DTU Yao et al. (2020), in which we follow the train-test split setting introduced in MVSNe RF. To evaluate the generalization ability of our model, we test the pretrained model on Ne RF Synthetic Dataset Mildenhall et al. (2021), the test set in DTU Dataset and large-scale scenes in Blended MVS Yao et al. (2020). |
| Dataset Splits | No | The paper mentions 'train set' and 'test set' for datasets, and refers to following a 'train-test split setting'. However, it does not explicitly mention a 'validation set' or 'validation split' or any specific details for it. |
| Hardware Specification | Yes | We train our generalizable model on a single RTX3090 GPU for 100k iterations using Adam optimizer with an initial learning rate of 5e-4 and a cosine annealing schedule with annealing α of 0.1. |
| Software Dependencies | No | The paper mentions using 'Adam optimizer' and 'Pytorch' but does not specify their version numbers or the versions of other critical software components like Python or CUDA, which are necessary for reproducible setup. |
| Experiment Setup | Yes | We train our generalizable model on a single RTX3090 GPU for 100k iterations using Adam optimizer with an initial learning rate of 5e-4 and a cosine annealing schedule with annealing α of 0.1. In our experiments, in the training stage, we selected 10 input views to compute the visibility and set the top-k as top-3 to perform visibility-based feature fetching. ... The base is set to 1.35 and we sample 16 points for each ray. For each iteration, the training batch is 512. In the feature-augmented learnable kernel, the k nearest neighbors are selected as 8. ...the finetuning is effectively fast and only consumes 50k iterations. In the first stage of finetuning setting, the features and parameters of networks have the initial learning rate of 1e-5 but the color of each point is trained with 1e-7. |