Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting
Authors: Huangjie Yu, Anpei Chen, Xin Chen, Lan Xu, Ziyu Shao, Jingyi Yu3152-3160
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we qualitatively and quantitatively evaluate our anisotropic approach for two typical MLP-based IBMR tasks (surface light field rendering and image-based relighting), followed by the evaluation of our GSS-based iterative bandwidth searching scheme. We run our experiments on a PC with 2.2 GHz Intel Xeon 4210 CPU 64GB RAM and Nvidia TITAN RTX GPU. For all the applications, we use a Re LU fully-connected network with 8 layers and 256 channels in different solutions. The length of features is 8192 for SLF and 1024 for relighting. All the models are trained using the Adam (Kingma and Ba 2014) optimizer with a learning rate of 10 4 using Py Torch (Paszke et al. 2019). As shown in Fig. 4, our approach generates high-quality results for various MLP-based neural rendering and relighting tasks. It faithfully recovers texture details and preserves smooth viewing transition under challenging heterogeneous input manifolds.Here we first compare our approach under the neural surface light field rendering task, which maps the spatial and angular coordinates of a surface to the RGB texture output. Let AFFM and IFFM denote our approach with anisotropic Fourier features mapping and the isotropic one from previous method, respectively. For thorough evaluation, we also compare against the state-of-the-art deep surface light field framework (Chen et al. 2018) denoted as D-SLF, and the one using standard MLP without feature mapping denoted as No Mapping. For quantitative evaluation, we adopt the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) as metrics similar to previous methods. As shown in Tab. 1, our approach consistently outperforms the other baselines in terms of all these metrics under both real and synthetic datasets. |
| Researcher Affiliation | Academia | Huangjie Yu1,3,4, Anpei Chen1, Xin Chen1, Lan Xu1, Ziyu Shao1, Jingyi Yu1,2 1 School of Information Science and Technology, Shanghai Tech University 2 Shanghai Engineering Research Center of Intelligent Vision and Imaging 3 University of Chinese Academy of Sciences 4 Shanghai Institute of Microsystem and Information Technology |
| Pseudocode | Yes | We provide the algorithm pseudocode of 2D GSS in Appendix C. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing the source code for its method, nor a link to a code repository. |
| Open Datasets | No | The paper mentions datasets used (e.g., "Ship", "Chicken", "Lucy", "Materials", "Lego", "Tang San Cai", "Beer Can", "kneeling knight imageset") but does not provide specific links, DOIs, or formal citations with author/year for public access to these datasets. |
| Dataset Splits | No | The paper does not provide specific numerical percentages or counts for training, validation, and test splits, nor does it refer to a standard split by citation. |
| Hardware Specification | Yes | We run our experiments on a PC with 2.2 GHz Intel Xeon 4210 CPU 64GB RAM and Nvidia TITAN RTX GPU. |
| Software Dependencies | No | The paper mentions using PyTorch and the Adam optimizer but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | For all the applications, we use a Re LU fully-connected network with 8 layers and 256 channels in different solutions. The length of features is 8192 for SLF and 1024 for relighting. All the models are trained using the Adam (Kingma and Ba 2014) optimizer with a learning rate of 10 4 using Py Torch (Paszke et al. 2019). |