LuminAIRe: Illumination-Aware Conditional Image Repainting for Lighting-Realistic Generation
Authors: Jiajun Tang, Haofeng Zhong, Shuchen Weng, Boxin Shi
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present the il Lumination-Aware conditional Image Repainting (Lumin AIRe) task to address the unrealistic lighting effects in recent conditional image repainting (CIR) methods. The environment lighting and 3D geometry conditions are explicitly estimated from given background images and parsing masks using a parametric lighting representation and learning-based priors. These 3D conditions are then converted into illumination images through the proposed physically-based illumination rendering and illumination attention module. With the injection of illumination images, physically-correct lighting information is fed into the lighting-realistic generation process and repainted images with harmonized lighting effects in both foreground and background regions can be acquired, whose superiority over the results of state-of-the-art methods is confirmed through extensive experiments. |
| Researcher Affiliation | Academia | Jiajun Tang1,2 Haofeng Zhong1,2 Shuchen Weng1,2 Boxin Shi1,2 1National Key Laboratory for Multimedia Information Processing 2National Engineering Research Center of Visual Technology School of Computer Science, Peking University {jiajun.tang, hfzhong, shuchenweng, shiboxin}@pku.edu.cn |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | No | To tackle the data shortage issue, we create the first dataset suitable for the Lumin AIRe task, named CAR-LUMINAIRE, with its data preparation process and data sample shown in Fig. 2. ... At last, 52,581 composited images at the resolution of 256 256 are collected, accompanied by parsing mask and normal map annotations, as shown in the data sample of Figure 2. (No public access information for the CAR-LUMINAIRE dataset is provided.) |
| Dataset Splits | No | The paper mentions using a 'test set' for a user study but does not specify exact training/validation/test dataset splits (percentages or counts) or reference predefined splits with citations for reproduction. |
| Hardware Specification | No | Please see supplementary materials for implementation details. (The paper does not provide specific hardware details in the main text; it defers to supplementary materials without providing the actual content in the main paper.) |
| Software Dependencies | No | Please see supplementary materials for implementation details. (The paper does not provide specific software dependencies with version numbers in the main text; it defers to supplementary materials without providing the actual content in the main paper.) |
| Experiment Setup | No | Please see supplementary materials for implementation details. (The paper explicitly defers experimental setup details, such as hyperparameters or specific training settings, to supplementary materials and does not include them in the main text.) |