Dual Associated Encoder for Face Restoration
Authors: YU-JU TSAI, Yu-Lun Liu, Lu Qi, Kelvin C.K. Chan, Ming-Hsuan Yang
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the effectiveness of DAEFR on both synthetic and real-world datasets, demonstrating its superior performance in restoring facial details. ... We evaluate our method with extensive experiments and ablation studies and demonstrate its effectiveness with superior quantitative and qualitative performances. |
| Researcher Affiliation | Collaboration | Yu-Ju Tsai1 Yu-Lun Liu2 Lu Qi1 Kelvin C.K. Chan3 Ming-Hsuan Yang1,3 1UC Merced 2National Yang Ming Chiao Tung University 3Google Research |
| Pseudocode | No | The paper describes its methods in detail through text and diagrams (Figure 1 and Figure 2) but does not provide explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Project page: https://liagm.github.io/DAEFR/. |
| Open Datasets | Yes | We train our model on the FFHQ dataset (Karras et al., 2019), which contains 70,000 high-quality face images. ... Testing Dataset. Our evaluation follows the settings in prior literature (Wang et al., 2022b; Gu et al., 2022; Zhou et al., 2022), and includes four datasets: the synthetic dataset Celeb A-Test and three real-world datasets, namely, LFW-Test, WIDER-Test, and BRIAR-Test. Celeb A-Test comprises 3,000 images selected from the Celeb A-HQ testing partition (Karras et al., 2018). LFW-Test consists of 1,711 images representing the first image of each identity in the validation part of the LFW dataset (Huang et al., 2008). Zhou et al. (Zhou et al., 2022) collected the WIDER-Test from the WIDER Face dataset (Yang et al., 2016), which comprises 970 face images. Lastly, the BRIAR-Test contains 2,120 face images selected from the BRIAR dataset (Cornett et al., 2023). |
| Dataset Splits | Yes | We train our model on the FFHQ dataset (Karras et al., 2019), which contains 70,000 high-quality face images. ... To generate the paired data, we synthesize the degraded images on the FFHQ dataset using the same procedure as the compared methods (Li et al., 2018; 2020; Wang et al., 2021; 2022b; Gu et al., 2022; Zhou et al., 2022). ... Testing Dataset. Our evaluation follows the settings in prior literature (Wang et al., 2022b; Gu et al., 2022; Zhou et al., 2022), and includes four datasets: the synthetic dataset Celeb A-Test and three real-world datasets, namely, LFW-Test, WIDER-Test, and BRIAR-Test. Celeb A-Test comprises 3,000 images selected from the Celeb A-HQ testing partition (Karras et al., 2018). LFW-Test consists of 1,711 images representing the first image of each identity in the validation part of the LFW dataset (Huang et al., 2008). ... We conduct the quantitative experiments on the LFW dataset (Huang et al., 2008) of the face recognition task with the official Arc Face (Deng et al., 2019) model with Verification performance (%) as our evaluation metric. The degradation parameters ranging from 10,000 to 40,000 correspond to varying levels of degradation. ... We utilize the LFW (Huang et al., 2008) Face recognition dataset s validation split, comprising 12,000 images. |
| Hardware Specification | Yes | The proposed method is implemented in Pytorch and trained with eight NVIDIA Tesla A100 GPUs. |
| Software Dependencies | No | The paper mentions 'Pytorch' and 'Adam optimizer' but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | In our implementation, the size of the input face image is 512 512 3, and the size of the quantized feature is 16 16 256. The codebooks contain N = 1,024 code items, and the channel of each item is 256. Throughout the entire training process, we employ the Adam optimizer (Kingma & Ba, 2014) with a batch size 32 and set the learning rate to 1.44 10 4. The HQ and LQ reconstruction codebook priors are trained for 700K and 400K iterations, respectively, and the feature association part is trained for 70K iterations. Finally, the feature fusion and code prediction stage is trained for 100K iterations. ... we set λper = 1.0 and λadv = 0.8 in our setting. ... we set the L2 loss weight λfeat = 10 in our experiments. ... In our experiment setting, we randomly sample σ, r, δ, and q from [0.1, 15], [0.8, 30], [0, 20], and [30, 100], respectively. |