Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models
Authors: Yong Zhong, Hongtao Liu, Xiaodong Liu, Fan Bao, Weiran Shen, Chongxuan Li
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, with various pretrained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data and achieves competitive results to the state-of-the-art methods. Our implementation is available at https://github.com/ML-GSAI/Reg-ADA-APA. |
| Researcher Affiliation | Academia | 1Gaoling School of AI, Renmin University of China, Beijing, China 2Beijing Key Lab of Big Data Management and Analysis Methods, Beijing, China 3Department of Computer Science Technology, Tsinghua University, Beijing, China {yongzhong, ht6, xiaodong.liu}@ruc.edu.cn, bf19@mails.tsinghua.edu.cn, {shenweiran, chongxuanli}@ruc.edu.cn |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our implementation is available at https://github.com/ML-GSAI/Reg-ADA-APA. |
| Open Datasets | Yes | FFHQ (Karras et al., 2019), LSUN CAT (Yu et al., 2015), and CIFAR-10 (Krizhevsky et al., 2009) datasets |
| Dataset Splits | No | No explicit specification of training/validation/test dataset splits with percentages, sample counts, or clear predefined split references for reproducibility of data partitioning. While 'training subsets' are created, a dedicated 'validation' split from the full dataset is not detailed. |
| Hardware Specification | Yes | A single experiment can be completed on 8 NVIDIA 2080Ti GPUs. Part of the computing resources supporting this work, totaled 720 A100 GPU hours, were provided by High-Flyer AI. |
| Software Dependencies | No | The paper mentions software components like PyTorch (via pytorch.org/vision/stable/models.html) and official code for ADA and APA, but does not specify version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Some parameters are shown in Tab. 3. The weight parameter λ controls the strength of our regularization term. We choose the weighting hyperparameter λ by performing grid search over [50, 20, 10, 5, 4, 2, 1, 0.5, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005] for FFHQ and LSUN CAT, and [1, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005, 0.00001, 0.000005] for CIFAR-10 according to FID following prior work (Karras et al., 2020a). Other parameters remain the same settings as ADA (Karras et al., 2020a) and APA (Jiang et al., 2021). |