Structural Pruning for Diffusion Models
Authors: Gongfan Fang, Xinyin Ma, Xinchao Wang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical assessment, undertaken across several datasets highlights two primary benefits of our proposed method: 1) Efficiency: it enables approximately a 50% reduction in FLOPs at a mere 10% to 20% of the original training expenditure; 2) Consistency: the pruned diffusion models inherently preserve generative behavior congruent with their pre-trained models. Code is available at https://github.com/Vain F/Diff-Pruning. |
| Researcher Affiliation | Academia | Gongfan Fang Xinyin Ma Xinchao Wang National University of Singapore gongfan@u.nus.edu, maxinyin@u.nus.edu, xinchao@nus.edu.sg |
| Pseudocode | Yes | Algorithm 1 Diff-Pruning |
| Open Source Code | Yes | Code is available at https://github.com/Vain F/Diff-Pruning. |
| Open Datasets | Yes | The efficacy of Diff-Pruning is empirically validated across six diverse datasets, including CIFAR-10 (32 × 32)[21], Celeb A-HQ (64 × 64)[32], LSUN Church (256 × 256), LSUN Bedroom (256 × 256) [57] and Image Net-1K (256 × 256). |
| Dataset Splits | No | The paper uses well-known public datasets, but it does not explicitly state the dataset splits (e.g., percentages, sample counts) for training, validation, or testing within the text. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types) used for the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers. |
| Experiment Setup | Yes | For example, when applying our method to an off-the-shelf diffusion model pre-trained on LSUN Church [57], we achieve an impressive compression rate of 50% FLOPs, with only 10% of the training cost required by the original models, equating to 0.5 million steps compared to the 4.4 million steps of the pre-existing models. |