Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Computation and Memory-Efficient Model Compression with Gradient Reweighting
Authors: Zhiwei Li, Yuesen Liao, Binrui Wu, Yuquan Zhou, Xupeng Shi, Dongsheng Jiang, Yin Li, Weizhong Zhang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted extensive experimental validation across various domains. Our approach achieves 50% sparsity and a 1.58 speedup in forward pass on Llama2-7B model with only 6 GB of memory usage, outperforming state-of-the-art methods with respect to both perplexity and zero-shot performance. We conduct extensive experiments in this section, which demonstrate the superior performance of our algorithm. In Section 5.1, we introduce the experimental setups. In Section 5.2, we compare our method with other pruning methods for LLMs and conduct several ablation experiments in Section 5.3. |
| Researcher Affiliation | Collaboration | 1Fudan University, 2Northeastern University, 3Huawei Inc. 4Shanghai Key Laboratory of Intelligent Information Processing |
| Pseudocode | Yes | Algorithm 1 Efficient Pruning with Reweighted Policy Gradient Estimator |
| Open Source Code | Yes | Yes, we will make our experimental code available in the supplementary materials. |
| Open Datasets | Yes | We follow [34], using C4 [40] for training and Wikitext2 [36], PTB [35] for testing, which reflect the generalization of the pruned model. The data used in our experiments is open source. |
| Dataset Splits | Yes | Training Details. We follow [34], using C4 [40] for training and Wikitext2 [36], PTB [35] for testing, which reflect the generalization of the pruned model. |
| Hardware Specification | No | We use gloo [43] as the distributed backend and built an 8-node cluster for training on the ImageNet-1K dataset [41] (each node equipped with a GPU). |
| Software Dependencies | No | Adam [26] is used to optimize structural parameters s with a learning rate of 5e-3 and batch size 8. We use gloo [43] as the distributed backend |
| Experiment Setup | Yes | Training Details. We follow [34], using C4 [40] for training and Wikitext2 [36], PTB [35] for testing, which reflect the generalization of the pruned model. Adam [26] is used to optimize structural parameters s with a learning rate of 5e-3 and batch size 8. Training is conducted for one epoch with frozen model weights. |