Efficient Neural Network Training via Forward and Backward Propagation Sparsification
Authors: Xiao Zhou, Weizhong Zhang, Zonghao Chen, SHIZHE DIAO, Tong Zhang
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results on real-world datasets demonstrate that compared to previous methods, our algorithm is much more effective in accelerating the training process, up to an order of magnitude faster. |
| Researcher Affiliation | Collaboration | 1 Hong Kong University of Science and Technology, 2 Tsinghua University xzhoubi@connect.ust.hk, zhangweizhongzju@gmail.com czh17@mails.tsinghua.edu.cn, sdiaoaa@ust.hk, tongzhang@tongzhang-ml.org Jointly with Google Research |
| Pseudocode | Yes | Algorithm 1 Completely Sparse Neural Network Training |
| Open Source Code | Yes | 3. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] |
| Open Datasets | Yes | Extensive experimental results on real-world datasets demonstrate that compared to previous methods, our algorithm is much more effective in accelerating the training process, up to an order of magnitude faster. ... on CIFAR-10 [17] using VGG-16 [35], Res Net-20 [11] and Wide Res Net-28-10 [45] ... on CIFAR-10/100 [17]. ... on a large-scale dataset Image Net [3] with Res Net-50 [11] and Mobile Net V1 [14] |
| Dataset Splits | No | The paper reports "Val Acc(%)" in its tables (e.g., Table 2) and states that experimental configurations are in the appendix ("we postpone the experimental configurations...into appendix."), but the provided text does not explicitly state the dataset splits (e.g., percentages or sample counts for train, validation, and test). |
| Hardware Specification | Yes | The GPU in test is RTX 2080 Ti and the deep learning framework is Pytorch [31]. |
| Software Dependencies | No | The paper mentions "Pytorch [31]" as the deep learning framework, but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | No | The paper states: "Due to the space limitation, we postpone the experimental configurations, calculation schemes on train-cost savings and train-computational time and additional experiments into appendix." Therefore, these details are not present in the provided main text. |