Channel Permutations for N:M Sparsity
Authors: Jeff Pool, Chong Yu
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we share an ablation study to show the importance of each part of our search algorithm, experimental results showing correlation between our quality metric and final network accuracy, improved sparse network accuracy using our techniques with insignificant overhead to training time, and the transformation of unstructured to structured sparse workloads. |
| Researcher Affiliation | Collaboration | Jeff Pool NVIDIA jpool@nvidia.com Chong Yu1,2 1Academy for Engineering and Technology, Fudan University 2NVIDIA chongy@nvidia.com |
| Pseudocode | Yes | Our final column permutation generation algorithm, shown in Algorithm 1, maintains the iterative greedy step as phase one. |
| Open Source Code | Yes | Code to use these techniques when generating a 2:4 sparse network is available at https://github.com/NVIDIA/apex/tree/master/apex/contrib/sparsity. |
| Open Datasets | Yes | We use the ILSVRC12 [29] dataset to evaluate Squeeze Net v1.0 [12] and Efficient Net B0 [32] |
| Dataset Splits | Yes | We use the ILSVRC12 [29] dataset to evaluate Squeeze Net v1.0 [12] and Efficient Net B0 [32] |
| Hardware Specification | Yes | Training and fine-tuning uses 8 V100 accelerators, the permutation search uses only one V100. |
| Software Dependencies | No | The paper mentions 'Py Torch' but does not specify a version number for it or any other software dependency. |
| Experiment Setup | No | The paper states 'train each network densely from scratch, using hyperparameters from public repositories' and 'resetting hyperparameters and optimizer state [22]' but does not explicitly provide the specific hyperparameter values or detailed training configurations within the text. |