HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks
Authors: Jinqi Xiao, Chengming Zhang, Yu Gong, Miao Yin, Yang Sui, Lizhi Xiang, Dingwen Tao, Bo Yuan
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on different datasets and hardware platforms demonstrate the effectiveness of our proposed approach. |
| Researcher Affiliation | Academia | 1 Rutgers University 2 Indiana University 3 Washington State University |
| Pseudocode | No | The paper describes the steps of its method but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any concrete statement or link regarding the public availability of its source code. |
| Open Datasets | Yes | We evaluate the performance of HALOC for compressing different CNN models on CIFAR10, and Image Net (Deng et al. 2009) datasets. |
| Dataset Splits | Yes | we then alternately update the parameters of τi,j and the corresponding selection probability pi,j for the i-th layer...the update of all pi,j s can be simultaneously performed via using the backward propagation on the validation dataset...LCE(P rob) is the cross-entropy loss on the validation dataset |
| Hardware Specification | Yes | We also measure the practical speedups brought by our hardware-aware solution on various computing hardware, including Nvidia Tesla V100, Nvidia Jetson TX2, and ASIC accelerator Eyeriss (Chen, Emer, and Sze 2016). |
| Software Dependencies | No | The paper mentions PyTorch/TensorFlow but does not specify their versions or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Training Details. We use the standard SGD optimizer with Nesterov momentum as 0.9 for model training. The learning rates are initilized as 0.1 and 0.01 for CIFAR-10 and Image Net, respectively, and they are then scaled down by 0.2 every 55 epochs. In addition, batch size and weight decay are set as 256 and 0.0001, respectively. |