Convolutional Neural Networks With Low-rank Regularization
Authors: Cheng Tai, Tong Xiao, Yi Zhang, Xiaogang Wang, Weinan E
ICLR 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluated the proposed method on CIFAR10 and ILSVRC12 datasets for a variety of modern CNNs, including Alex Net, NIN, VGG and Google Net with success. For example, the forward time of VGG16 is reduced by half while the performance is still comparable. Empirical success suggests that low-rank tensor decompositions can be a very useful tool for speeding up large CNNs. |
| Researcher Affiliation | Academia | 1The Program in Applied and Computational Mathematics, Princeton University 2Department of Electronic Engineering, The Chinese University of Hong Kong 3Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor |
| Pseudocode | No | The paper presents a theorem that describes the algorithm, but it does not use a structured pseudocode block or algorithm environment. |
| Open Source Code | Yes | For exact specifications of the parameters, the reader may check https://github.com/chengtaipu/lowrankcnn. |
| Open Datasets | Yes | In this section, we evaluate the proposed scheme on the CIFAR-10 and the ILSVRC12 datasets with several CNN models. ... ILSVRC12 (Russakovsky et al., 2015) is a well-known large-scale benchmark dataset for image classification. |
| Dataset Splits | Yes | The learning learning rate is initially set to 0.01 and decreases by a factor of 10 every time the validation error stops decreasing. ... We use the single center crop during the testing stage, and evaluate the performance by the top-5 accuracy on the validation set. |
| Hardware Specification | Yes | Our results in Table 3 are based on Nvidia Titan GPUs and Torch 7 with cudnn backend. |
| Software Dependencies | Yes | Our results in Table 3 are based on Nvidia Titan GPUs and Torch 7 with cudnn backend. |
| Experiment Setup | Yes | The batch size is 100. The learning learning rate is initially set to 0.01 and decreases by a factor of 10 every time the validation error stops decreasing. Some models have dropout units with probability 0.25 inserted after every Re LU. |