EigenNet: Towards Fast and Structural Learning of Deep Neural Networks
Authors: Ping Luo
IJCAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Both the training wall-clock time and number of updates are reduced by using Eigen Net, compared to stochastic gradient descent on various datasets, including MNIST, CIFAR-10, and CIFAR-100. |
| Researcher Affiliation | Academia | Ping Luo Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong pluo@ie.cuhk.edu.hk |
| Pseudocode | Yes | Algorithm 1 summarizes the training procedure. |
| Open Source Code | No | The paper does not provide any explicit statement about releasing its source code or a link to a code repository for the described methodology. |
| Open Datasets | Yes | Eigen Net is extensively evaluated on three datasets, including an ablation study on MNIST [Larochelle et al., 2007], a compatibility study with the recent advanced deep architectures, NIN [Lin et al., 2014] and Inception [Ioffe and Szegedy, 2015], on the CIFAR-10 and CIFAR-100 datasets [Krizhevsky, 2009]. |
| Dataset Splits | No | The paper specifies training and test set sizes for MNIST ('12,000 randomly rotated digit images for training and 50,000 images for test') but does not explicitly mention a separate validation dataset split for any of the experiments. |
| Hardware Specification | No | The paper states that 'All models are trained by splitting a batch of 256 samples on 8 GPUs' but does not specify the model or type of GPU, or any other hardware components like CPU or memory. |
| Software Dependencies | No | The paper mentions deep learning frameworks and optimizers but does not provide specific version numbers for any software dependencies like Python, PyTorch, or TensorFlow libraries. |
| Experiment Setup | Yes | For all algorithms, we initialize network parameters by sampling from a standard normal distribution and employ a batch of 64 samples for each update, where the parameters are updated with a momentum value of 0.9. In particular, for Eigen Net, the whitening interval γ, the number of mini-batches τ, and the pruning threshold ρ are selected from {100,200,500,1000}, {10,20,30}, and {0.01,0.001}, respectively. |