Understanding Locally Competitive Networks
Authors: Rupesh Srivastava, Jonathan Masci, Faustino Gomez, and Juergen Schmidhuber
ICLR 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on MNIST, CIFAR-10, CIFAR-100 and the Image Net dataset show that promising results are obtained for datasets of varying size and complexity. All experiments in this section are performed on the MNIST (Le Cun et al., 1998) dataset. |
| Researcher Affiliation | Academia | Rupesh Kumar Srivastava, Jonathan Masci, Faustino Gomez & J urgen Schmidhuber Istituto Dalle Molle di studi sull Intelligenza Artiļ¬ciale (IDSIA) Scuola universitaria professionale della Svizzera italiana (SUPSI) Universit a della Svizzera italiana (USI) Lugano, Switzerland {rupesh, jonathan, tino, juergen}@idsia.ch |
| Pseudocode | No | No pseudocode or algorithm blocks are explicitly presented in the paper. The methods are described in narrative text. |
| Open Source Code | No | An implementation of the network in Krizhevsky et al. (2012), with some minor differences (Donahue et al., 2013), is available publicly. For the experiments in this section, the penultimate-layer activations obtained using this model were downloaded from Cloud CV Batra et al. (2013). (Footnote 2: https://github.com/torontodeeplearning/convnet/) |
| Open Datasets | Yes | All experiments in this section are performed on the MNIST (Le Cun et al., 1998) dataset. CIFAR-10 is a dataset of 32 32 color images of 10 classes split into a training set of size 50,000 and testing set of size 10,000 (6000 images per class) (Krizhevsky & Hinton, 2009). The utility of the submasks obtained for a large convolutional network trained on the Image Net Large Scale Visual Recognition Challenge 2012 (ILSVRC-2012) (Deng et al., 2012) dataset is evaluated. |
| Dataset Splits | Yes | The submasks for the entire test set (10K examples) were then extracted and visualized using t-SNE. CIFAR-10 is a dataset of 32 32 color images of 10 classes split into a training set of size 50,000 and testing set of size 10,000 (6000 images per class) (Krizhevsky & Hinton, 2009). The best value of k is obtained using a validation set, though we found that k = 5 with distance weighting usually worked well. |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | For the CIFAR experiments, we used the models described in Goodfellow et al. (2013a) since they use locally competitive activations (maxout), are trained with dropout, and good hyperparameter settings for them are available (Goodfellow et al., 2013b). |
| Experiment Setup | Yes | The batch size for this experiment was 100, which means that each pass over the training set consists of 500 weight updates. minibatch gradient descent with momentum was used for training the networks. The best value of k is obtained using a validation set, though we found that k = 5 with distance weighting usually worked well. |