Regularization for Unsupervised Deep Neural Nets
Authors: Baiyang Wang, Diego Klabjan
AAAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we compare the performance of these methods based on their likelihood and classification error rates for various pattern recognition data sets. |
| Researcher Affiliation | Academia | Baiyang Wang, Diego Klabjan Department of Industrial Engineering and Management Sciences, Northwestern University, 2145 Sheridan Road, C210 Evanston, Illinois 60208 |
| Pseudocode | Yes | Algorithm 1. (Partial Drop Connect) |
| Open Source Code | No | The paper does not provide a link to its source code or explicitly state that it is open-source. |
| Open Datasets | Yes | We compare the empirical performance of the aforementioned regularization methods on the following data sets: MNIST, NORB (image recognition); 20 Newsgroups, Reuters21578 (text classification); ISOLET (speech recognition). |
| Dataset Splits | Yes | The MNIST data set consists of 282 pixels of handwritten 0-9 digits. There are 50,000 training examples, 10,000 validation and 10,000 testing examples. |
| Hardware Specification | Yes | All results are obtained using Ge Force GTX TITAN X in Theano. |
| Software Dependencies | No | The paper mentions 'Theano' but does not specify its version number or any other software dependencies with version details. |
| Experiment Setup | Yes | The number of pretraining epochs is 100 per layer and the number of finetuning epochs is 300, with a finetuning learning rate of 0.1. For L2 + AL1, SNP, and INP which need re-calibration, we cut the 100 epochs into two halves (4 quarters for INP). For regularization parameters, we apply the following ranges: p = 0.8 0.9 for DO/DC/SNP/INP; λ = 10 5 10 4 for L2, similar to Hinton (2010); μ = 0.01 0.1 for L2 +AL1; p0 = 0.5, q = 0.7 0.9 or the reverse for PDO/PDC. |