Proximal Mapping for Deep Regularization
Authors: Mao Li, Yingyi Ma, Xinhua Zhang
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluated the empirical performance of Prox Net for multiview learning on supervised learning (two tasks) and unsupervised learning (crosslingual word embedding). Prox LSTM was evaluated on sequence classification. We used the Ray Tune library to select the hyper-parameters for all baseline methods [41]. Details on data preprocessing, experiment setting, optimization, and additional results are given in Appendix G. Here we highlight the major results and experiment setup. |
| Researcher Affiliation | Academia | Mao Li, Yingyi Ma, Xinhua Zhang Department of Computer Science, University of Illinois at Chicago Chicago, IL 60607 {mli206, yma36, zhangx}@uic.edu |
| Pseudocode | No | The paper does not contain a pseudocode block or a clearly labeled algorithm block. |
| Open Source Code | Yes | All code and data are available at https://github.com/learndeep2019/Prox Net. |
| Open Datasets | Yes | Dataset. We first evaluated Prox Net on a large-scale sketch-photo paired database Sketchy [42]. It consists of 12,500 photos and 75,471 hand-drawn sketches of objects from 125 classes. [...] Dataset. We used the Wisconsin X-ray Micro-Beam Database (XRMB) corpus which consists of simultaneously recorded speech and articulatory measurements from 47 American English speakers and 2357 utterances [43]. [...] Dataset. We obtained 36K pairs of English-German word as training examples from the parallel news commentary corpora [WMT 2012-2018, 46], using the word alignment method from [47] and [48]. [...] Datasets. We tested on four sequence datasets: Japanese vowels [JV, 55] which contains time series data for speaker recognition based on uttered vowels; Human Activity Recognition [HAR, 56] which classifies activity; Arabic Digits [AD, 57] which recognizes digits from speeches; and IMDB [58], a large movie review dataset for sentiment classification. |
| Dataset Splits | No | The paper mentions training and testing splits for some datasets (e.g., Sketchy: '80 pairs from each class to form the training set, and then used the remaining 20 pairs for testing.'; XRMB: '35 speakers for training and 12 speakers for testing.'). For crosslingual word embedding, it states 'A validation set was employed to select the hidden dimension...', but no specific split percentages or counts for this validation set are provided. For the sequence datasets, no specific train/validation/test splits are given. |
| Hardware Specification | Yes | Unless otherwise specified, our implementations were based on Py Torch and all training was conducted on a NVIDIA Ge Force 2080 Ti GPU. All methods were trained using Res Net-18 as the feature extractor. [...] The experiments used the Extreme Science and Engineering Discovery Environment (XSEDE) at the PSC GPU-AI. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research. |
| Software Dependencies | No | The paper mentions using 'Py Torch' and 'Ray Tune library' but does not specify their version numbers or other software dependencies with specific versioning. |
| Experiment Setup | Yes | Prox Net was trained by Adam with a weight decay of 0.0001 and a learning rate of 0.001, with the latter divided by 10 after 200 epochs. The mini-batch size was 100, which, in conjunction with the low dimensionality of proximal layer (d = 20), allows the SVD to be solved instantaneously. At training time, we employed an adaptive trade-off parameter λ, which is defined in (11). We set the hyper-parameters k = 0.5 and α0 = 0.1. All experiments were run five times to produce mean and standard deviation. |