Stable and Efficient Representation Learning with Nonnegativity Constraints
Authors: Tsung-Han Lin, H. T. Kung
ICML 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we use a multi-layer deep architecture for representation learning, where we use K-means for feature learning and NOMP for representation encoding. |
| Researcher Affiliation | Academia | Tsung-Han Lin THLIN@EECS.HARVARD.EDU H. T. Kung KUNG@HARVARD.EDU School of Engineering and Applied Sciences, Harvard University, Cambridge MA, USA |
| Pseudocode | Yes | NOMP iterates the following steps for up to k rounds: 1. Initialize the residual vector r(0) = x and round number l = 1. Select the atom dil that has the highest positive correlation with the residual, il = arg maxi di, r(l 1) . Terminate if the largest correlation is less than or equal to zero. 2. Approximate the coefficients of the selected atoms by nonnegative least squares. z(l) = arg minz x Pl h=1 dihzih 2 s.t. zih 0 3. Compute the new residual r(l) = x Dz(l). Increment l by 1. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described, nor does it contain explicit statements about code release or links to repositories. |
| Open Datasets | Yes | We use a dictionary learned from 6 6 images patches from CIFAR-10. We first evaluate NOMP using the full CIFAR-10 dataset... The CIFAR-100 dataset is one of such datasets... Finally, we evaluate NOMP on the STL-10 dataset... |
| Dataset Splits | Yes | We choose k as 20 and 5 for NOMP and OMP, respectively, in the experiments.9 This choice is cross-validated over k = {1, 3, 5, 10, 20}. We report accuracy from the 5-fold cross validation on the training set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cluster specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers required to replicate the experiment. |
| Experiment Setup | Yes | Standard preprocessing steps are applied on image data to generate data vectors for layer-1. These include mean subtraction, contrast normalization , and ZCA-whitening, followed by sign-splitting as described in Section 3.2. We encode 6 6 patches, pool the sparse codes over the four quadrants of an image, and concatenate the four representations. We choose k as 20 and 5 for NOMP and OMP, respectively, in the experiments. The patch sizes and features sizes are 6 6, 9 9, 15 15, and 3200, 6400, 6400 for the three layers, respectively. we employ a linear classifier (L2-SVM). |