On Finite-Sample Identifiability of Contrastive Learning-Based Nonlinear Independent Component Analysis
Authors: Qi Lyu, Xiao Fu
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments are used to validate the theorems. In this section, we validate our theoretical results using synthetic and real data experiments. Fig. 2 shows the n ICA performance in terms of MI using different network width R s under N= 5,000 and N= 10,000. Fig. 4 shows the averaged classification errors using SVM and logistic regression, respectively. |
| Researcher Affiliation | Academia | Qi Lyu 1 Xiao Fu 1 1School of EECS, Oregon State University, Corvallis, OR, United States. Correspondence to: Xiao Fu <xiao.fu@oregonstate.edu>, Qi Lyu <lyuqi@oregonstate.edu>. |
| Pseudocode | No | The paper does not contain any sections, figures, or blocks explicitly labeled as "Pseudocode" or "Algorithm". |
| Open Source Code | No | The paper does not contain any statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | In addition to synthetic data, we also use the EEG eye dataset from the UCI repository (Dua & Graff, 2017). |
| Dataset Splits | No | We use 12,000 data samples as the training set to learn h( ). Then, we train simple classifiers (i.e., SVM and logistic regression) using bs = h(x). The classifiers are tested using 3000 test samples. The paper specifies training and test sets, but no explicit validation set. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU/CPU models, memory, or cloud computing instances) used for running the experiments. |
| Software Dependencies | No | For optimization, we use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate 5 10 4. We model h( ) and phi( ) using three-hidden-layer neural networks. The activation function is Re LU. we estimate the MI using kernel density estimation (Kozachenko & Leonenko, 1987). We compute the MI between each of the recovered by and the ground truth s s. Then, we use the Hungarian algorithm (Kuhn, 1955) to fix the permutation ambiguity. No specific version numbers for software are provided. |
| Experiment Setup | Yes | We model h( ) and phi( ) using three-hidden-layer neural networks. We test various R s (i.e., the number of hidden neurons) for each layer, where R {4, 8, 16, 32, 64, 128, 256, 512}. The activation function is Re LU. For optimization, we use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate 5 10 4. |