Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning
Authors: James Chapman, Lennie Wells, Ana Lawry Aguila
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our algorithms show far faster convergence and recover higher correlations than the previous state-of-the-art on all standard CCA and Deep CCA benchmarks. |
| Researcher Affiliation | Academia | James Chapman*, Ana Lawry Aguila University College London {james.chapman.19, ana.aguila.18}@ucl.ac.uk Lennie Wells* University of Cambridge ww347@cam.ac.uk |
| Pseudocode | Yes | Algorithm 2: DMCCA-EY Loss Function in Python |
| Open Source Code | Yes | To ensure the reproducibility of our work, we have made the following resources publicly available: Code for stochastic CCA and deep CCA experiments: https://github.com/jameschapman19/GEP-EY. For Self-Supervised Learning experiments, our modified version of solo-learn is accessible at: https://github.com/jameschapman19/solo-learn. Standalone Py Torch implementation of the SSL-EY loss function: https://github.com/jameschapman19/SSL-EY. |
| Open Datasets | Yes | We apply stochastic Partial Least Squares (PLS) to an extremely high-dimensional dataset from the UK Biobank (Sudlow et al., 2015) |
| Dataset Splits | No | The paper mentions evaluating on a 'validation set' for various experiments (e.g., 'empirical correlations between the representations on a validation set', 'Top-1 and Top-5 accuracies on the validation set') but does not provide specific details on how this validation set is split or its size/proportion for any dataset. |
| Hardware Specification | Yes | Experiment CPU/GPU Resources Stochastic CCA NVIDIA Ge Force RTX 2080 Ti Deep CCA NVIDIA Ge Force RTX 2080 Ti Deep MCCA NVIDIA Ge Force RTX 2080 Ti Stochastic PLS NVIDIA Ge Force GTX 1650 Ti SSL 4-8 NVIDIA Ge Force RTX 2080 Ti, Quadro RTX 8000 Quadro RTX 6000, or NVIDIA Ge Force GTX 1080 Ti GPU devices |
| Software Dependencies | No | The paper mentions using 'solo-learn' and 'PyTorch' (via pseudocode) but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Parameters: For each method, we searched over a hyperparameter grid using Biewald (2020). Parameter Values minibatch size 5,20,50,100 components 5 epochs 1 seed 1, 2, 3, 4, 5 lr 0.01, 0.001, 0.0001 γ10 0.01,0.1,1,10 |