The spiked matrix model with generative priors
Authors: Benjamin Aubin, Bruno Loureiro, Antoine Maillard, Florent Krzakala, Lenka Zdeborová
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our theoretical results by illustrating the performance of the spectral algorithms when the spikes come from real datasets. We demonstrate the performance of LAMP on the spiked-matrix model when the spike is taken to be one of the fashion-MNIST images showing considerable improvement over canonical PCA. In Fig. 1 we solve the fixed point equations (12) and plot the MMSE obtained from the fixed point in a heat map... Fig. 3 where we explicitly compare runs of AMP on finite size instances with the results of the asymptotic state evolution... For illustration purposes, we display the behaviour of this spectral method on the spiked Wigner model with spikes coming from the Fashion-MNIST dataset in Fig. 4. |
| Researcher Affiliation | Academia | Université Paris-Saclay, CNRS, CEA, Institut de physique théorique, 91191, Gif-sur-Yvette, France. Laboratoire de Physique de l École Normale Supérieure, PSL University & CNRS & Sorbonne Universités, Paris, France. |
| Pseudocode | Yes | Algorithm 1: AMP algorithm for the spiked Wigner model with single-layer generative prior. |
| Open Source Code | Yes | Note that we provide a demonstration notebook in [37] that compares AMP, LAMP and PCA numerical performances. [37] Benjamin Aubin, Bruno Loureiro, Antoine Maillard, Florent Krzakala, and Lenka Zdeborová. Demonstration codes the spiked matrix model with generative priors. https://github.com/benjaminaubin/StructuredPrior_demo. |
| Open Datasets | Yes | We demonstrate the performance of LAMP on the spikedmatrix model when the spike is taken to be one of the fashion-MNIST images showing considerable improvement over canonical PCA. For illustration purposes, we display the behaviour of this spectral method on the spiked Wigner model with spikes coming from the Fashion-MNIST dataset in Fig. 4. [40] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. |
| Dataset Splits | No | The paper mentions using the Fashion-MNIST dataset but does not provide specific details on how it was split into training, validation, or test sets, nor does it refer to predefined splits or cross-validation. |
| Hardware Specification | Yes | We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research. |
| Software Dependencies | No | The paper mentions a demonstration notebook and implies code usage but does not provide specific version numbers for any software, libraries, or programming languages. |
| Experiment Setup | Yes | Input: Y Rp p and W Rp k: Initialize to zero: (g, ˆv, Bv, Av)t=0. Initialize with: ˆvt=1 = N(0, σ2), ˆzt=1 = N(0, σ2), and ˆct=1 v = 1p, ˆct=1 z = 1k, t = 1. Dots correspond to simulations of PCA (red squares), LAMP (green crosses) for k = 104 and AMP (blue points) for k = 5.103, σ2 = 1. |