Cascade of phase transitions in the training of energy-based models
Authors: Dimitrios Bachtis, Giulio Biroli, Aurélien Decelle, Beatriz Seoane
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We start with an analytical investigation using simplified architectures and data structures, and end with numerical analysis of real trainings on real datasets. Our study tracks the evolution of the model s weight matrix through its singular value decomposition, revealing a series of phase transitions associated to a progressive learning of the principal modes of the empirical probability distribution. |
| Researcher Affiliation | Academia | 1Laboratoire de Physique de l Ecole Normale Supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France. 2Departamento de Física Teórica I, Universidad Complutense, 28040 Madrid, Spain. 3Université Paris-Saclay, CNRS, INRIA Tau team, LISN, 91190, Gif-sur-Yvette, France. |
| Pseudocode | No | The paper describes algorithms and processes in prose and mathematical equations but does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | The code to reproduce the experiments is freely available in https://github.com/Aurelien Decelle/Torch RBM. |
| Open Datasets | Yes | For this purpose, we will consider 3 real data sets: (i) The Human Genome Dataset (HGD), (ii) MNIST and (iii) Celeb A, see details in the SI D. ... [34] 1000 Genomes Project Consortium et al. A global reference for human genetic variation. Nature, 526(7571):68, 2015. |
| Dataset Splits | No | The paper mentions a "training dataset D = {v(1), , v(M)}" but does not specify explicit training, validation, and testing splits or percentages for these datasets. Table 1 lists hyperparameters but not dataset splits. |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for the experiments (e.g., GPU/CPU models, memory specifications). It states in the checklist that "the experiments do not need particular resources and can be trained on personal laptop." |
| Software Dependencies | No | The paper mentions the "python library skimage" and the GitHub repository name "Torch RBM" which implies Python and PyTorch, but it does not specify exact version numbers for these or other software dependencies. |
| Experiment Setup | Yes | The hyperparameters used for each training (no. of visible and hidden units Nv and Nh, respectively, learning rate ϵ or minibatch size Nms) are given in Table 1. |