Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
Authors: Liu Ziyin, Yizhou Xu, Isaac Chuang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theory and experiments demonstrate that a combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning. Full derivations and experimental validations are provided in the appendix. ... See Figure 2 for the emergence of layer and neuron balances in a Re LU network. We train on the MNIST dataset... See Figure 3 for an experiment with deep linear and nonlinear networks. ... See Figure 5-6, where we train a two-layer linear network... |
| Researcher Affiliation | Collaboration | Liu Ziyin1,3, , Yizhou Xu2, , Isaac Chuang1 1Massachusetts Institute of Technology 2 Ecole Polytechnique F ed erale de Lausanne 3NTT Research |
| Pseudocode | No | The paper includes many theoretical derivations and theorems but does not contain any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The experiments are only for demonstration and are straightforward to reproduce following the description. |
| Open Datasets | Yes | For Figure 2, we train on the MNIST dataset... For Figure 1, we train Res Net18 on CIFAR 10... Figure 4: Alignment of representations of two Vi T models pretrained on Image Net. |
| Dataset Splits | Yes | A.4 Unversal Representation Learning in MLP: For Figure 3, we train two independent 6-layer networks on MNIST. ... During training, we measure the representation alignment between every pair of layers, defined as the cosine similarity between the two sides of (12), averaged over the test set. |
| Hardware Specification | No | 8. Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: Personal computers are sufficient for all our experiments. |
| Software Dependencies | No | 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We use pytorch. We use MNIST and CIFAR datasets, and the publically available pretrained weights from the pytorch website. |
| Experiment Setup | Yes | A.1 Res Net: For Figure 1, we train Res Net18 on CIFAR 10 using SGD with momentum 0.9, batchsize 128 and weight decay 5 10 4. The learning rate is 0.1 at the beginning, 0.01 after the 100 th epoch and 0.001 after the 150 th epoch. |