Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable
Authors: Alan Jeffares, Mihaela Van Der Schaar
ICML 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | This position paper asserts that, in many prominent cases, there is little evidence to suggest that these phenomena appear in real-world applications and these efforts may be inefficient in driving progress in the broader field. Consequently, we argue against viewing them as isolated puzzles that require bespoke resolutions or explanations. However, despite this, we suggest that deep learning phenomena do still offer research value by providing unique settings in which we can refine our broad explanatory theories of more general deep learning principles. This position is reinforced by analyzing the research outcomes of several prominent examples of these phenomena from the recent literature. We revisit the current norms in the research community in approaching these problems and propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning. |
| Researcher Affiliation | Academia | 1Department of Applied Mathematics and Theoretical Physics, University of Cambridge. Correspondence to: Alan Jeffares <EMAIL>. |
| Pseudocode | No | The paper describes conceptual frameworks and recommendations but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described within this position paper. It references a third-party implementation (https://github.com/teddykoker/grokking) in the context of an illustrative example, but this is not the authors' own code for the paper's contribution. |
| Open Datasets | No | This position paper analyzes existing deep learning phenomena and discusses research approaches, but it does not conduct its own experiments using a specific dataset or provide access information for any dataset it directly uses or creates. |
| Dataset Splits | No | The paper is a position paper that does not describe new experiments or introduce new datasets. Therefore, it does not provide dataset split information. |
| Hardware Specification | No | The paper is a position paper focusing on theoretical and conceptual arguments regarding deep learning phenomena. It does not describe any specific experiments conducted by the authors that would require hardware specifications. |
| Software Dependencies | No | The paper is a position paper discussing research methodologies and phenomena, and does not conduct experiments requiring specific software dependencies or versions to be listed. |
| Experiment Setup | No | The paper is a position paper that provides a critical viewpoint and recommendations on deep learning phenomena. It does not describe any specific experimental setups, hyperparameters, or training configurations conducted by the authors. |