Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry
Authors: Sai Sumedh R. Hindupur, Ekdeep S Lubana, Thomas Fel, Demba Ba
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform a suite of experiments which involve training Re LU, Jump Re LU, Top K and Spa DE SAEs on synthetic Gaussian clusters, semi-synthetic formal-language model activations and natural vision model activations. |
| Researcher Affiliation | Collaboration | 1School of Engineering and Applied Science, Harvard University 2CBS-NTT Program in Physics of Intelligence, Harvard University 3Physics of Artificial Intelligence Group, NTT Research, Inc., Sunnyvale, CA, USA 4Kempner Institute, Harvard University |
| Pseudocode | No | The paper describes algorithms and mathematical formulations (e.g., 'arg min z 0,D B x x Dz 2 2 + λR(z)' for sparse coding) but does not include a distinct 'Pseudocode' or 'Algorithm' block with structured, numbered steps for its proposed methods or other described processes. |
| Open Source Code | Yes | The code to replicate synthetic experiments is available at: https://github.com/Sai-Sumedh/Sae Concept Duality-Spa DE, formal language experiments is at: https://github.com/Ekdeep SLubana/spade Formal Grammars, and vision experiments is at: https://github.com/Kempner Institute/Overcomplete. |
| Open Datasets | Yes | We use Imagenette, a 10-class subset of Image Net [50], containing 1.5k images per class. |
| Dataset Splits | Yes | The dataset consists of 1 million data points per concept, yielding a total of 6 million samples. Of these, we use 70% (700,000 points) for training. [...] The dataset contains 6.4 million data points per concept, yielding a total of 32 million samples, of which 70% (approximately 22 million points) are used for training. |
| Hardware Specification | Yes | The synthetic experiments (separability, heterogeneity) and vision experiments were run on NVIDIA A100 40GB GPUs, while the formal language experiments were run on NVIDIA RTX A6000 48GB GPUs. |
| Software Dependencies | No | The paper mentions using the Adam optimizer (e.g., in C.1 and C.2) but does not specify version numbers for any programming languages, libraries, or frameworks used in the implementation (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | All models are trained using the Adam optimizer with a learning rate of 10 2, which follows a cosine decay schedule from 10 2 to 10 4. The momentum parameter is set to 0.9, and we use a batch size of 512. Training runs for approximately 8000 iterations, and gradient clipping is applied (gradient norms are clipped at 1) to stabilize optimization. Regularization parameters are selected such that sparsity levels remain comparable across models. Specifically, the regularization coefficient γ is chosen in the range 10 6 to 1 for Re LU and Jump Re LU SAEs, between 4 and 64 (powers of 2) for Top K SAE, and in the range 10 6 to 1 for Spa DE. |