Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Tracing the Representation Geometry of Language Models from Pretraining to Post-training
Authors: Melody Li, Kumar Krishna Agrawal, Arna Ghosh, Komal Teru, Adam Santoro, Guillaume Lajoie, Blake A. Richards
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | With OLMo (1B-7B) and Pythia (160M-12B) models, we uncover a consistent non-monotonic sequence of three geometric phases during autoregressive pretraining. To study LLM representation geometry at intermediate stages of the training lifecycle, we analyze checkpoints from three publicly released model suites. For all experiments, we used 15000 sequences from the the Fine Web sample-10BT dataset to probe the geometry of model representations. |
| Researcher Affiliation | Collaboration | 1Computer Science, Mc Gill University, Canada 2Mila Quebec AI Institute, Canada 3UC Berkeley, USA 4Cohere, Canada 5Google Deepmind, Canada 6Mathematics and Statistics, Université de Montréal, Canada |
| Pseudocode | No | The paper describes methodologies and theoretical findings in prose and mathematical equations but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | No | All code and data will be released at https://github.com/melodylizx/ Tracing-the-representation-geometry-of-language-models . The paper states that the code "will be released" in the near future, but it is not available at the time of publication. |
| Open Datasets | Yes | Fine Web: The Fine Web dataset [Penedo et al., 2024] ... The dataset is accessible on Hugging Face at https://huggingface.co/datasets/Hugging Face FW/fineweb. Sci Q: The Sci Q dataset [Welbl et al., 2017] ... The dataset is accessible on Hugging Face at https://huggingface.co/datasets/allenai/sciq. Trivia QA: The Trivia QA dataset [Joshi et al., 2017] ... The dataset is accessible on Hugging Face at https://huggingface.co/datasets/mandarjoshi/trivia_qa. Anthropic Helpful-Harmless (HH): ... The original dataset is accessible on Hugging Face at https://huggingface.co/datasets/Anthropic/hh-rlhf. |
| Dataset Splits | Yes | Supervised Fine-Tuning (SFT) adapts pre-trained LLMs by further training on a curated dataset DSFT = {(xi, yi)}NSFT i=1 typically consisting of instruction-response pairs. ... We evaluate the robustness of the SFT model by contrasting its performance on held-out examples from DSFT (In-Distribution, ID) with its performance on examples from a related but distinct dataset DOOD (Out-of-Distribution, OOD)... Table 5: Hyperparameter configurations used for ID and OOD loss eval. In-distribution dataset Anthropic-HH (test split) Out-of-distribution dataset Alpaca Farm Human-ANN chat (train split) |
| Hardware Specification | Yes | All of our LLM inference experiments were run either on a single 80GB A100 or a 40GB L40S GPU. The finetuning experiments (SFT and DPO) were run on a single node consisting of 4 A100 GPUs. |
| Software Dependencies | No | The paper does not explicitly state specific version numbers for software dependencies like programming languages or libraries (e.g., Python, PyTorch, TensorFlow, scikit-learn). |
| Experiment Setup | Yes | Table 3: Hyperparameter configurations used for computing Rank Me and αRe Q in Figure 2. Dataset Fine Web sample-10BT Max sequence length 512 Number of sequences 15000 Batch size 16 |