Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
A multiscale analysis of mean-field transformers in the moderate interaction regime
Authors: Giuseppe Bruno, Federico Pasqualotto, Andrea Agazzi
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We characterize the limiting dynamics in each phase, exemplifying our results with some simulations. ... This section presents numerical simulations of the transformer model in Eq. (1). All experiments are conducted in dimension d = 3 or or d = 2 to facilitate visualization and are designed to validate our theoretical findings. |
| Researcher Affiliation | Academia | Giuseppe Bruno Department of Mathematics and Statistics University of Bern EMAIL Federico Pasqualotto Department of Mathematics University of California, San Diego EMAIL Andrea Agazzi Department of Mathematics and Statistics University of Bern EMAIL |
| Pseudocode | No | The paper describes mathematical models and dynamical systems using equations and prose, but it does not contain any clearly labeled pseudocode blocks or algorithms. |
| Open Source Code | Yes | The code is available at [28]. [28] Git Hub-Repository. https://github.com/gbruno16/multiscale_transformers. |
| Open Datasets | No | For both scenarios presented in Figure 1, the initial state consists of N = 104 tokens sampled independently and identically uniformly from the sphere S2. ... The initial configuration comprises N = 104 i.i.d. tokens. Their elevation angle ψ is sampled uniformly on [-π/2, π/2], while their azimuthal angles, θi ∈ [0, 2π), are distributed according to the mixture density g(θ)... |
| Dataset Splits | No | The paper focuses on theoretical analysis and numerical simulations with generated data, not on machine learning experiments with predefined training, validation, or test dataset splits. |
| Hardware Specification | Yes | The experiments are performed on a single Nvidia H100. ... Calculations were performed on UBELIX, the HPC cluster at the University of Bern. |
| Software Dependencies | No | The attention mechanism is implemented using the official Py Torch function torch.nn.functional.scaled_dot_product_attention() but specific version numbers for PyTorch or other libraries are not provided. |
| Experiment Setup | Yes | We set the inverse temperature parameter β = 30 and use a time step of dt = 10^-2. ... For the experiment in Figure 2 we set β = 10. The initial configuration comprises N = 104 i.i.d. tokens. ... we simulate the evolution of 5 × 10^4 tokens, initially sampled from a superposition of three Gaussian densities on S1, through the transformer model with parameters β = 50, d = 2, Q = K = Id, V = Id, and dt = 10^-3. |