Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Simulation of Graph Algorithms with Looped Transformers
Authors: Artur Back De Luca, Kimon Fountoulakis
ICML 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We prove by construction that this architecture can simulate individual algorithms... We provide empirical validation of the results in Appendix B.3. |
| Researcher Affiliation | Academia | 1David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada. |
| Pseudocode | Yes | Algorithm 1 Looped Transformer (Giannou et al., 2023) |
| Open Source Code | No | No explicit statement about providing open-source code for the described methodology or a link to a code repository was found. |
| Open Datasets | Yes | We validate our theoretical results using graphs in the CLRS Algorithmic Reasoning Benchmark (Veliˇckovi c et al., 2022). |
| Dataset Splits | Yes | For graph-related tasks, the training and validation sets include graphs with 16 nodes, while the test set contains graphs with 64 nodes. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | The paper mentions software components like ReLU and softmax, but does not provide specific version numbers for any libraries, frameworks, or programming languages used in the implementation. |
| Experiment Setup | Yes | In our empirical verifications, we consistently apply specific parameters: an angular increment δ set at 10 2, a maximum value Ωof 105, and a temperature parameter T at 10 7. |