Simulation of Graph Algorithms with Looped Transformers
Authors: Artur Back De Luca, Kimon Fountoulakis
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We prove by construction that this architecture can simulate individual algorithms... We provide empirical validation of the results in Appendix B.3. |
| Researcher Affiliation | Academia | 1David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada. |
| Pseudocode | Yes | Algorithm 1 Looped Transformer (Giannou et al., 2023) |
| Open Source Code | No | No explicit statement about providing open-source code for the described methodology or a link to a code repository was found. |
| Open Datasets | Yes | We validate our theoretical results using graphs in the CLRS Algorithmic Reasoning Benchmark (Veliˇckovi c et al., 2022). |
| Dataset Splits | Yes | For graph-related tasks, the training and validation sets include graphs with 16 nodes, while the test set contains graphs with 64 nodes. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | The paper mentions software components like ReLU and softmax, but does not provide specific version numbers for any libraries, frameworks, or programming languages used in the implementation. |
| Experiment Setup | Yes | In our empirical verifications, we consistently apply specific parameters: an angular increment δ set at 10 2, a maximum value Ωof 105, and a temperature parameter T at 10 7. |