Looped Transformers as Programmable Computers
Authors: Angeliki Giannou, Shashank Rajput, Jy-Yong Sohn, Kangwook Lee, Jason D. Lee, Dimitris Papailiopoulos
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | One limitation is that the constructions presented in this paper have not been experimentally validated for efficiency. |
| Researcher Affiliation | Academia | 1University of Wisconsin-Madison 2Yonsei University 3Princeton University. |
| Pseudocode | Yes | Algorithm 1 Looped Transformer |
| Open Source Code | Yes | The code is available at https://github.com/jysohn1108/Looped_TF |
| Open Datasets | No | The paper does not conduct experiments involving datasets, focusing instead on theoretical constructions and emulation capabilities. |
| Dataset Splits | No | The paper does not describe experiments with data splits (train/validation/test), as its focus is on theoretical framework and emulation. |
| Hardware Specification | No | The paper focuses on theoretical constructions and proofs regarding transformer capabilities and does not provide details on specific hardware used for experiments. |
| Software Dependencies | No | The paper describes a theoretical framework and its constructions but does not specify software dependencies with version numbers for reproducibility of experiments. |
| Experiment Setup | No | The paper presents theoretical work on constructing transformer-based computational units and does not include details on experimental setups or hyperparameters. |