A Logic for Expressing Log-Precision Transformers
Authors: William Merrill, Ashish Sabharwal
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We prove any log-precision transformer classifier can be equivalently expressed as a first-order logic sentence that, in addition to standard universal and existential quantifiers, may also contain majority-vote quantifiers. This is the tightest known upper bound and first logical characterization of log-precision transformers. |
| Researcher Affiliation | Collaboration | William Merrill New York University willm@nyu.edu Ashish Sabharwal Allen Institute for AI ashishs@allenai.org |
| Pseudocode | Yes | Algorithm 1 node C(n, i) Return the type of gate i in circuit Cn. Algorithm 2 edge C(n, i, j) If Cn contains an edge i ! j, return the argument number of that edge. Otherwise, return 1. |
| Open Source Code | No | The paper does not mention any open-source code for the methodology described. |
| Open Datasets | No | The paper does not discuss or provide access information for any dataset used for training, as it is a theoretical paper. |
| Dataset Splits | No | The paper does not provide information about training/validation/test dataset splits, as it is a theoretical work without empirical evaluations. |
| Hardware Specification | No | The paper does not provide specific hardware details, as it is a theoretical work without experimental setup. |
| Software Dependencies | No | The paper does not provide specific software dependency details with version numbers, as it is a theoretical work without experimental setup. |
| Experiment Setup | No | The paper does not contain specific experimental setup details (e.g., hyperparameter values), as it is a theoretical work without an empirical setup. |