Grammar-Aligned Decoding
Authors: Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, Loris D'Antoni
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our evaluation on code generation and structured NLP tasks shows how ASAp often produces outputs with higher likelihood (according to the LLM s distribution) than existing GCD techniques, while still enforcing the desired grammatical constraints. We evaluate ASAp on two structured prediction tasks: formal program synthesis and constituency parsing. |
| Researcher Affiliation | Academia | University of Wisconsin-Madison University of California San Diego |
| Pseudocode | Yes | Algorithm 1 ASAp algorithm |
| Open Source Code | Yes | Our code, datasets, and checkpoints are available at: https://github.com/ebmoon/transformers-GAD. |
| Open Datasets | Yes | Our code, datasets, and checkpoints are available at: https://github.com/ebmoon/transformers-GAD. |
| Dataset Splits | Yes | We employ a standard train-validation-test split of 70-10-20%. |
| Hardware Specification | Yes | Our experiments are conducted on 4 NVIDIA RTX A6000 GPUs and 4 NVIDIA A100 GPUs. |
| Software Dependencies | Yes | Our implementation is based on Python 3.10 and Py Torch 2.1.2. |
| Experiment Setup | Yes | Key hyperparameters include a learning rate of 2e-4, a warmup ratio of 0.03, a maximum sequence length of 2048, Lo RA alpha of 32, Lo RA dropout of 0.05, and Lo RA rank of 64. |