Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
DINGO: Constrained Inference for Diffusion LLMs
Authors: Tarun Suresh, Debangshu Banerjee, Shubham Ugare, Sasa Misailovic, Gagandeep Singh
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on multiple open-source diffusion LLMs and benchmarks show that DINGO significantly outperforms standard unconstrained decoding, achieving up to a 68% improvement on challenging tasks such as the GSM-symbolic benchmark for symbolic reasoning [Mirzadeh et al., 2024] and a JSON generation benchmark [Nous Research, 2024]. |
| Researcher Affiliation | Academia | Tarun Suresh , Debangshu Banerjee , Shubham Ugare, Sasa Misailovic, Gagandeep Singh Department of Computer Science, University of Illinois Urbana-Champaign |
| Pseudocode | Yes | Algorithm 1 DINGO DP |
| Open Source Code | Yes | The code is available at DINGO. |
| Open Datasets | Yes | Extensive experiments on multiple open-source diffusion LLMs and benchmarks show that DINGO significantly outperforms standard unconstrained decoding, achieving up to a 68% improvement on challenging tasks such as the GSM-symbolic benchmark for symbolic reasoning [Mirzadeh et al., 2024] and a JSON generation benchmark [Nous Research, 2024]. |
| Dataset Splits | No | The paper mentions prompting LLMs with 4-shot examples and sets generation lengths, but does not explicitly describe train/test/validation splits of the datasets themselves for model training or evaluation reproducibility. |
| Hardware Specification | Yes | We run experiments on a 48-core Intel Xeon Silver 4214R CPU with 2 Nvidia RTX A5000 GPUs. |
| Software Dependencies | No | DINGO is implemented using Py Torch Paszke et al. [2019] and the Hugging Face transformers library Wolf et al. [2020]. The token-level DFA is implemented in Rust using a highly efficient regex-DFA library. Specific version numbers for these software components are not provided. |
| Experiment Setup | Yes | We set the generation length to 128, number of blocks to 8, and total diffusion steps to 64 and prompt the LLMs with 4-shot examples from GSM-Symbolic [Mirzadeh et al., 2024]. For JSON Generation, We set the generation length to 128, number of blocks to 1, and the total diffusion steps to 64. |