Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
Authors: Hanjun Dai, Rishabh Singh, Bo Dai, Charles Sutton, Dale Schuurmans
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally evaluated the approach on both synthetic and real-world tasks. For a program synthesis problem, we observe significant accuracy improvements over the baseline methods. More notably, for a software testing task, a fuzz test guided by an EBM achieves comparable performance to a well-engineered fuzzing engine on several open source software projects. |
| Researcher Affiliation | Industry | Hanjun Dai, Rishabh Singh, Bo Dai, Charles Sutton, Dale Schuurmans Google Research, Brain Team EMAIL |
| Pseudocode | Yes | Algorithm 1: Main algorithm of ALOE; Algorithm 2: Update sampler q |
| Open Source Code | Yes | Please also refer to our open sourced implementation for more details 1. 1https://github.com/google-research/google-research/tree/master/aloe |
| Open Datasets | Yes | We first collect synthetic 2D data in a continuous space [60]... We collect three software binaries (namely libpng, libmpeg2 and openjpeg) from OSS-Fuzz3 as test target... Data is generated synthetically, following Devlin et al. [3]. |
| Dataset Splits | No | The paper mentions "10k test examples" and "training data is generated on the fly" but does not provide specific training/validation/test splits or mention a validation set. |
| Hardware Specification | No | The paper does not specify any hardware details such as GPU or CPU models used for experiments. |
| Software Dependencies | No | The paper does not specify software versions for any dependencies (e.g., Python, TensorFlow, PyTorch) used for implementation. |
| Experiment Setup | Yes | All the methods learn the same score function f, which is parameterized by a 4-layer MLP with ELU [61] activations and hidden layer size= 256. ... we train a truncated EBM with a window size of 64. ... We train baseline seq2seq with 483M examples (denoted as seq2seq-init), and fine-tune with additional 264M samples (denoted as seq2seq-tune) with reduced learning rate. |