Gradient-based Discrete Sampling with Automatic Cyclical Scheduling
Authors: Patrick Pynadath, Riddhiman Bhattacharya, ARUN HARIHARAN, Ruqi Zhang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate the superiority of our method in sampling complex multimodal discrete distributions. |
| Researcher Affiliation | Academia | Patrick Pynadath Department of Computer Science Purdue University West Lafayette, IN, 47907 ppynadat@purdue.edu.edu Riddhiman Bhattacharya Department of Management Purdue University West Lafayette, IN, 47907 bhatta76@purdue.edu Arun Hariharan Department of Computer Science Purdue University West Lafayette, IN, 47907 harihar4@purdue.edu Ruqi Zhang Department of Computer Science Purdue University West Lafayette, IN, 47907 ruqiz@purdue.edu |
| Pseudocode | Yes | Algorithm 1 Cyclical Sampling Algorithm |
| Open Source Code | Yes | We released our code at the following link: https://github.com/patrickpynadath1/automatic_cyclical_sampling. |
| Open Datasets | Yes | We evaluate our sampling method on both Restricted Boltzmann Machines (RBMs) and deep convolutional Energy-Based Models (EBMs). For RBMs, we measure accuracy by comparing the Maximum Mean Divergence (MMD) between samples generated by our method and Block Gibbs, which can be considered the ground truth. We sample on EBMs to demonstrate our method s scalability to more complex distributions. Experimental details are provided in Appendices D.2 and D.3 for RBM and EBM sampling, respectively. |
| Dataset Splits | Yes | We train these models for 50,000 iterations total with 40 sampling steps per iteration and use the parameters corresponding to the best log likelihood scores on the validation dataset. |
| Hardware Specification | Yes | All experiments were run on a single RTX A6000. |
| Software Dependencies | No | The paper does not explicitly list specific software dependencies with their version numbers (e.g., "Python 3.8, PyTorch 1.9"). While code is provided, specific versions are not detailed in the text. |
| Experiment Setup | Yes | For ACS, we use ρ = .5, βmax = .95, ζ = .5, cycle length s = 20 for all the datasets. |