Incremental Sampling Without Replacement for Sequence Models
Authors: Kensen Shi, David Bieber, Charles Sutton
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally demonstrate that using Unique Randomizer leads to higher-quality samples in program synthesis and combinatorial optimization. |
| Researcher Affiliation | Industry | 1Google. Correspondence to: Kensen Shi <kshi@google.com>. |
| Pseudocode | Yes | Algorithm 1: Using Unique Randomizer to sample outputs of P without replacement. Algorithm 2: Random choice operation and trie construction for Unique Randomizer. |
| Open Source Code | Yes | Our Python implementations of Unique Randomizer, its batched version, Stochastic Beam Search, and the Hindsight Gumbel Estimator can be found at https://github. com/google-research/unique-randomizer. |
| Open Datasets | Yes | We use the Search-based Pseudocode to Code (SPo C) dataset (Kulal et al., 2019)... |
| Dataset Splits | No | The paper mentions 'test splits' but does not specify training, validation, and test splits with explicit percentages or sample counts, nor does it refer to predefined validation splits. |
| Hardware Specification | No | The paper mentions running experiments 'with a GPU' but does not specify the model, type, or any other detailed hardware specifications. |
| Software Dependencies | No | The paper states 'Our Python implementations' but does not provide specific version numbers for Python or any associated libraries/dependencies. |
| Experiment Setup | Yes | We use a temperature τ = 0.8 when sampling with replacement and τ = 0.4 when sampling without replacement, which we found to be the best among {0.2, 0.4, 0.6, 0.8, 1.0}. For TSP...We set τ = 0.3, 0.2, and 0.15 for n = 20, 50, and 100 nodes, respectively. |