On Testing of Uniform Samplers
Authors: Sourav Chakraborty, Kuldeep S. Meel7777-7784
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present a prototype implementation of Barbarik and use it to test three state of the art uniform samplers over the support defined by combinatorial constraints. To evaluate the runtime performance of Barbarik and test the uniformity of the state of the art samplers, we implemented a prototype of Barbarik and employed SPUR (Achlioptas, Hammoudeh, and Theodoropoulos 2018) as the ideal uniform sampler, i.e., U in the Algorithm 1. |
| Researcher Affiliation | Academia | Sourav Chakraborty Indian Statistical Institute Kolkata Kuldeep S. Meel School of Computing National University of Singapore |
| Pseudocode | Yes | Algorithm 1 Barbarik(A, U, S, ε, η, δ, ϕ) Algorithm 2 Bias(ˆσ, L, S) Algorithm 3 kernel(ϕ, σ1, σ2, τ) |
| Open Source Code | Yes | An extended version of the paper along with open source tool is available at https://github.com/meelgroup/barbarik |
| Open Datasets | Yes | We employed publicly available benchmark suite used in the evaluation of Quicksampler (Dutra et al. 2018) and Uni Gen2, which included bit-blasted versions of constraints arising in bounded model checking of circuits, bit-blasted versions of SMTLib benchmarks, constraints arising from automated program synthesis, and constraints arising from ISCAS89 circuits with parity conditions on randomly chosen subsets of outputs and next-state variables. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits, percentages, or sample counts. |
| Hardware Specification | Yes | The experiments were conducted on a high-performance computer cluster, where each node consists of E5-2690 v3 CPU with 24 cores and 96GB of RAM. |
| Software Dependencies | No | The paper mentions implementing a prototype in Python but does not provide specific version numbers for Python or any other software dependencies. |
| Experiment Setup | Yes | We set tolerance parameter ε, intolerance parameter η, and confidence δ for Barbarik to be 0.6, 0.9, and 0.1 respectively. We use the default parameters for Quicksampler, Uni Gen2, and Search Tree Sampler, which were also employed in previous case studies for uniformity. |