Rao-Blackwellized Stochastic Gradients for Discrete Distributions
Authors: Runjing Liu, Jeffrey Regier, Nilesh Tripuraneni, Michael Jordan, Jon Mcauliffe
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 5, we exhibit the benefits of our procedure on synthetic data, a semi-supervised classification problem, and a pixel attention task. |
| Researcher Affiliation | Collaboration | Runjing Liu 1 Jeffrey Regier 2 Nilesh Tripuraneni 2 Michael I. Jordan 1 2 Jon Mc Auliffe 1 3 1Department of Statistics, University of California, Berkeley 2Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 3The Voleon Group. |
| Pseudocode | No | The paper describes procedures using mathematical formulations and textual explanations but does not include any clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | We work with the MNIST dataset (Lecun et al., 1998). |
| Dataset Splits | Yes | We used 50 000 MNIST digits in the training set, 10 000 digits in the validation set, and 10 000 digits in the test set. |
| Hardware Specification | Yes | Training was run on a p3.2xlarge instance on Amazon Web Services. |
| Software Dependencies | No | The paper mentions various methods and models (e.g., REINFORCE, Gumbel-softmax, Adam) but does not provide specific software names along with their version numbers required for reproduction. |
| Experiment Setup | Yes | We initialized the optimization with K-means. Figure 3 shows that Rao-Blackwellization improves the convergence rate, with faster convergence when more categories are summed. |