DisARM: An Antithetic Gradient Estimator for Binary Latent Variables
Authors: Zhe Dong, Andriy Mnih, George Tucker
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Dis ARM on several generative modeling benchmarks and show that it consistently outperforms ARM and a strong independent sample baseline in terms of both variance and log-likelihood. |
| Researcher Affiliation | Industry | Zhe Dong Google Research, Brain Team zhedong@google.com Andriy Mnih Deep Mind amnih@google.com George Tucker Google Research, Brain Team gjt@google.com |
| Pseudocode | Yes | we followed the techniques used in (Tucker et al., 2017; Grathwohl et al., 2018; Yin and Zhou, 2019) to extend Dis ARM to this setting (summarized in Appendix Algorithm 1). |
| Open Source Code | Yes | Code and additional information: https://sites.google.com/view/disarm-estimator. |
| Open Datasets | Yes | We evaluate the gradient estimators on three benchmark generative modeling datasets: MNIST, Fashion MNIST and Omniglot. |
| Dataset Splits | Yes | We use the standard split into train, validation, and test sets. |
| Hardware Specification | Yes | Furthermore, training the model on a P100 GPU was nearly twice as slow for RELAX, while ARM, Dis ARM and REINFORCE LOO trained at the same speed. |
| Software Dependencies | No | The paper mentions using "Adam (Kingma and Ba, 2015)" as an optimizer but does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | The models were trained with Adam (Kingma and Ba, 2015) using a learning rate 10 4 on mini-batches of 50 examples for 106 steps. |