On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients
Authors: Difan Zou, Quanquan Gu
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiment results verify our theoretical findings. In this section, we will evaluate the empirical performance of the aforementioned four stochastic gradient HMC algorithms, including SG-HMC, SVRG-HMC, SAGA-HMC and CVG-HMC, on both synthetic and real-world datasets. |
| Researcher Affiliation | Academia | Difan Zou 1 Quanquan Gu 1 1Department of Computer Science, UCLA. Correspondence to: Quanquan Gu <qgu@cs.ucla.edu>. |
| Pseudocode | Yes | Algorithm 1 Noisy Gradient Hamiltonian Monte Carlo |
| Open Source Code | No | The paper does not provide an explicit statement about the release of its source code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | We carry out the experiments on Covtype dateset 5, which has 581012 instances with 54 attributes. 5Available at https://archive.ics.uci.edu/ml/datasets/covertype |
| Dataset Splits | No | The paper mentions extracting training datasets with sizes n={500, 5000} and using 'the rest for test', but does not explicitly provide information about a separate validation split, specific percentages, or counts for each split. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions software like Stan and TensorFlow but does not provide specific version numbers for these or any other ancillary software components used in the experiments. |
| Experiment Setup | Yes | For HMC based algorithms, We run all four algorithms using the same step size (η = {2 × 10−3, 3 × 10−4} for n = {500, 5000}) and mini-batch size B = 16 with 2 × 10^4 steps (2000 proposals with 10 internal leapfrog steps each). For ULD based algorithms we use the same batch size and tune the step size such that they converge fast. |