Dimensionally Tight Bounds for Second-Order Hamiltonian Monte Carlo
Authors: Oren Mangoubi, Nisheeth Vishnoi
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our simulations on synthetic data suggest that, when our regularity condition is satisfied, leapfrog HMC performs better than its competitors both in terms of accuracy and in terms of the number of gradient evaluations it requires. Moreover, our simulations on synthetic data suggest that, when our regularity condition is satisfied, leapfrog HMC performs better than its competitors both in terms of accuracy and in terms of the number of gradient evaluations it requires. Finally, we perform simulations to evaluate the performance of the HMC algorithm analyzed in this paper, and show that its performance is competitive in both accuracy and speed with the Metropolis-adjusted version of HMC despite the lack of a Metropolis filter, when performing Bayesian logistic regression on synthetic data. |
| Researcher Affiliation | Academia | Oren Mangoubi EPFL omangoubi@gmail.com Nisheeth K. Vishnoi EPFL nisheeth.vishnoi@gmail.com |
| Pseudocode | Yes | Algorithm 1 Unadjusted HMC |
| Open Source Code | Yes | All simulations were implemented on MATLAB (see the Git Hub repository https://github.com/mangoubi/HMC for our MATLAB code used to implement these algorithms). |
| Open Datasets | No | We consider the setting of Bayesian logistic regression with standard normal prior, with synthetic independent variable" data vectors generated as Xi = Zi/‖Zi‖2 for Z1,...,Zr ∼ N(0,Id) iid, for dimension d = 1000 and r = d. To generate the synthetic dependent variable" binary data, a vector β = (β1,...,βd) of regression coefficients was first generated as β = W/‖W‖2 where W ∼ N(0,Id). The binary dependent variable synthetic data Y1,...,Yd were then generated as independent Bernoulli random variables, setting Yi = 1 with probability 1/(1 + e−β·Xi) and Yi = 0 otherwise. |
| Dataset Splits | No | The paper uses synthetic data generated as described in the text and discusses evaluation by comparing to a benchmark obtained by running MALA for a long time, but it does not provide explicit training, validation, or test dataset splits in terms of percentages or counts, nor does it refer to standard predefined splits. |
| Hardware Specification | No | The paper states, 'All simulations were implemented on MATLAB', but it does not provide any specific details about the hardware used (e.g., CPU, GPU models, memory). |
| Software Dependencies | No | The paper mentions 'All simulations were implemented on MATLAB' but does not specify a version number for MATLAB or any other software dependencies, making it difficult to reproduce the exact software environment. |
| Experiment Setup | Yes | Each Markov chain was initialized at a point X0 chosen randomly as X0 ∼ N(0,Id). To compare the accuracy, we computed the marginal accuracy" (MA) of the samples generated by each chain over a fixed number (50,000) of numerical steps for different step sizes η in the interval [0.1,0.6]. The accuracy-optimizing step size was η = 0.35 for UHMC. The autocorrelation time-optimizing step size was η = 0.5 for UHMC. When running UHMC and MHMC, we used a trajectory time T equal to π/3, rounded down to the nearest multiple of η. |