Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
Authors: Rui Luo, Jianhong Wang, Yaodong Yang, Jun WANG, Zhanxing Zhu
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions. It simulates the Nosé-Hoover dynamics of a continuously-tempered Hamiltonian system built on the distribution of interest. A significant advantage of this method is that it is not only able to efficiently draw representative i.i.d. samples when the distribution contains multiple isolated modes, but capable of adaptively neutralising the noise arising from mini-batches and maintaining accurate sampling. While the properties of this method have been studied using synthetic distributions, experiments on three real datasets also demonstrated the gain of performance over several strong baselines with various types of neural networks plunged in. |
| Researcher Affiliation | Academia | Rui Luo1, Jianhong Wang 1, Yaodong Yang 1, Zhanxing Zhu2, and Jun Wang 1 1University College London, 2Peking University |
| Pseudocode | Yes | Algorithm 1 Thermostat-assisted continuously-tempering Hamiltonian Monte Carlo |
| Open Source Code | No | The paper does not provide an explicit statement or link for the availability of its source code. |
| Open Datasets | Yes | We then move on to the tasks of image classification on three real datasets: EMNIST3, Fashion-MNIST4 and CIFAR-10. 3https://www.nist.gov/itl/iad/image-group/emnist-dataset 4https://github.com/zalandoresearch/fashion-mnist |
| Dataset Splits | Yes | EMNIST Balanced is selected as the dataset, where 47 categories of images are split into a training set of size 112,800 and a complementary test set of size 18,800. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions the use of neural network types (MLPs, RNNs, CNNs) and optimizers (Adam, momentum SGD), but does not specify software dependencies with version numbers (e.g., PyTorch 1.x, TensorFlow 2.x). |
| Experiment Setup | Yes | Each method will keep running for 1000 epochs in either sampling or training before the evaluation and comparison. (...) The batch size is fixed at 128 for all methods in both sampling and training tests. For readability, we introduce a 7-tuple [ηθ,ηξ,cθ,cξ,γθ,γξ,K] as the specification to set up TACT-HMC (see Alg. 1). In this experiment, TACT-HMC is configured as [0.0015,0.0015,0.05,0.05,1.0,1.0,50]. |