Stochastic Quasi-Newton Langevin Monte Carlo
Authors: Umut Simsekli, Roland Badeau, Taylan Cemgil, Gaël Richard
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate the effectiveness of the approach on both synthetic and real datasets. Our experiments on two challenging applications show that our method achieves fast convergence rates similar to Riemannian approaches while at the same time having low computational requirements similar to diagonal preconditioning approaches. 4. Experiments |
| Researcher Affiliation | Academia | 1: LTCI, CNRS, Télécom Paris Tech, Université Paris-Saclay, 75013, Paris, France 2: Department of Computer Engineering, Bo gaziçi University, 34342, Bebek, Istanbul, Turkey |
| Pseudocode | Yes | Algorithm 1: Hessian Approximated MCMC |
| Open Source Code | No | The paper mentions using 'the Matlab code that can be downloaded from the authors websites' for a specific application (speech enhancement from Sim sekli et al., 2015) but does not provide concrete access or release statements for the HAMCMC methodology itself. |
| Open Datasets | Yes | on a large movie ratings dataset, namely the Movie Lens 1M (grouplens.org). |
| Dataset Splits | Yes | We randomly select 10% of the data as the test set and partition the rest of the data as illustrated in Fig. 4. |
| Hardware Specification | Yes | These experiments are conducted on a standard laptop computer with 2.5GHz Quad-core Intel Core i7 CPU and all the methods are implemented in Matlab. We conduct our experiments on a small-sized cluster with 4 interconnected computers each of them with 4 Intel Xeon 2.93GHz CPUs and 192 GB of memory. |
| Software Dependencies | No | The paper states that methods are implemented in 'Matlab' and 'C by using a low-level message passing protocol, namely the Open MPI library,' but it does not provide specific version numbers for any of these software dependencies. |
| Experiment Setup | Yes | In these experiments, we determine the step size as ϵt = (aϵ/t)0.51. In all our experiments, for each method, we have tried several values for the hyper-parameters with the same rigor and we report the best results. We also report all of the hyper-parameters used in the experiments (including Sections 4.2-4.3) in the Supplement. the size of the data subsamples is selected as NΩ= N/100. We set M = 2 for HAMCMC. For each method, we set NΩ= IJ/10 and we generate 2500 samples where we compute the Monte Carlo averages by using the last 200 samples. For HAMCMC, we set M = 5. For this experiment, we have implemented all the algorithms in C by using a low-level message passing protocol, namely the Open MPI library. In order to keep the implementation simple, for HAMCMC we set M = 2. In these experiments, we use a constant step size for each method and discard the first 50 samples as burn-in. |