Momentum Particle Maximum Likelihood
Authors: Jen Ning Lim, Juan Kuntz, Samuel Power, Adam Michael Johansen
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 6, we demonstrate empirically the efficacy of our method and study the effects of various design choices. As a large-scale experiment, we benchmark our proposed method for training Variational Autoencoders (Kingma and Welling, 2014) against current methods for training latent variable models. |
| Researcher Affiliation | Collaboration | 1University of Warwick 2Polygeist 3University of Bristol. |
| Pseudocode | Yes | Algorithm 1 A single subsampled step. In pink, we indicate the existing subsampling scheme of Kuntz et al. (2023). We indicate our proposed additions in teal. |
| Open Source Code | Yes | The code for reproducibility is available online: https://github.com/jenninglim/mpgd. |
| Open Datasets | Yes | For this task, we consider two datasets: MNIST (Le Cun et al., 1998) and CIFAR-10 (Krizhevsky and Hinton, 2009). |
| Dataset Splits | No | The paper states 'We use N = 5000 images for training and 3000 for testing' but does not explicitly mention or detail a validation split. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU models, or cloud computing instance types used for experiments. |
| Software Dependencies | No | The paper does not specify versions for any software dependencies or libraries used, such as Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | MPD. For step sizes, we have hθ = hx = 10 4. The number of particles is set to M = 5. For the momentum parameters, we use γθ = γx = 0.9 with the momentum coefficient µθ = 0.95, µx = 0 (or equivalently, ηθ 556, ηx 11, 111 )for MNIST and µθ = 0.95, µx = 0.5 (or equivalently, ηθ 556, , ηx 55, 555 ) for CIFAR. We use the RMSProp preconditioner (see Appendix J.1) with β = 0.9. |