Iterated Denoising Energy Matching for Sampling from Boltzmann Densities
Authors: Tara Akhound-Sadegh, Jarrid Rector-Brooks, Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, Alexander Tong
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate i DEM on a suite of tasks ranging from standard synthetic energy functions to invariant n-body particle systems. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains 2 5 faster, which allows it to be the first method to train using energy on the challenging 55-particle Lennard-Jones system. |
| Researcher Affiliation | Collaboration | 1Mila Qu ebec AI Institute 2Mc Gill University 3Dreamfold 4Universit e de Montr eal 5University of Oxford 6Ciela Institute 7Center for Computational Astrophysics, Flatiron Institute 8Jagiellonian University 9CIFAR. |
| Pseudocode | Yes | Algorithm 1 ITERATED DENOISING ENERGY MATCHING |
| Open Source Code | Yes | Code for i DEM is available at https://github.com/jarridrb/dem. |
| Open Datasets | Yes | Datasets. We evaluate i DEM on four datasets, a 40-Gaussian mixture model (GMM), and three equivariant potentials: A 4-particle double-well potential (DW-4), a 13-particle Lennard-Jones potential (LJ-13), and a 55-particle Lennard-Jones potential (LJ-55) (see F.4 for details). ...We use a 40 Gaussian mixture density in 2 dimensions as proposed by Midgley et al. (2023b)... The energy function for the DW-4 dataset was introduced in K ohler et al. (2020)... For the experimental results, we evaluate using the MCMC samples from Klein et al. (2023b). |
| Dataset Splits | Yes | To evaluate the efficacy of our samples we use a validation and test set from the the MCMC samples in Klein et al. (2023b) as the Ground truth samples. |
| Hardware Specification | Yes | All networks were optimized using Adam and were performed on NVIDIA A100 GPUs with 40GB of VRAM. |
| Software Dependencies | No | The paper mentions software components like "Adam" (optimizer) and implies the use of a deep learning framework (e.g., PyTorch given the context of neural networks and GPUs), but it does not specify any software versions for these or other dependencies (e.g., Python version, specific library versions). |
| Experiment Setup | Yes | For i DEM, we use a geometric noise schedule σ(t) = σmin( σmax / σmin )t and tune over learning rate as well as σmin and σmax. For PIS, we tune over the learning rate and the coefficient of the Brownian motion. For DDS, we tune over using their proposed exponential or Euler integration, σmax and αmax when using the exponential integration, and βmin and βmax for Euler integration... i DEM was trained with a geometric noise schedule with σmin = 1e 5, σmax = 1, K = 500 samples for computing the regression target SK and we clipped the norm of SK to 70. i DEM was trained with a learning rate of 5e 4. |