Generative Marginalization Models
Authors: Sulin Liu, Peter Ramadge, Ryan P Adams
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate marginalization models (MAM) on both MLE and EB settings for discrete problems including images, text, molecules and phyiscal systems. We compare MAMs with baselines that support arbitrary marginal inference3: Anyorder ARM [24], ARM [38], Parallel Any-order ARMs (P-AO-ARM) [24] and Probabilistic Circuit (PC) [54]. |
| Researcher Affiliation | Academia | 1Princeton University. Correspondence to: Sulin Liu <sulinl@princeton.edu>. |
| Pseudocode | Yes | We present the algorithms for training MAM for maximum likelihood and energy-based training settings in Algorithm 1 and Algorithm 2. |
| Open Source Code | Yes | Code is available at github.com/Princeton LIPS/Ma M. |
| Open Datasets | Yes | We evaluate MAMs on Binary MNIST [63], CIFAR-10 [37] and Image Net32 [14, 10]. We evaluate MAM on the molecular generation benchmark MOSES [55] refined from the ZINC database [72]. We train a character-level generative model on Text8 [45]. |
| Dataset Splits | Yes | We use the training and test split of [63] provided in https://github.com/yburda/iwae/tree/master [6]. The CIFAR-10 dataset [37] comprises 60,000 32x32 color images across 10 classes, split into 50,000 training and 10,000 test images. |
| Hardware Specification | Yes | All models are trained on a single NVIDIA A100. The evaluation time is tested on an NVIDIA GTX 1080Ti. |
| Software Dependencies | No | The paper mentions software components like 'Adam' and 'Adam W' for optimization, and 'U-Net' and 'Transformer' for neural network architectures. However, it does not specify version numbers for any key software libraries (e.g., 'PyTorch 1.x' or 'TensorFlow 2.x'), programming languages, or specific solvers used to run the experiments. |
| Experiment Setup | Yes | Batch size is 128 for MNIST and 32 for CIFAR-10 and Image Net. Adam is used with learning rate 0.0001. Gradient clipping is set to 100. Both AO-ARM and MAM conditionals are trained for 100 epochs on MNIST, 800 epochs on CIFAR-10, 16 epochs on Image Net. |