Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
Authors: Rob Brekelmans, Sicong Huang, Marzyeh Ghassemi, Greg Ver Steeg, Roger Baker Grosse, Alireza Makhzani
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our bounds on estimating the MI of VAEs and GANs trained on the MNIST and CIFAR datasets, and showcase significant gains over existing bounds in these challenging settings with high ground truth MI. |
| Researcher Affiliation | Academia | 1 Information Sciences Institute, University of Southern California 2 Vector Institute 3 University of Toronto 4 MIT EECS / IMES / CSAIL |
| Pseudocode | No | The paper does not contain explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | For more details, see the public Git Hub repository, https://github.com/huangsicong/ais_mi_estimation. |
| Open Datasets | Yes | We used MNIST (Le Cun et al., 1998) and CIFAR-10 (Krizhevsky & Hinton, 2009) datasets in our experiments. |
| Dataset Splits | No | The network was trained for 300 epochs with the learning rate of 0.0001 using the Adam optimizer (Kingma & Ba, 2014), and the checkpoint with the best validation loss was used for the evaluation. |
| Hardware Specification | Yes | All experiments are run on on Tesla P100 or Quadro RTX 6000 or Tesla T4 GPUs. |
| Software Dependencies | No | The paper mentions software components like 'Adam optimizer' and 'HMC' but does not provide specific software names with version numbers. |
| Experiment Setup | Yes | The network was trained for 300 epochs with the learning rate of 0.0001 using the Adam optimizer (Kingma & Ba, 2014)... and For all MNIST experiments in Table 1, we evaluated on a single batch size of 128 simulated data. For all CIFAR experiments in Table 1 we used a single batch of 32 simulated data. |