MINDE: Mutual Information Neural Diffusion Estimation
Authors: Giulio Franzese, Mustapha BOUNOUA, Pietro Michiardi
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our results, which derive from a thorough experimental protocol over all the variants of our approach, indicate that our method is more accurate than the main alternatives from the literature, especially for challenging distributions. Furthermore, our methods pass MI self-consistency tests, including data processing and additivity under independence, which instead are a pain-point of existing methods. |
| Researcher Affiliation | Collaboration | Giulio Franzese1, , Mustapha Bounoua1,2, Pietro Michiardi1 1EURECOM,2Ampere Software Technology, giulio.franzese@eurecom.fr |
| Pseudocode | Yes | Algorithm 1: MINDE C (Single Training Step), Algorithm 2: MINDE C, Algorithm 3: MINDE J (Single Training Step), Algorithm 4: MINDE J |
| Open Source Code | Yes | Code available. |
| Open Datasets | Yes | Considering as random variable A a sample from the MNIST (resolution 28 28) dataset... (Section 5.2) and Using the text prompt samples from LAION dataset Schuhmann et al. [2022], we synthetically generate image samples. (Section E) |
| Dataset Splits | No | The paper mentions '100k training samples' and '10k test samples' and refers to using the same training procedure as Czy z et al. [2023] which includes 'early stopping strategy'. However, it does not explicitly provide specific percentages or counts for a separate validation set split. |
| Hardware Specification | No | The paper does not specify the hardware used for running the experiments, such as particular GPU models, CPU types, or cloud computing resources. It only mentions using 'a simple, stacked multi-layer perception (MLP) with skip connections' and 'Stable Diffusion... using the original code-base and pre-trained checkpoints'. |
| Software Dependencies | No | The paper mentions using 'the package benchmark-mi1 implementation' and 'Adam optimizer [Kingma & Ba, 2015]' but does not provide specific version numbers for Python, PyTorch, CUDA, or other key software dependencies required for replication. |
| Experiment Setup | Yes | The hyper-parameters are presented in Table 2 and Table 3 for MINDE-J and MINDE-C respectively. (Section C.3) and these tables specify 'Width', 'Time embed', 'Batch size', 'Lr', and 'Iterations'. |