S$Ω$I: Score-based O-INFORMATION Estimation
Authors: Mustapha Bounoua, Giulio Franzese, Pietro Michiardi
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments validate our approach on synthetic data, and demonstrate the effectiveness of SΩI in the context of a real-world use case. |
| Researcher Affiliation | Collaboration | 1Ampere Software Technology, France 2Department of Data Science, Eurecom, France. |
| Pseudocode | Yes | Algorithm 1 SΩI Training step |
| Open Source Code | Yes | We provide code-base for SΩI implementation at 1. The training of SΩI is carried out using Adam optimizer (Kingma & Ba, 2015). 1https://github.com/Mustapha Bounoua/soi |
| Open Datasets | Yes | We consider the Visual Behavior project, which used the Allen Brain Observatory to collect a highly standardized dataset consisting of recordings of neural activity in mice (Allen-Institute, 2022). |
| Dataset Splits | No | For each experiment, we use 100k samples for training the various neural estimators, and 10k samples at inference time, to estimate O-INFORMATION. The paper explicitly mentions training and testing sets, but a separate validation split is not specified. |
| Hardware Specification | No | No specific hardware details such as GPU or CPU models, memory, or cluster specifications are provided in the paper. |
| Software Dependencies | No | The paper mentions software components like 'Adam optimizer' and frameworks like 'VP-SDE formulation', but it does not specify exact version numbers for any of them (e.g., PyTorch 1.9, Python 3.8). |
| Experiment Setup | Yes | For our method SΩI, we use the VP-SDE formulation (Song et al., 2021) and learn a unique denoising network to estimate the various score terms. The denoiser is a simple, stacked multilayer perceptron (MLP) with skip connections, adapted to the input dimension. We apply importance sampling (Huang et al., 2021; Song et al., 2021) at both training and inference time. Finally, we use 10-sample Monte Carlo estimates for computing integrals. (Table 1 provides hyperparameters like Width, Time embed, Batch size, Lr, Iterations). |