Neural Methods for Point-wise Dependency Estimation
Authors: Yao-Hung Hubert Tsai, Han Zhao, Makoto Yamada, Louis-Philippe Morency, Russ R. Salakhutdinov
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of our approaches in 1) MI estimation, 2) self-supervised representation learning, and 3) cross-modal retrieval task. We empirically analyze the advantages of PD neural estimation on three applications. First, we cast the challenging MI estimation problem to be a PD estimation problem. Second, our PD estimation objectives also inspire new losses for contrastive self-supervised representation learning. Third, we study the use of PD estimation for data containing information across modalities. We empirically analyze the advantages of PD neural estimation on three applications. Benchmarking on Correlated Gaussians. |
| Researcher Affiliation | Collaboration | Yao-Hung Hubert Tsai1, Han Zhao2 , Makoto Yamada34, Louis-Philippe Morency1, Ruslan Salakhutdinov1 1Carnegie Mellon University, 2D.E. Shaw & Co., 3 Kyoto University, 4RIKEN AIP |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled as 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | We make our experiments publicly available at https://github.com/yaohungt/ Pointwise_Dependency_Neural_Estimation. |
| Open Datasets | Yes | Benchmarking on Correlated Gaussians... MNIST [30] and CIFAR10 [29]. The estimator is trained on the training split. |
| Dataset Splits | No | While the paper mentions 'training split' and 'test split', it does not provide specific percentages or counts for these splits, nor does it explicitly detail a separate 'validation' split with reproducible information in the main text. It references standard datasets but does not define the splits used for them. |
| Hardware Specification | No | The paper mentions 'NVIDIA s GPU support' but does not specify any particular GPU model, CPU, or other detailed hardware specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions TensorFlow [1] and PyTorch [38] as frameworks, but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | No | The paper states 'Network, learning rate, optimizer, and batch size are fixed for all MI neural estimators' and 'Network, learning rate, optimizer, and batch size are fixed for all the methods', but it does not provide the concrete values for these hyperparameters in the main text. It refers to 'more training details in Supplementary'. |