Function-space regularized Rényi divergences
Authors: Jeremiah Birrell, Yannis Pantazis, Paul Dupuis, Luc Rey-Bellet, Markos Katsoulakis
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present numerical results on both synthetic and real datasets, showing the utility of these new divergences in both estimation and GAN training applications; in particular, we demonstrate significantly reduced variance and improved training performance. |
| Researcher Affiliation | Academia | Jeremiah Birrell1, Yannis Pantazis2, Paul Dupuis3, Luc Rey-Bellet1, Markos A. Katsoulakis1 1University of Massachusetts, Amherst, 2Foundation for Research & Technology Hellas, 3Brown University, {jbirrell, luc, markos}@umass.edu, pantazis@iacm.forth.gr, paul_dupuis@brown.edu |
| Pseudocode | No | The paper contains mathematical derivations and theoretical results but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about making the source code available or links to a code repository. |
| Open Datasets | Yes | We consider single cell mass cytometry measurements on 16 bone marrow protein markers (d = 16) coming from healthy and disease individuals with acute myeloid leukemia Levine et al. (2015). Following Weber et al. (2019), we create two datasets... Also, on the CIFAR-10 dataset Krizhevsky et al. (2009). And on the Rot MNIST dataset, obtained from randomly rotating the original MNIST digit dataset Le Cun et al. (1998). |
| Dataset Splits | No | The paper details training parameters and test set evaluation but does not specify the methodology or size of any validation splits. |
| Hardware Specification | No | The paper mentions using 'high performance computing equipment' in the acknowledgments but does not provide specific details such as CPU/GPU models or memory. |
| Software Dependencies | No | The paper mentions the use of 'Adam optimizer' and 'ReLU activations' and cites related neural network architectures, but it does not provide specific version numbers for software libraries or dependencies used for implementation. |
| Experiment Setup | Yes | We used a NN with one fully connected layer of 64 nodes, ReLU activations, and a poly-softplus final layer (for CC-Rényi). We trained for 10000 epochs with a minibatch size of 500. |