A Test of Relative Similarity for Model Selection in Generative Models
Authors: Eugene Belilovsky, Wacha Bounliphone, Matthew Blaschko, Ioannis Antonoglou, Arthur Gretton
ICLR 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In experiments on deep generative models, including the variational auto-encoder and generative moment matching network, the tests provide a meaningful ranking of model performance as a function of parameter and training settings. |
| Researcher Affiliation | Collaboration | Wacha Bounliphone,12 Eugene Belilovsky,12 & Matthew B. Blaschko2 1Centrale Sup elec & Inria Saclay, Universit e Paris-Saclay, 92295 Chˆatenay-Malabry, France 2ESAT-PSI, KU Leuven, Kasteelpark Arenberg 10, 3001 Leuven, Belgium {wacha.bounliphone,eugene.belilovsky}@inria.fr matthew.blaschko@esat.kuleuven.be Ioannis Antonoglou Google Deepmind 5 New Street Square London EC4A 3TW, UK ioannisa@google.com Arthur Gretton Gatsby Computational Neuroscience Unit University College London 25 Howland Street London W1T 4HG, UK arthur.gretton@gmail.com |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | We have made code for performing the test is available.1 Code and examples can be found at https://github.com/eugenium/MMD and Code for our method is available.2 https://github.com/eugenium/MMD |
| Open Datasets | Yes | We use the MNIST and Frey Face datasets for our analysis (Le Cun et al., 1998; Kingma and Welling, 2014; Goodfellow et al., 2014). |
| Dataset Splits | Yes | We use 1500 Frey Face samples as the target in Relative MMD and 15000 images from MNIST. and We use another 20000 images as the target data in Relative MMD. and We use the same training set of 55000, validation set of 5000 and test set of 10000 as in (Li et al., 2015b; Goodfellow et al., 2014). |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for experiments (e.g., GPU/CPU models, memory specifications). |
| Software Dependencies | No | The paper mentions general software components like 'backpropagation' and refers to associated software from cited works, but it does not specify version numbers for any libraries, frameworks, or solvers (e.g., PyTorch 1.9, TensorFlow 2.x). |
| Experiment Setup | Yes | We use the architecture from Kingma and Welling (2014) with a hidden layer at both the encoder and decoder and a latent variable layer as shown in Figure 5a. We use sigmoidal activation for the hidden layers of encoder and decoder. For the Frey Face data, we use a Gaussian prior on the latent space and data space. For MNIST, we used a Bernoulli prior for the data space. We have 400 hidden nodes (both encoder and decoder) and 20 latent variables in the reference model for our experiments. The auto-encoder (indicated in orange) is trained separately and has 1024 and 32 hidden nodes in decode and encode hidden layers. The GMMN has 10 variables generated by the prior, and the hidden layers have 64, 256, 256, 1024 nodes in each layer respectively. |