DppNet: Approximating Determinantal Point Processes with Deep Networks
Authors: Zelda E. Mariet, Yaniv Ovadia, Jasper Snoek
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we show across multiple datasets that DPPNET is orders of magnitude faster than competing approaches for DPP sampling, while generating high-likelihood samples and performing as well as DPPs on downstream tasks. |
| Researcher Affiliation | Collaboration | Zelda Mariet Massachusetts Institute of Technology Cambridge, Massachusetts 02139, USA zelda@csail.mit.edu Yaniv Ovadia & Jasper Snoek Google Brain Cambridge, Massachusetts 02139, USA {yovadia, jsnoek}@google.com |
| Pseudocode | Yes | Algorithm 1 Sampling and greedy mode for DPPNET Input: Initial set S, target size k, feature matrix Φ while |S| < k do v DPPNET(S, Φ) if sampling then i Multinomial(v/ v ) else if greedy mode then i argmax v S S {i} return S |
| Open Source Code | No | The paper does not provide an explicit statement about open-sourcing the code or a link to a code repository. |
| Open Datasets | Yes | We evaluate the performance of DPPNETs on varying ground set sizes through the MNIST [29], Celeb A [34], and Movie Lens [20] datasets. |
| Dataset Splits | No | The paper mentions training and testing sets, but does not explicitly specify the splits for training, validation, and testing. It states: 'We train DPPNET with the embeddings corresponding to randomly subsampled ground sets of size N = 100 of the training sets of each dataset; during testing (i.e., in the results below), the trained models are fed feature representations from the corresponding test sets.' |
| Hardware Specification | No | The paper states 'while all methods were run on CPU, DPPNET is amenable to further acceleration using GPUs,' but does not provide specific hardware details like CPU or GPU models, or memory amounts. |
| Software Dependencies | No | The paper mentions 'Our models are trained with Tensor Flow using the Adam optimizer' but does not specify the version number for TensorFlow or any other software dependencies. |
| Experiment Setup | Yes | Bandwidths were set to β = 0.0025 for MNIST and β = 0.1 for Celeb A, chosen in order to obtain a DPP sample size 20: for a DPP with kernel L, the expected sample size is given by ES PL[|S|] = Tr[L(L + I) 1]. For MNIST, Figure 4 shows images selected by the baselines and the DPPNET, chosen among 100 digits with all identical labels; visually, DPPNET and DPP samples provide a wider coverage of writing styles. However, the NLL of samples from DPPNET decay significantly, whereas the DPPNET mode maintains competitive performance with DPP samples. For this reason, all further experiments focus on greedy mode samples drawn from the DPPNET. For MNIST, Figure 4 shows images selected by the baselines and the DPPNET, chosen among 100 digits with all identical labels; visually, DPPNET and DPP samples provide a wider coverage of writing styles. However, the NLL of samples from DPPNET decay significantly, whereas the DPPNET mode maintains competitive performance with DPP samples. For this reason, all further experiments focus on greedy mode samples drawn from the DPPNET. For the MNIST encodings, the VAE encoder consists of a 2d-convolutional layer with 64 filters of height and width 4 and strides of 2, followed by a 2d convolution layer with 128 filters (same height, width and strides), then by a dense layer of 1024 neurons. The encodings are of length 32. Celeb A encodings were generated by a VAE using a Wide Residual Network [49] encoder with 10 layers and filter-multiplier k = 4, a latent space of 32 full-covariance Gaussians, and a deconvolutional decoder trained end-to-end using an ELBO loss. |