Categorical Normalizing Flows via Continuous Transformations
Authors: Phillip Lippe, Efstratios Gavves
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We start our experiments by evaluating Graph CNF on two benchmarks for graph generation, namely molecule generation and graph coloring. Further, to test generality we evaluate CNFs on other categorical problems, specifically language and set modeling. For the normalizing flows, we use a sequence of logistic mixture coupling layers (Ho et al., 2019) mapping a mixture of logistic distributions back into a single mode. Before each coupling layer, we include an activation normalization layer and invertible 1x1 convolution (Kingma and Dhariwal, 2018). For reproducibility, we provide all hyperparameter details in Appendix D, and make our code publicly available. |
| Researcher Affiliation | Academia | Phillip Lippe University of Amsterdam, QUVA lab p.lippe@uva.nl Efstratios Gavves University of Amsterdam egavves@uva.nl |
| Pseudocode | Yes | Algorithm 1 Training procedure for the logistic mixture encoding in CNFs |
| Open Source Code | Yes | For reproducibility, we provide all hyperparameter details in Appendix D, and make our code publicly available.1 Code available here: https://github.com/phlippe/Categorical NF |
| Open Datasets | Yes | We perform experiments on the Zinc250k (Irwin et al., 2012) dataset which consists of 250,000 drug-like molecules. ... We also evaluated our model on the Moses (Polykovskiy et al., 2018) dataset and achieved similar scores as shown in Appendix C. ... We experiment with two popular character-level datasets, Penn Treebank (Marcus et al., 1994) and text8 (Mahoney, 2011) ... We also test a word-level dataset, Wikitext103 (Merity et al., 2017) ... The data contains different categorical attributes regarding potential credit risk, and we have used the following 9 attributes: checking_status , credit_history , savings_status , employment , housing , job , own_telephone , foreign_worker , and class . The task is to model the joint density function of those 9 attributes over a dataset of 1000 entries. We have used the first 750 entries for training, 100 for validation, and 150 for testing. |
| Dataset Splits | Yes | Overall, we create a train/val/test size of 192k/24k/24k for the small dataset, and 450k/20k/30k for the large graphs. ... The Zinc250k (Irwin et al., 2012) dataset we use contains 239k molecules of which we use 214k molecules for training, 8k for validation, and 17k for testing. ... We train and test the models on a sequence length of 256. ... We have used the first 750 entries for training, 100 for validation, and 150 for testing. |
| Hardware Specification | Yes | The experiments for graph coloring and molecule generation have been executed on a single NVIDIA Titan RTX GPU. The set and language experiments have been executed on a single NVIDIA GTX1080Ti in 4 to 16 hours. |
| Software Dependencies | Yes | All experiments have been implemented using the deep learning framework Py Torch (Paszke et al., 2019). |
| Experiment Setup | Yes | For reproducibility, we provide all hyperparameter details in Appendix D, and make our code publicly available. ... We summarize our hyperparameters in Table 9. |