The continuous categorical: a novel simplex-valued exponential family
Authors: Elliott Gordon-Rodriguez, Gabriel Loaiza-Ganem, John Cunningham
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we demonstrate that the continuous categorical outperforms standard choices empirically, across a simulation study, an applied example on multi-party elections, and a neural network compression task. |
| Researcher Affiliation | Collaboration | 1Department of Statistics, Columbia University 2Layer 6 AI. |
| Pseudocode | Yes | Algorithm 1 Ordered rejection sampler |
| Open Source Code | Yes | 1Our code is available at https://github.com/ cunningham-lab/cb_and_cc |
| Open Datasets | Yes | We next consider a real-world example from the 2019 UK general election (Uberoi et al., 2019). |
| Dataset Splits | No | The paper mentions 'We split the data into an 80/20 training and test set at random.' for the UK Election Data, but does not explicitly provide details about a validation set or a three-way split for any dataset. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments (e.g., GPU/CPU models, memory specifications, or cloud instances). |
| Software Dependencies | No | The paper mentions using 'Adam (Kingma & Ba, 2015)' as an optimizer but does not specify its version or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Secondly, we extend our linear model to a more flexible neural network by adding a hidden layer with 20 units and Re LU activations. We train both models using Adam (Kingma & Ba, 2015). We build on the MNIST experiment of Hinton et al. (2015); we first train a teacher neural net with two hidden layers of 1200 units and Re LU activations, regularized using batch normalization, and we use its fitted values to train smaller student networks with a single hidden layer of 30 units. We can also bring them towards the centroid while conserving their relative order by varying the temperature, T, of the final softmax (Hinton et al., 2015). |