Stability Regularization for Discrete Representation Learning
Authors: Adeel Pervez, Efstratios Gavves
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the method in a broad range of settings, showing competitive performance against the state-of-the-art. We validate our stability regularization procedure on a number of tasks with Bernoulli and categorical variables to show its effectiveness and wide applicability. |
| Researcher Affiliation | Academia | Anonymous authors Paper under double-blind review. Due to double-blind review, explicit institutional affiliations are not provided in the paper text. |
| Pseudocode | Yes | Algorithm 1 Stability Regularization. Algorithm 2 Stability Regularization with Mean Centering. |
| Open Source Code | No | The paper mentions using 'the authors Py Torch implementation' or 'the authors code' when comparing against baselines (e.g., in Section 5.5 and 5.3), but it does not explicitly state that the authors' own source code for the method described in this paper is available or provide a link. |
| Open Datasets | Yes | We perform experiments with categorical VAE on dynamically binarized MNIST. Physics Simulation Experiments The first set of NRI experiments are physics simulations from the original NRI proposal Kipf et al. (2018). We train the model on CIFAR-10 with 50k 32x32 labeled images and STL-10 with 100k 96x96 unlabeled images. |
| Dataset Splits | Yes | Synthetic data was generated using the authors code with 50k training and 10k validation and test samples each. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., CPU, GPU models, memory specifications) used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'Adam' for optimization and 'Py Torch implementation' (Section 5.5) but does not specify version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | We train all experiments for 500 epochs. We train using Adam with a learning rate of 1e-4. For Gumbel-Softmax we use temperatures in {0.1, 0.5, 1.0}. The stability regularization coefficients are chosen from {20, 50, 100}. The parameter ρ is set to 0.9. We use ρ {0.8, 0.9, 0.95} and a stability constraint of 0.6. We train with Adam with a learning rate of 8e-5. |