Are Disentangled Representations Helpful for Abstract Visual Reasoning?
Authors: Sjoerd van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, Olivier Bachem
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we conduct a large-scale study that investigates whether disentangled representations are more suitable for abstract reasoning tasks. Using two new tasks similar to Raven s Progressive Matrices, we evaluate the usefulness of the representations learned by 360 state-of-the-art unsupervised disentanglement models. Based on these representations, we train 3600 abstract reasoning models and observe that disentangled representations do in fact lead to better down-stream performance. |
| Researcher Affiliation | Collaboration | Sjoerd van Steenkiste IDSIA, USI, SUPSI sjoerd@idsia.ch Francesco Locatello ETH Zurich, MPI-IS locatelf@ethz.ch Jürgen Schmidhuber IDSIA, USI, SUPSI, NNAISENSE juergen@idsia.ch Olivier Bachem Google Research, Brain Team bachem@google.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is made available as part of disentanglement_lib at https://git.io/JelEv. |
| Open Datasets | Yes | We create two new visual abstract reasoning tasks similar to Raven s Progressive Matrices [61] based on two disentanglement data sets: d Sprites [27], and 3dshapes [42]. |
| Dataset Splits | Yes | We train each of these models using a batch size of 32 for 100K iterations where each mini-batch consists of newly generated random instances of the abstract reasoning tasks. Similarly, every 1000 iterations, we evaluate the accuracy on 100 mini-batches of fresh samples. |
| Hardware Specification | Yes | Reproducing these experiments requires approximately 2.73 GPU years (NVIDIA P100). |
| Software Dependencies | No | The paper mentions that code is available as part of 'disentanglement_lib' but does not specify particular software dependencies with version numbers (e.g., Python, PyTorch/TensorFlow, CUDA versions). |
| Experiment Setup | Yes | We train each of these models using a batch size of 32 for 100K iterations where each mini-batch consists of newly generated random instances of the abstract reasoning tasks. We first sample a set of 10 hyper-parameter configurations from our search space and then trained WRe N models using these configurations... The remaining experimental details are presented in Appendix A. |