Abstract Diagrammatic Reasoning with Multiplex Graph Networks
Authors: Duo Wang, Mateja Jamnik, Pietro Lio
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin. |
| Researcher Affiliation | Academia | Duo Wang & Mateja Jamnik & Pietro Lio Department of Computer Science and Technology University of Cambridge Cambridge, United Kingdom {Duo.Wang,Mateja.Jamnik,Pietro.Lio}@cl.cam.ac.uk |
| Pseudocode | No | The paper describes its architecture and components in detail with diagrams, but it does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide a link to open-source code for the described methodology or explicitly state its release. |
| Open Datasets | Yes | For PGM dataset (Barrett et al. (2018)), MXGNet outperforms WRe N, the previous state-of-the-art model, by a considerable margin. For the RAVEN dataset (Zhang et al. (2019)), MXGNet, without any auxiliary training with additional labels, achieves 83.91% test accuracy, outperforming 59.56% accuracy by the best model with auxiliary training for the RAVEN dataset. |
| Dataset Splits | Yes | Table 2 shows validation and test accuracies for all three data regimes with and without auxiliary training. |
| Hardware Specification | Yes | We used batch size of 64, and distributed the training across 2 Nvidia Geforce Titan X GPUs. |
| Software Dependencies | No | The architecture is implemented in Pytorch framework. During training, we used RAdam optimiser (Liu et al. (2019)). The paper mentions 'Pytorch framework' and 'RAdam optimiser' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | During training, we used RAdam optimizer Liu et al. (2019) with learning rate 0.0001, β1 = 0.9,β2 = 0.999. We used batch size of 64 |