Neural Algorithmic Reasoning with Causal Regularisation
Authors: Beatrice Bevilacqua, Kyriacos Nikiforou, Borja Ibarz, Ioana Bica, Michela Paganini, Charles Blundell, Jovana Mitrovic, Petar Veličković
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on the CLRS algorithmic reasoning benchmark, where we show up to 3 improvements on the OOD test data. We conducted an extensive set of experiments to answer the following main questions: Can our model, Hint-Re LIC, which relies on the addition of our causality-inspired self-supervised objective, outperform the corresponding base model in practice? What is the importance of such objective when compared to other changes made with respect to the previous state-of-the-art model? How does Hint-Re LIC compare to a model which does not leverage hints at all, directly predicting the output from the input? Are hints necessary? Results. Figure 6 compares the out-of-distribution (OOD) performances of the Triplet-GMPNN baseline, which we have re-trained and evaluated in our experiments, to our model Hint-Re LIC, as described above. |
| Researcher Affiliation | Collaboration | 1Purdue University 2Deep Mind. Work done while Beatrice Bevilacqua was at Deep Mind. Correspondence to: Beatrice Bevilacqua <bbevilac@purdue.edu>. |
| Pseudocode | No | The paper describes algorithms and methods but does not provide any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that the code for their methodology is open-source or publicly available. |
| Open Datasets | Yes | We evaluate our method on the CLRS algorithmic reasoning benchmark, where we show up to 3 improvements on the OOD test data. This benchmark, namely the CLRS algorithmic benchmark, represents data as graphs, showing that the graph formulation is general enough to include several algorithms, and not just the graph-based ones. On the CLRS benchmark, Ibarz et al. (2022) has recently presented several improvements in the architecture and learning procedure in order to obtain better performances. |
| Dataset Splits | No | The paper mentions training and testing but does not explicitly state the dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments (e.g., GPU models, CPU types, memory). |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | We use the best hyperparameters of the Triplet-GMPNN (Ibarz et al., 2022) base model, and we only reduce the batch size to 16. We set the temperature parameter τ to 1e 1 and the weight of the KL loss α to 1. We implement the similarity function as ϕ(f(xal t , it), f(xak t , it)) = h(f(xal t ), it), h(f(xak t , it)) /τ with h a two-layers MLP with hidden and output dimensions equal to the input one, and Re LU non-linearities. |