Out-of-Distribution Generalization by Neural-Symbolic Joint Training
Authors: Anji Liu, Hongming Xu, Guy Van den Broeck, Yitao Liang
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that our proposed framework is capable of achieving superior outof-distribution generalization performance on two tasks: (i) learning multi-digit addition, where it is trained on short sequences of digits and tested on long sequences of digits; (ii) predicting the optimal action in the Tower of Hanoi, where the model is challenged to discover a policy independent of the number of disks in the puzzle. [...] In this section, we evaluate the proposed NTOC model on the multi-digit addition task and the Tower of Hanoi action prediction task. |
| Researcher Affiliation | Collaboration | Anji Liu2, Hongming Xu3, Guy Van den Broeck2, Yitao Liang1,3 1 Institute for Artificial Intelligence, Peking University 2 Computer Science Department, University of California, Los Angeles 3 Beijing Institute of General Artificial Intelligence (BIGAI) {liuanji, guyvdb}@cs.ucla.edu, xuhongming@bigai.ai, yitaol@pku.edu.cn |
| Pseudocode | Yes | Algorithm 1: Symbolic Rule Search |
| Open Source Code | Yes | Our code can be found at https://github.com/sbx126/NToC. |
| Open Datasets | Yes | Given two numbers represented by a sequence of MNIST images, the goal is to predict the sum of both numbers. [...] For all methods, 100 MNIST images are given for pretraining. |
| Dataset Splits | No | The paper mentions training and testing sets ('training the models on numbers of length 3 and test them with numbers of length 5-20', '1,000 training samples, while testing them on 10,000 samples') but does not specify a separate validation split or its size/percentage. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python version, library versions like PyTorch or TensorFlow). |
| Experiment Setup | No | The paper provides minimal experimental setup details such as the number of pretraining samples (e.g., '100 MNIST images are given for pretraining', '500 samples to teach the model') and training samples ('1,000 training samples'), but lacks specific hyperparameters like learning rates, batch sizes, optimizers, or detailed training schedules. |