Analysis for Abductive Learning and Neural-Symbolic Reasoning Shortcuts
Authors: Xiao-Wen Yang, Wen-Da Wei, Jie-Jing Shao, Yu-Feng Li, Zhi-Hua Zhou
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical studies demonstrate the rationality of the analysis. Moreover, the proposal is suitable for many ABL and Ne Sy algorithms and can be easily extended to handle other cases of reasoning shortcuts.In this section, we empirically corroborate two principal findings that have previously been supported by theoretical evidence: (1). As the complexity of the knowledge base increases, both the neural-symbolic approaches and the ABL algorithms exhibit lower shortcut risk. (2). The selection of the distance function Dis influences the performance of the ABL algorithm significantly, where a good choice of Dis can assist in alleviating the reasoning shortcuts. |
| Researcher Affiliation | Academia | 1National Key Laboratory for Novel Software Technology, Nanjing University, China. 2School of Artificial Intelligence, Nanjing University, China. |
| Pseudocode | No | The paper describes methods textually and with mathematical formulations but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about open-source code availability or links to a code repository for the described methodology. |
| Open Datasets | Yes | We consider the MNIST-Addition experiment (Manhaeve et al., 2019). The input of this task is a pair of MNIST images (Le Cun et al., 1998), and the output is the sum of the individual digits.BDD-OIA (Xu et al., 2020) is a commonly used dataset, which predicts the current feasible actions (Y = {move forward, stop, turn left, turn right}). |
| Dataset Splits | No | For the MNIST-Addition task, the paper states 'each dataset has a fixed training sample size of 30,000 and the test dataset remains unchanged.' For BDD-OIA, 'The training set consists of 16,000 frames, while the test set contains 4,500 annotated data points.' No explicit validation split is mentioned. |
| Hardware Specification | Yes | All our experiments are implemented by Pytorch and are conducted on an NVIDIA A800. |
| Software Dependencies | No | The paper mentions 'Pytorch' as the implementation framework and 'Adam optimizer (Kingma & Ba, 2015)' but does not provide specific version numbers for PyTorch or other libraries, which is required for reproducibility. |
| Experiment Setup | Yes | For ABL algorithms, we use the Adam optimizer (Kingma & Ba, 2015) with the learning rate of 3e 4 to train our networks. We use the Adam optimizer (Kingma & Ba, 2015) with the learning rate of 5e 3 to train our networks. The loss function to train the ABL algorithms is BCELoss. |