On the Paradox of Learning to Reason from Data
Authors: Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, Guy Van den Broeck
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We attempt to answer this question by training and testing a neural model (e.g. BERT [Devlin et al., 2019]) on a confined problem space (see Fig. 1 and Sec. 2) consisting of logical reasoning problems written in English [Johnson et al., 2017; Sinha et al., 2019]. |
| Researcher Affiliation | Academia | Honghua Zhang , Liunian Harold Li , Tao Meng , Kai-Wei Chang and Guy Van den Broeck University of California, Los Angeles {hzhang19, liunian.harold.li, tmeng, kwchang, guyvdb}@cs.ucla.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | 3https://github.com/joshuacnf/paradox-learning2reason |
| Open Datasets | No | The paper describes generating its own datasets (RP and LP) using specific sampling algorithms, but it does not provide concrete access information (link, DOI, repository, or formal citation with authors/year) for these specific datasets to be publicly available. |
| Dataset Splits | No | The paper states 'we then split it as training/validation/test set' but does not provide specific percentages or sample counts for these splits. It also mentions 'See training details in appendix4' for a paper (https://arxiv.org/abs/2205.11502) but does not contain the details in the provided text. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'BERT-base model' but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | No | The paper states 'See training details in appendix4' which refers to an external arXiv paper (2205.11502) and thus does not provide specific experimental setup details such as hyperparameters or training configurations within the main text of this paper. |