Learning Higher-Order Logic Programs From Failures
Authors: Stanisław J. Purgał, David M. Cerna, Cezary Kaliszyk
IJCAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show Hopper (when optimized) is capable of outperforming Popper on most tasks, especially the harder tasks we introduced in this work (Section 4). ... We ran Popper using optimal settings and minimal BK. ... Table 1: We ran Popper, Hopper, optimized Hopper , and Metagol HO on a single core with a timeout of 300 second. Times denote the average of 5 runs. |
| Researcher Affiliation | Academia | Stanisław J. Purgał1 , David M. Cerna2,3 , Cezary Kaliszyk1 1University of Innsbruck, Innsbruck, Austria 2Czech Academy of Sciences Institute of Computer Science (CAS ICS), Prague, Czechia 3Research Institute for Symbolic Computation (RISC), Johannes Kepler University, Linz, Austria |
| Pseudocode | No | The paper describes logical definitions and programs but does not include any explicitly labeled pseudocode blocks or algorithms. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing its own source code for Hopper or a link to a repository for its implementation. It mentions a GitHub link for 'Metagol system' which is a third-party system. |
| Open Datasets | Yes | We reevaluated 7 of the tasks presented in [Cropper and Morel, 2021a] and 2 presented in [Cropper et al., 2020]. Additionally, we added 8 list manipulation tasks, 3 tree manipulation tasks, and 2 arithmetic tasks (separated by type in Table 1). |
| Dataset Splits | No | The paper describes input as 'sets of positive E + and negative E examples' for Popper, but does not provide specific details on how datasets were split into training, validation, or test sets with percentages, counts, or explicit splitting methodologies. |
| Hardware Specification | No | The paper states, 'We ran Popper, Hopper, optimized Hopper , and Metagol HO on a single core', which is a general description but lacks specific hardware details such as CPU/GPU models, memory, or clock speeds. |
| Software Dependencies | No | The paper mentions Popper uses a 'multi-shot solving framework' and 'ASP paradigm', citing [Gebser et al., 2019] and [Lifschitz, 2019], which refer to Clingo and Answer Set Programming. However, it does not explicitly state specific version numbers for any key software components or libraries used in their implementation of Hopper. |
| Experiment Setup | Yes | We ran Popper using optimal settings and minimal BK. ... For Popper and Hopper, settings such as max var significantly impact performance. ... The parameters max var and max body have a significant impact on the size of the single clause hypothesis space. |