What’s Hot in the Answer Set Programming Competition
Authors: Martin Gebser, Marco Maratea, Francesco Ricca
AAAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The ASP competition series aims at assessing and promoting the evolution of ASP systems and applications. Its growing range of challenging application-oriented benchmarks inspires and showcases continuous advancements of the state of the art in ASP. ... the sixth edition of the ASP competition aimed at balancing the hardness of instances selected per benchmark domain. To this end, an instance selection process inspired by the 2014 SAT Competition3 has been incorporated. First, the empirical hardness of all available instances has been evaluated by running the three top-performing systems from the previous edition, and then a balanced selection was made among instances of varying difficulty. Running the three reference systems exhaustively took about 212 CPU days on the competition platform and led to a classification of the available instances. |
| Researcher Affiliation | Academia | Martin Gebser University of Potsdam, Germany gebser@cs.uni-potsdam.de Marco Maratea Universit a di Genova, Italy marco@dibris.unige.it Francesco Ricca Universit a della Calabria, Italy ricca@mat.unical.it |
| Pseudocode | No | The paper describes the instance selection process, noting it 'has itself been implemented in ASP', but it does not provide pseudocode or an algorithm block within the paper. |
| Open Source Code | No | The paper refers to the competition website for full details ('http://aspcomp2015.dibris.unige.it/') and mentions that the instance selection process was implemented in ASP, but it does not explicitly state that the source code for the methodology or analysis presented in *this* paper is open-source or provide a direct link to it. |
| Open Datasets | Yes | The competition benchmarks included 26 domains for which encodings and instances were already available from earlier editions (Calimeri, Ianni, and Ricca 2014). For four of these domains, the benchmark authors kindly provided fresh and more challenging instance sets, suitable for making meaningful system comparisons. Application-oriented benchmarks from six new domains were submitted in addition, thus doubling the number of domains stemming from applications. ... The material collected on the website4 of the sixth edition of the ASP competition contributes a broad range of benchmark domains, empirical hardness data for all instances provided by benchmark authors, as well as a so-called uniform encoding of balanced instance selection. |
| Dataset Splits | Yes | Running the three reference systems exhaustively took about 212 CPU days on the competition platform and led to a classification of the available instances. Instances solved in less than 20 seconds by each of the three systems as well as those where all of them failed in the grounding phase were classified as too easy or non-groundable. ... The other instances are partitioned into four hardness categories. The instance selection was then balanced by aiming at 20% of instances from each category plus another 20% picked freely, where concrete instances are drawn at random among the respective candidates. ... While 20 instances per domain were picked for the competition, the selection process can be adopted and customized to furnish other representative benchmark suites in the future. |
| Hardware Specification | No | The paper states, 'Running the three reference systems exhaustively took about 212 CPU days on the competition platform', but it does not provide specific hardware details such as CPU models, memory, or GPU specifications. |
| Software Dependencies | No | The paper mentions 'ASP-Core2 format' and various ASP systems (e.g., ME-ASP, WASP+DLV, LP2NORMAL+CLASP) but does not specify version numbers for these or any other ancillary software dependencies. |
| Experiment Setup | Yes | In the Main track, each system was allotted 20 minutes per run, including the time for grounding, preprocessing, and search. ... As an idea borrowed from past QBF evaluations, the three top-performing systems were granted an order of magnitude more time per instance, i.e., 3 hours rather than 20 minutes only, in the Marathon track. |