Learning to Learn in Interactive Constraint Acquisition
Authors: Dimosthenis Tsouros, Senne Berden, Tias Guns
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we perform an experimental evaluation of our proposed approaches, aiming to answer the following research questions: |
| Researcher Affiliation | Academia | Dimos Tsouros, Senne Berden, Tias Guns KU Leuven, Belgium dimos.tsouros@kuleuven.be, senne.berden@kuleuven.be, tias.guns@kuleuven.be |
| Pseudocode | Yes | Algorithm 1: Generic Constraint Acquisition Template |
| Open Source Code | Yes | 1Our code is available online at: https://github.com/Dimosts/ActiveConLearn |
| Open Datasets | No | The paper describes the problem instances used as "benchmarks" (e.g., "9x9 Sudoku", "Exam Timetabling", "Nurse rostering") and their characteristics (number of variables, domains, target constraint network size, bias size). However, it does not provide concrete access information (URLs, DOIs, specific repository names, or formal citations to external public datasets) for a 'training dataset' in the typical machine learning sense. The problems seem to be defined by their structure and parameters rather than being external, pre-existing datasets with access information. |
| Dataset Splits | Yes | We used an increasing portion of the dataset as the training set, to evaluate their performance in different stages of the acquisition process, with the rest of the candidates being the test set." and "A grid search, coupled with 10-fold cross-validation, was conducted, using balanced accuracy as the metric to address class imbalance. |
| Hardware Specification | Yes | All the experiments were conducted on a system carrying an Intel(R) Core(TM) i7-2600 CPU, 3.40GHz clock speed, with 16 GB of RAM. |
| Software Dependencies | No | The paper mentions software like Python, CPMpy, OR-Tools CP-SAT, and Scikit-Learn, but does not provide specific version numbers for these components, only citing papers for some of them. |
| Experiment Setup | Yes | We used RF and GNB in their default settings, while we tuned the most important hyperparameters for MLP and SVM. For tuning, we used the final dataset for all benchmarks, having labeled all candidate constraints. A grid search, coupled with 10-fold cross-validation, was conducted, using balanced accuracy as the metric to address class imbalance. Hyperparameter combinations surpassing a 10-second training time were omitted to ensure relevance in interactive scenarios. For query generation, we used PQ-Gen from [Tsouros, Berden, Guns, 2023], with a cutoff of 1 second to return the best query found. |