Using Large Language Models to Improve Query-based Constraint Acquisition
Authors: Younes Mechqrane, Christian Bessiere, Ismail Elabbassi
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we first describe how our BERT component is fine-tuned. We then introduce the benchmark problems used to evaluate LLMACQ and to compare it to QUACQ2. Finally, we report the results of acquiring problems with LLMACQ and QUACQ2 and we analyze these results. |
| Researcher Affiliation | Academia | 1Ai movement International Artificial Intelligence Center of Morocco University Mohammed VI Polytechnic, Rabat, Morocco 2CNRS, University of Montpellier, France |
| Pseudocode | Yes | Algorithm 1: ACQNOGOODS |
| Open Source Code | Yes | The source code used to obtained the results reported in this paper is available at https://github.com/mechqrane/LLmAcq. |
| Open Datasets | No | For our experiments we trained BERT on a vocabulary X = {x1, . . . , x100}, D = {1, . . . , 10}, and a language Γ = {...} ... We used CHATGPT-4 to generate templates for the relations in Γ. ... We allocated 80% of this generated data for training purposes and the remaining 20% for validation. ... The paper describes how it generated its own data but does not provide public access or specific citations for it. |
| Dataset Splits | Yes | We allocated 80% of this generated data for training purposes and the remaining 20% for validation. |
| Hardware Specification | No | Fine-tuning BERT was done on Google Colab GPUaccelerated environment. |
| Software Dependencies | No | We used the PyTorch implementation of BERT BASE and Adam W optimizer [Loshchilov and Hutter, 2019] with a learning rate of 10 5 and a batch size of 128. ... The solver used in line 4 is Google OR-Tools CP-SAT. |
| Experiment Setup | Yes | We used the PyTorch implementation of BERT BASE and Adam W optimizer [Loshchilov and Hutter, 2019] with a learning rate of 10 5 and a batch size of 128. The maximum number of epochs was set to 40. ... The value of the cutoff max Pos was set to 10. |