Solving Explainability Queries with Quantification: The Case of Feature Relevancy
Authors: Xuanxiang Huang, Yacine Izza, Joao Marques-Silva
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results, obtained using random forests (RFs) induced from well-known publicly available datasets, demonstrate that the proposed solution outperforms existing state-of-the-art solvers for Quantified Boolean Formulas (QBF) by orders of magnitude. |
| Researcher Affiliation | Academia | 1 IRIT, University of Toulouse, France 2 IRIT, CNRS, Toulouse, France 3 CREATE, National University of Singapore, Singapore |
| Pseudocode | Yes | Algorithm 1 Deciding FRP for an arbitrary classifier |
| Open Source Code | Yes | All the materials for replicating the experiments are available at https://github.com/Xuanxiang Huang/frp RF-experiments |
| Open Datasets | Yes | The evaluation comprises 27 datasets that originate from the Penn ML Benchmarks (Olson et al. 2017). |
| Dataset Splits | No | The paper does not explicitly provide details about training/validation/test dataset splits, only mentions training and testing phases implicitly. |
| Hardware Specification | Yes | the experiments were performed on a Mac Book Pro with a 6Core Intel Core i7 2.6 GHz processor with 16 GByte RAM, running mac OS Monterey. |
| Software Dependencies | Yes | a prototype of FRPCGR was implemented in Python7. The Py SAT toolkit (Ignatiev, Morgado, and Marques-Silva 2018) was used to implement the FRP encodings, and configured to run the Glucose 4 (Audemard and Simon 2018)8 SAT solver. The QBF solvers we used are Dep QBF (Lonsing and Egly 2017)9 and CAQE (Rabe and Tentrup 2015)10. Moreover, we combined CAQE with preprocessor Bloqqer (Biere, Lonsing, and Seidl 2011)11. |
| Experiment Setup | Yes | The RF models are trained with varying the maximum depth from 4 to 6 and the number of trees from 20 to 100, so that we obtain the most accurate models... The time limit for deciding one query was set to 1200 seconds, and we capped the time for finishing 200 queries by 5 hours. |