Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Abduction-Based Explanations for Machine Learning Models
Authors: Alexey Ignatiev, Nina Narodytska, Joao Marques-Silva1511-1519
AAAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results, obtained on well-known datasets, validate the scalability of the proposed approach as well as the quality of the computed solutions. |
| Researcher Affiliation | Collaboration | Alexey Ignatiev,1,3 Nina Narodytska,2 Joao Marques-Silva1 1Faculty of Science, University of Lisbon, Portugal 2VMware Research, CA, USA 3ISDCT SB RAS, Irkutsk, Russia EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1: Computing a subset-minimal explanation; Algorithm 2: Computing a smallest size explanation |
| Open Source Code | No | The paper mentions that its prototype implementation is "written in Python" and uses other open-source tools (e.g., Py SMT, RC2), but it does not provide an explicit statement or a link to its own source code for the methodology described in the paper. |
| Open Datasets | Yes | The benchmarks considered include the well-known text-based datasets from the UCI Machine Learning Repository1 and Penn Machine Learning Benchmarks2, as well as the widely used MNIST digits database3. ... 1https://archive.ics.uci.edu/ml/ 2https://github.com/EpistasisLab/penn-ml-benchmarks/ 3http://yann.lecun.com/exdb/mnist/ |
| Dataset Splits | No | The paper states that neural networks were trained on datasets and evaluated on samples, but it does not explicitly specify the training/validation/test split percentages, absolute sample counts for each split, or detailed splitting methodology needed for reproduction. |
| Hardware Specification | Yes | all benchmarks were ran on a Macbook Pro having an Intel Core i7 2.8GHz processor with 8GByte of memory on board. |
| Software Dependencies | Yes | CPLEX 12.8.0 (IBM ILOG 2018) is used as a MILP oracle accessed via its of๏ฌcial Python API. |
| Experiment Setup | Yes | Time limit was set to 1800 seconds while memory limit was set to 4GByte. ... Each neural network considered has one hidden layer with i {10, 15, 20} neurons. |