Augmenting Automated Spectrum Based Fault Localization for Multiple Faults
Authors: Prantik Chatterjee, Jose Campos, Rui Abreu, Subhajit Roy
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Empirical Analysis We implement the proposed multiverse analysis approach in a tool called ARTEMIS. To evaluate the effectiveness of our proposed approach, we experimentally evaluate the following research questions: RQ1: What are the optimal choices of hyperparameters for ARTEMIS? RQ2: How does ARTEMIS compare against the most popular SBFL metrics? |
| Researcher Affiliation | Academia | Prantik Chatterjee1 , Jos e Campos2,3 , Rui Abreu2,4 and Subhajit Roy1 1Indian Institute of Technology Kanpur, India 2Faculty of Engineering of University of Porto, Porto, Portugal 3LASIGE, Faculdade de Ciencias, Universidade de Lisboa, Lisboa, Portugal 4INESC-ID, Porto, Portugal |
| Pseudocode | Yes | Algorithm 1: EXPLORER(A, E, n, S, µ, β, p) ... Algorithm 2: SIMULATE(A, E, C, S, p) ... Algorithm 3: MERGE(L , n) |
| Open Source Code | Yes | We provide an appendix with additional experimental results and the source code of ARTEMIS in https://github.com/prantikchatterjee/Artemis IJCAI23.git |
| Open Datasets | Yes | We have performed our experiments on six java project repositories, Chart, Closure, Lang, Math, Mockito and Time from the DEFECTS4J benchmark suite [Just et al., 2014]. In each fold, we set aside the spectrums from one project as test data and search for the best hyperparameters over the spectrums from the remaining five projects. |
| Dataset Splits | No | The paper describes using cross-validation for hyperparameter selection on the training data, but does not explicitly define a separate 'validation' dataset split with that specific terminology. |
| Hardware Specification | No | The paper does not provide specific hardware details such as CPU/GPU models, memory, or detailed computer specifications used for running its experiments. |
| Software Dependencies | No | The paper mentions using EVOSUITE, ULYSIS, and DDU but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Our multiverse analysis (Algorithm 1) uses the following hyperparameters: µ, β, the underlying SBFL metric S and p. To select the best hyperparameter combination and to show that the configuration generalizes, we use k-fold cross validation. We employ a grid search to select the best value of hyperparameters. We search for µ in a range from 1 to 20. For β, we restrict the search in {5, 10, 15, 20, . . . , 100}. |