Diagnosing Software Faults Using Multiverse Analysis
Authors: Prantik Chatterjee, Abhijit Chatterjee, Jose Campos, Rui Abreu, Subhajit Roy
IJCAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that the Multiverse Analysis not just improves the efficiency of fault localization but also achieves better coverage and generates smaller test-suites over DDU, the current state-of-the-art technique. On average, our approach reduces the developer effort over DDU by over 16% for more than 92% of the instances. Further, the improvements over DDU are indeed statistically significant on the paired Wilcoxon Signed-rank test. |
| Researcher Affiliation | Academia | Prantik Chatterjee1 , Abhijit Chatterjee1 , Jos e Campos2 , Rui Abreu3 and Subhajit Roy1 1Indian Institute of Technology, Kanpur, India 2LASIGE, Faculdade de Ciˆencias, University of Lisbon, Portugal 3INESC-ID and IST, University of Lisbon, Portugal |
| Pseudocode | Yes | Algorithm 1: Ulysis |
| Open Source Code | Yes | Ulysis is available in EVOSUITE as part of pull request #293, https://github.com/Evo Suite/evosuite/pull/293. |
| Open Datasets | Yes | We have performed our experiments on DEFECTS4J version 1.4.0 [Just et al., 2014] which is a benchmark suite consisting of six diverse Java project repositories. |
| Dataset Splits | No | The paper describes evaluating performance on '111 valid instances' from the DEFECTS4J benchmark and generating multiple test-suites, but it does not specify explicit training, validation, or test dataset splits in terms of percentages or counts for data partitioning. |
| Hardware Specification | Yes | We have done these experiments on a 16 core virtual machine with Intel Xeon processors having 2.1 GHz core frequency and 32 gigabytes of RAM. |
| Software Dependencies | No | The paper mentions using 'EVOSUITE' and 'GZOLTAR tool' but does not provide specific version numbers for these software dependencies in their experimental setup. |
| Experiment Setup | Yes | To take into account the randomization within EVOSUITE, for each fault, we have generated 5 test-suites using a time limit of 600 seconds on each fitness function. ... All experiments are performed at branch granularity, i.e., the program components are branches. |