Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Detecting Interactions from Neural Networks via Topological Analysis
Authors: Zirui Liu, Qingquan Song, Kaixiong Zhou, Ting-Hsiang Wang, Ying Shan, Xia Hu
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results validate that the PID algorithm outperforms the state-of-the-art baselines. |
| Researcher Affiliation | Collaboration | Zirui Liu Dept. of Computer Science Texas A&M University College Station, TX EMAIL Qingquan Song Dept. of Computer Science Texas A&M University College Station, TX EMAIL Kaixiong Zhou Dept. of Computer Science Texas A&M University College Station, TX EMAIL Ting-Hsiang Wang Dept. of Computer Science Texas A&M University College Station, TX EMAIL Ying Shan Tencent Beijing, China EMAIL Xia Hu Dept. of Computer Science Texas A&M University College Station, TX EMAIL |
| Pseudocode | Yes | We list the full algorithm in Appendix A. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | We utilize ten synthetic datasets that contain a mixture of pairwise interactions and higher-order interactions, as shown in the Appendix E.1. All ten datasets and MLP structures are the same as those in [6]. We compare our PID and NID on five real world binary classification datasets. The statistics of these datasets are shown in Appendix F.1 Table 6. We trained a simple CNN to classify images on the MNIST dataset [28] and Fashion MNIST dataset [29]. |
| Dataset Splits | Yes | We ran ten trials of AG, NID, and PID on each dataset and removed two trials with the highest and lowest AUC scores. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper does not provide specific version numbers for ancillary software dependencies used in the experiments. |
| Experiment Setup | Yes | We remark the norm p in Algorithm 1 is set to 2 across all experiments, which captures the Euclidean distance of points in persistence diagrams [20]. The detailed experimental settings can be found in Appendix E.1. A more detailed experiment setting can be found in Appendix F.1. The detailed experiment setting can be found in Appendix G. To answer Q2, we also compare the result based on MLPs with different architectures (Appendix E.3 Figure 8) and regularization strength (Appendix E.3 Figure 10, Figure 9). |