Benchmarking Deletion Metrics with the Principled Explanations
Authors: Yipei Wang, Xiaoqian Wang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct experiments to 1) Validate TRACE s optimality and the capability of serving as the principled explanation; and 2) Use TRACE to assess the impact of different settings (as discussed in Section 3) to address the OOD concern in deletion metric. |
| Researcher Affiliation | Academia | 1Elmore Family School of Electrical and Computer Engineering, Purdue University, IN, USA. Correspondence to: Xiaoqian Wang <joywang@purdue.edu>. |
| Pseudocode | Yes | The associated pseudo-code is provided in Appendix D. |
| Open Source Code | Yes | The implementation of TRACE is available as open source at Git Hub. |
| Open Datasets | Yes | Experiments utilize the Image Net1k (ILSVRC2012) dataset (Deng et al., 2009) with images resized to 224 224. |
| Dataset Splits | Yes | The results are demonstrated in Figure 2(b). We present this comparison using TRACE-SA-Le Mo. And the resolution of both TRACE and the deletion metric is set to t = 7 7 = 49. |
| Hardware Specification | Yes | Experiments are carried out on Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz with Quadro RTX 6000 GPUs. |
| Software Dependencies | No | The paper mentions "torchvision" but does not specify a version number or provide version numbers for any other software libraries or tools. |
| Experiment Setup | Yes | For SA, we use K = 5000 iterations, initial temperatures of T0 = 2 for -y and T0 = 0.1 for -p, and a cooling rate of η = 0.999. |