On Second-Order Group Influence Functions for Black-Box Predictions
Authors: Samyadeep Basu, Xuchen You, Soheil Feizi
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 7. Experiments Our goal through the experiments is to observe if the second-order approximations of group influence functions improve the correlation with the ground truth estimate across different settings. We compare the computed second-order group influence score with the ground truth influence (which is computed by leave-k-out retraining for a group with size k). Our metric for evaluation is the Pearson correlation which measures how linearly the computed influence and the actual ground truth estimate are related. We perform our experiments primarily on logistic regression where the group influence function is well-defined. Additionally we also check the accuracy of first-order and second-order group influence functions in case of neural networks. |
| Researcher Affiliation | Academia | Samyadeep Basu 1 Xuchen You 1 Soheil Feizi 1 1Department of Computer Science, University of Maryland College Park. Correspondence to: Samyadeep Basu <sbasu12@cs.umd.edu>. |
| Pseudocode | No | The paper provides mathematical derivations and explains concepts like conjugate gradients, but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about making the source code for its methodology publicly available, nor does it provide a link to a code repository. |
| Open Datasets | Yes | The second set of experiments are done with the standard handwritten digits database MNIST (Le Cun, Bottou, Bengio, and Haffner, 1998) which consists of 10 classes of different digits. |
| Dataset Splits | No | The paper mentions using 'training points' and evaluating on a 'test-point' and refers to 'leave-k-out retraining', but it does not specify explicit train/validation/test dataset splits (e.g., percentages or counts) or a detailed methodology for partitioning the data for reproduction. |
| Hardware Specification | No | The paper does not specify any particular hardware components such as GPU models, CPU types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper mentions techniques like 'conjugate gradients' but does not specify any software names with version numbers (e.g., specific Python libraries like TensorFlow or PyTorch and their versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | For MNIST, we used a regularized Hessian with a value of λ = 0.01 and conducted experiments for a relatively simple two hidden layered feed-forward network with sigmoid activations... In our experiments we relaxed the L0 norm to L1 norm and solved the projected gradient descent step of the optimization in Equation (18) using (Duchi et al., 2008). |