Explaining Preferences with Shapley Values
Authors: Robert Hu, Siu Lun Chau, Jaime Ferrando Huertas, Dino Sejdinovic
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To demonstrate the utility of PREF-SHAP, we apply our method to a variety of synthetic and real-world datasets and show that richer and more insightful explanations can be obtained over the baseline. We apply PREF-SHAP to unrankable synthetic and real-world datasets to connect theory with practice. We split data, i.e. matches with their outcomes, into train (80%), validation (10%), and test (10%) and explain the model on a random subset of the data. |
| Researcher Affiliation | Collaboration | Robert Hu Amazon London Siu Lun Chau Department of Statistics University of Oxford Jaime Ferrando Huertas Shaped New York Dino Sejdinovic School of Computer and Mathematical Sciences University of Adelaide |
| Pseudocode | Yes | We summarize the procedure of PREF-SHAP in Algorithm 1. Algorithm 1 PREF-SHAP |
| Open Source Code | Yes | We release a high-performant implementation of PREF-SHAP at [22]. [22] Code for Pref-SHAP. https://github.com/Mr Huff/PREF-SHAP. |
| Open Datasets | Yes | For our real-world datasets, we consider publicly available datasets Chameleon, Pokémon and Tennis. The Chameleon dataset [44] considers 106 contests between 35 male dwarf chameleons. [44] Devi Stuart-Fox, David Firth, Adnan Moussalli, and Martin Whiting. Multiple signals in chameleon contests: Designing and analysing animal contests as a tournament. Animal Behaviour, 71:1263 1271, 06 2006. doi: 10.1016/j.anbehav.2005.07.028. The Tennis dataset considers professional tennis matches between 1991 and 2017 in all major tournaments each year. The data is provided publicly by ATP World Tour [45]. [45] Tennis dataset. https://datahub.io/sports-data/atp-world-tour-tennis-data, 2022. |
| Dataset Splits | Yes | We split data, i.e. matches with their outcomes, into train (80%), validation (10%), and test (10%) and explain the model on a random subset of the data. |
| Hardware Specification | Yes | We perform all our experiments on a single Nvidia A100 GPU. |
| Software Dependencies | No | The paper mentions software like FALKON but does not provide specific version numbers for any software dependencies required to reproduce the experiments. |
| Experiment Setup | No | The paper states that 'hyperparameters for the kernels are selected using gradient descent' but does not provide specific values for these hyperparameters (e.g., learning rate, batch size, epochs, specific kernel parameters) or other detailed training configurations used in the experiments. |