Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution
Authors: Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, Sameer Singh
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compare SARFA with existing approaches on agents trained to play board games (Chess and Go) and Atari games (Breakout, Pong and Space Invaders). We show through illustrative examples (Chess, Atari, Go), human studies (Chess), and automated evaluation methods (Chess) that SARFA generates saliency maps that are more interpretable for humans than existing approaches. For instance, in our human studies, SARFA improves accuracy of solving chess puzzles by nearly 25% and reduces the time taken by 31% over existing approaches. |
| Researcher Affiliation | Collaboration | Media and Data Science Research, Adobe Systems Inc., Noida, Uttar Pradesh, India 201301 Indian Institute of Technology Kharagpur, West Bengal, India 721302 Indian Institute of Technology Madras, Chennai, India 600036 Department of Computer Science, University of California, Irvine, California, USA |
| Pseudocode | No | The paper provides mathematical equations to describe the SARFA method but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | For the code release and demo videos, see https://nikaashpuri.github.io/sarfa-saliency/. All of our code and more detailed results are available in our Github repository: https://nikaashpuri.github.io/sarfa-saliency/. |
| Open Datasets | Yes | For experiments on chess, we use the Stockfish agent (https://stockfishchess.org/). For experiments on Go, we use the pre-trained Mini Go RL agent (https://github.com/tensorflow/minigo). ... To automatically compare the saliency maps generated by different perturbation-based approaches, we introduce a Chess saliency dataset. ... The complete dataset is available in our Github repository: https://nikaashpuri.github.io/sarfa-saliency/. |
| Dataset Splits | No | The paper mentions using pre-trained agents and creating a 'Chess saliency dataset' consisting of 100 chess puzzles, but it does not specify any explicit training, validation, or test dataset splits (e.g., percentages, sample counts, or citations to standard splits) needed for reproduction. |
| Hardware Specification | No | The paper mentions using pre-trained agents (Stockfish, Mini Go, Atari agents) and their associated algorithms, but it does not provide any specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) for the machines used to conduct the experiments described in the paper. |
| Software Dependencies | Yes | For experiments on chess, we use the Stockfish 10 agent: https://stockfishchess.org/. |
| Experiment Setup | Yes | For chess and Go, we perturb the board position by removing one piece at a time. We do not remove a piece if the resulting position is illegal. For instance, in chess, we do not remove the king. For Atari, we use the perturbation technique described in Greydanus et al. (2018). The technique perturbs the input image by adding a Gaussian blur localized around a pixel. The blur is constructed using the Hadamard product to interpolate between the original input image and a Gaussian blur. |