SEAT: Stable and Explainable Attention
Authors: Lijie Hu, Yixin Liu, Ninghao Liu, Mengdi Huai, Lichao Sun, Di Wang
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, through intensive experiments on various datasets, we compare our SEAT with other baseline methods using RNN, Bi LSTM and BERT architectures, with different evaluation metrics on model interpretation, stability and accuracy. Results show that, besides preserving the original explainability and model performance, SEAT is more stable against input perturbations and training randomness, which indicates it is a more faithful explanation. |
| Researcher Affiliation | Academia | 1 King Abdullah University of Science and Technology, Thuwal, Saudi Arabia 2 Lehigh University, Bethlehem, Pennsylvania, USA 3 University of Georgia, Athens, Georgia, USA 4 Iowa State University, Ames, Iowa, USA |
| Pseudocode | Yes | Algorithm 1 Finding a SEAT |
| Open Source Code | No | The paper does not provide an explicit statement or link to the open-source code for the methodology described. |
| Open Datasets | Yes | In all experiments, we use four datasets: Stanford Sentiment Treebank (SST) (Socher et al. 2013), Emotion Recognition (Emotion) (Mohammad et al. 2018), Hate (Basile et al. 2019) and Rotten Tomatoes (Rotten T) (Pang and Lee 2005). |
| Dataset Splits | No | The paper mentions using datasets for training but does not provide specific percentages or counts for train/validation/test splits, or details about cross-validation setup in the main text. |
| Hardware Specification | No | The paper does not specify the hardware used to run the experiments (e.g., GPU models, CPU models, or cloud computing instance types). |
| Software Dependencies | No | The paper mentions using certain models (RNN, Bi LSTM, BERT) and a specific loss function (Binary Cross Entropy) but does not provide specific version numbers for software libraries or dependencies (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | For the encoder, we consider three kinds of networks as feature extractors: RNN, Bi LSTM, and BERT. For the decoder, we apply one simple MLP followed by a tanh-attention layer (Bahdanau, Cho, and Bengio 2015) and a softmax layer (Vaswani et al. 2017). [...] We select the Binary Cross Entropy loss as D1 and D2 in (6). [...] The perturbation radius is set as δx=1e-3. |