Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions
Authors: Sanjay Kariyappa, Freddy Lecue, Saumitra Mishra, Christopher Pond, Daniele Magazzeni, Manuela Veloso
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our studies on a diverse set of models trained on text classification tasks show that SP-PI and MP-PI provide significantly better attributions compared to prior work. |
| Researcher Affiliation | Industry | JPMorgan Chase AI Research. |
| Pseudocode | Yes | Algorithm 1 Multi-pass progressive Inference |
| Open Source Code | Yes | Code is provided in the supplementary material. |
| Open Datasets | Yes | For IMDB (Maas et al., 2011) and SST-2 (Socher et al., 2013) datasets, we use models that are available on the Hugging Face repository. For AG-News (Zhang et al., 2015), Twitter-Finance, Twitter-Sentiment (Rosenthal et al., 2017), Twitter-Emotion (Mohammad et al., 2018) and True Teacher (Gekhman et al., 2023) datasets, we fine-tune GPT-2 (Radford et al., 2018) or Llama-2 (Touvron et al., 2023) models |
| Dataset Splits | No | The paper mentions using a "test set" for evaluation but does not provide specific training, validation, or test dataset split percentages or sample counts. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running experiments. |
| Software Dependencies | No | The paper mentions using "Adam optimizer" and "Lo RA" parameters but does not provide specific version numbers for software dependencies like programming languages, deep learning frameworks (e.g., PyTorch, TensorFlow), or other libraries. |
| Experiment Setup | Yes | We train all models for 10 epochs with a learning rate of 5 × 10−5. We use the Adam optimizer and a batch size of 16. ... For Lo RA, we use a rank=16, alpha=32 and lora dropout=0.1 |