A Survey on Model Compression and Acceleration for Pretrained Language Models
Authors: Canwen Xu, Julian McAuley
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this survey, we focus on the inference stage and review the current state of model compression and acceleration for pretrained language models, including benchmarks, metrics and methodology. |
| Researcher Affiliation | Academia | University of California, San Diego {cxu, jmcauley}@ucsd.edu |
| Pseudocode | No | The paper is a survey and does not contain any structured pseudocode or algorithm blocks for its own work. |
| Open Source Code | No | The paper is a survey and does not provide source code for any methodology developed within this paper. It mentions external tools like Code Carbon and PyTorch Sparse API, but these are not code for the paper's own methods. |
| Open Datasets | No | The paper is a survey and does not use datasets for training its own models. It refers to established benchmarks like GLUE, Super GLUE, and SQuAD as contexts for other research, not as datasets used in its own work. |
| Dataset Splits | No | The paper is a survey and does not conduct its own experiments or define dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is a survey and does not conduct its own experiments, so it does not specify hardware used for running experiments. It mentions hardware in the context of other research (e.g., 'sparse tensor cores in Nvidia A100'), but not for its own work. |
| Software Dependencies | No | The paper is a survey and does not specify any software dependencies with version numbers for its own research work or experiments. |
| Experiment Setup | No | The paper is a survey and does not conduct its own experiments; thus, it does not provide details about an experimental setup, hyperparameters, or training configurations. |