Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Value-Guided KV Compression for LLMs via Approximated CUR Decomposition
Authors: Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Cur DKV on two popular long-context benchmarks Long Bench [Bai et al., 2023] and Ruler [Hsieh et al., 2024], spanning 24 tasks with LLa MA-3.1-8B-Instruct [Grattafiori et al., 2024] and Mistral-7B-Instruct [Jiang et al., 2023]. Our experiments demonstrate that Cur DKV consistently outperforms existing attention-based KV compression baselines such as Snap KV [Li et al., 2024b], Chunk KV [Liu et al., 2025], and Streaming LLM [Xiao et al., 2024]. |
| Researcher Affiliation | Academia | Department of Electrical Engineering Indian Institute of Technology Delhi, India EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 provides the pseudocode of Cur DKV. |
| Open Source Code | Yes | 1We have uploaded the source code and datasets as supplementary; we are committed to release them upon acceptance of the paper. |
| Open Datasets | Yes | We evaluate Cur DKV on two popular long-context benchmarks Long Bench [Bai et al., 2023] and Ruler [Hsieh et al., 2024]... We have uploaded the source code and datasets as supplementary |
| Dataset Splits | No | The paper mentions evaluating on Long Bench and Ruler benchmarks and using a "question-agnostic setting" but does not explicitly detail the training/test/validation splits for these datasets. While standard benchmarks often have predefined splits, the paper does not specify which ones were used or if custom splits were created. |
| Hardware Specification | Yes | H2O [Zhang et al., 2023] is purposefully omitted from the evaluation as it throws out-of-memory error when run on a single NVIDIA A100-80GB GPU card |
| Software Dependencies | No | The paper mentions several software components like Flash Attention and Grouped Query Attention, but does not provide specific version numbers for any of the software dependencies used in their implementation. |
| Experiment Setup | Yes | For all these 24 tasks, we evaluate only on the more challenging question-agnostic setting [NVIDIA, 2024], where the questions are omitted during compression and only the context is compressed. As argued by Feng et al. [2025], this setup mimics more challenging scenarios, where the compression method is unaware of the questions being passed to the model during inference. Following Xiao et al. [2024], we use attention sinks of size s = 4 for all the baselines (all the baseline numbers are produced in-house). In our implementation, we use r = 20. a default value of α = 0.20 is used in our implementation. |