Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings
Authors: Edouard YVINEC, Arnaud Dapogny, Kevin Bailly
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In order to evaluate the proposed JLCM method, we considered both computer vision and natural language processing tasks and models. All details are available in Appendix C. First, we compare multiple methods (clustering method, multiple scaling factors or codebooks) for the initialization, as detailed in Section 3.2. Second, we evaluate each component of the proposed JLCM method through an ablation study. Third, we compare the proposed method to the current state-of-the-art DNN memory compression techniques. In Table 3, we report our results on Image Net models. |
| Researcher Affiliation | Collaboration | Sorbonne Universit e1, CNRS, ISIR, f-75005, 4 Place Jussieu 75005 Paris, France Datakalab2, 114 boulevard Malesherbes, 75017 Paris, France |
| Pseudocode | Yes | These steps are summarized in Algorithm 1 at lines 2 to 4. ... The proposed approach is summarized in Algorithm 1 (in Appendix B). |
| Open Source Code | No | The paper does not provide an explicit statement about the release of source code or a link to a code repository. |
| Open Datasets | Yes | In Table 3, we report our results on Image Net models. ... For the Image Net experiments, we use the datasets as described in Deng et al. (2009). ... For the Llama 7B experiments, we evaluate its performance on a wide range of common sense reasoning tasks, namely: Winograde Sakaguchi et al. (2021), OBQA Mihaylov et al. (2018), Hellaswag Zellers et al. (2019), PIQA Bisk et al. (2020), and BoolQ Clark et al. (2019). ... We also used Diffusion DB Wang et al. (2022) as a prompt reference. |
| Dataset Splits | Yes | For the Image Net experiments, the calibration set consists in 128 images uniformly sampled from the training set, following the common practice for GPTQ methods. |
| Hardware Specification | Yes | All the experiments were run on a server with 8 Nvidia A100 GPUs and 240 Go of RAM. |
| Software Dependencies | Yes | All methods and models are implemented in PyTorch 1.12 with CUDA 11.6. |
| Experiment Setup | Yes | All details are available in Appendix C. ... C.1 EXPERIMENTS SETUP ... For the Image Net experiments, the calibration set consists in 128 images uniformly sampled from the training set, following the common practice for GPTQ methods. |