Lossy Compression for Lossless Prediction
Authors: Yann Dubois, Benjamin Bloem-Reddy, Karen Ullrich, Chris J. Maddison
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using these objectives, we train a generic image compressor that achieves substantial rate savings (more than 1000 on Image Net) compared to JPEG on 8 datasets, without decreasing downstream classification performance. We evaluated our framework focusing on two questions: (i) What compression rates can our framework achieve at what cost? (ii) Can we train a general purpose predictive image compressor? |
| Researcher Affiliation | Collaboration | Yann Dubois Vector Institute yanndubois96@gmail.com Benjamin Bloem-Reddy The University of British Columbia benbr@stat.ubc.ca Karen Ullrich Facebook AI Research karenu@fb.com Chris J. Maddison University of Toronto Vector Institute cmaddis@cs.toronto.edu |
| Pseudocode | Yes | Algorithm 1 BINCE s forward pass for x |
| Open Source Code | Yes | Code is at github.com/Yann Dubs/lossyless. The code to train our main compressor is in Appx. E.7, the code to replicate all our results is at anonymous. |
| Open Datasets | Yes | Using these objectives, we train a generic image compressor that achieves substantial rate savings (more than 1000 on Image Net) compared to JPEG on 8 datasets... We compressed samples from a 2D banana source distribution [36], assuming rotation invariant tasks... We compressed the STL10 dataset [37]... on the small MSCOCO dataset [43]. |
| Dataset Splits | Yes | We optimized hyper-parameters on validation using random search. |
| Hardware Specification | No | The paper mentions training on 'a single GPU' but does not specify the model, manufacturer, or other detailed specifications of the hardware used for experiments. |
| Software Dependencies | No | The paper mentions using specific models like 'Res Net18' and 'Ballé et al. s [28] hyperprior' but does not list programming languages (e.g., Python) or library versions (e.g., PyTorch 1.x, TensorFlow 2.x) with version numbers. |
| Experiment Setup | Yes | For all experiments, we train the compressors, freeze them, train the downstream predictors, and finally evaluate both on a test set. For classical compressors, standard neural compressors (VC) and our VIC, we used either reconstructions X as inputs to the predictors or representations Z. As BINCE does not provide reconstructions, we predicted from the compressed Z using a multi-layer perceptron (MLP). We used Res Net18 [35] for encoders and image predictors. For entropy models we used Ballé et al. s [28] hyperprior, which uses uniform quantization. We optimized hyper-parameters on validation using random search. |