Optimal and Approximate Adaptive Stochastic Quantization
Authors: Ran Ben-Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher, Shay Vargaftik
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments indicate that our algorithms may open the door to using ASQ more extensively in a variety of ML applications. We implement our algorithms in C++ and demonstrate their efficiency. We evaluate our solutions compared to state-of-the-art ASQ methods on a variety of distributions considering different vector sizes and number of quantization values and demonstrate a speedup of up to four orders of magnitude. |
| Researcher Affiliation | Collaboration | Ran Ben Basat UCL Yaniv Ben-Itzhak VMware Research Michael Mitzenmacher Harvard University Shay Vargaftik VMware Research |
| Pseudocode | Yes | Algorithm 1 QUIVER; Algorithm 2 Accelerated QUIVER; Algorithm 3 Basic Dynamic Programming Algorithm; Algorithm 4 Apx. QUIVER |
| Open Source Code | Yes | We open source the code of the paper [30]. All our results are reproducible and our code is open sourced [30]. |
| Open Datasets | Yes | We present results for the Log Normal distribution and defer to Appendix H results for Normal, Exponential, Trunc Norm, and Weibull distributions. ... All our results are reproducible and our code is open sourced [30]. We do not use any proprietary data and the synthetic data can be reproduced by anyone. |
| Dataset Splits | No | The paper describes using input vectors generated from various distributions and evaluating performance across different dimensions and quantization values, but does not specify typical training, validation, or test dataset splits. |
| Hardware Specification | Yes | Unless stated otherwise, we use a g4dn.4xlarge AWS EC2 server with custom Intel Cascade Lake CPUs with 64 GB RAM and Ubuntu 22.04 OS. |
| Software Dependencies | Yes | We measure the sort and quantize operations using the same EC2 server that is also equipped with an NVIDIA T4 GPU, Py Torch v2.1.2, and CUDA tool kit v12.3. |
| Experiment Setup | No | The paper mentions 'different vector sizes and number of quantization values' and that ALQ uses '10 iterations', but it does not provide specific hyperparameters like learning rate, batch size, or optimizer settings for training models. |