FracBits: Mixed Precision Quantization via Fractional Bit-Widths
Authors: Linjie Yang, Qing Jin10612-10620
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct quantitative experiments using Frac Bits and compare it with previous quantization approaches... |
| Researcher Affiliation | Collaboration | Linjie Yang,1 Qing Jin2 1Byte Dance Inc. 2Northeastern University |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. |
| Open Datasets | Yes | We conducted experiments on Mobile Net V1/V2 and Res Net18... on Image Net dataset. |
| Dataset Splits | No | The paper refers to using Image Net dataset but does not explicitly provide specific train/validation/test dataset splits (percentages or counts) or detailed splitting methodology. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory used for running experiments. |
| Software Dependencies | No | The paper mentions using specific quantization schemes (SAT, PACT, Do Re Fa) but does not provide specific version numbers for software dependencies like programming languages, libraries, or frameworks. |
| Experiment Setup | Yes | We set κ to 0.1 for all computation cost constrained experiments, and 1 for all model size constrained experiments. ... We set the initial value of κ to bt +0.5 in each layer for all experiments... For all experiments, we use cosine learing rate scheduler without restart. Learning rate is initially set to 0.05 and updated every iteration for totally 150 epochs. We use SGD optimizer with a momentum weight of 0.9 without damping, and weight decay of 4 10 5. The batch size is set to 2048 for all models. |