SFC: Achieve Accurate Fast Convolution under Low-precision Arithmetic
Authors: Liulu He, Yufei Zhao, Rui Gao, Yuan Du, Li Du
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted experiments on image classification tasks to demonstrate the effectiveness of our algorithms. Experiments were conducted on the Image Net dataset, which contains 1.4 million images of size 224 224 3, distributed across 1,000 classes. We conducted quantization on the following algorithms: 1) Direct convolution, 2) The well-known Winograd F(4 4, 3 3) algorithms, which have been extensively researched for their quantization methods in recent years, and 3) our proposed SFC algorithms, including 1D and 2D format. |
| Researcher Affiliation | Academia | 1School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China 2Interdisciplinary Research Center for Future Intelligent Chips (Chip-X), Nanjing University, Suzhou 215163, China. |
| Pseudocode | No | The paper describes mathematical formulations and transformations but does not provide structured pseudocode or an explicitly labeled algorithm block. |
| Open Source Code | No | The paper does not provide a direct link or explicit statement about the public availability of its source code for the described methodology. |
| Open Datasets | Yes | Experiments were conducted on the Image Net dataset, which contains 1.4 million images of size 224 224 3, distributed across 1,000 classes. |
| Dataset Splits | Yes | We randomly selected 500 images from training set to create the calibration set for PTQ fine-tuning. Model accuracy was evaluated on the validation set. |
| Hardware Specification | Yes | We develop RTL code for the convolution accelerator based on the SFC-6(7 7,3 3) algorithm. The resource consumption and timing report are synthesized using Xilinx Vivado tools. The DSP48 hardcore is a crucial resource on FPGA... Platform zcu102 xc7vx980t alveo U50 xczu19eg |
| Software Dependencies | No | The paper mentions "Xilinx Vivado tools" and "Torch Vision" but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | All batch normalization layers were fused into the preceding convolution layers prior to quantization. All the data in the spatial domain are quantized to int8, and the data in the transformation domain are quantized from int8 to int4. Direct convolution and our SFC algorithms were quantized using Ada Quant (Hubara et al., 2020), while the Winograd algorithm was processed with Scaling Gradient Backward (Jain et al., 2020), due to observed convergence differences with Ada Quant in the Winograd F(4 4, 3 3). |