Simple Augmentation Goes a Long Way: ADRL for DNN Quantization
Authors: Lin Ning, Guoyang Chen, Weifeng Zhang, Xipeng Shen
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate the efficiency of augmented DRL on mixed precision quantization of DNNs, we experiment with four different networks: Cifar Net (Krizhevsky, 2012), Res Net20 (He et al., 2016), Alex Net (Krizhevsky et al., 2012) and Res Net50 (He et al., 2016). The first two networks work on Cifar10 (Krizhevsky, 2012) dataset while the last two work on the Image Net (Deng et al., 2009) dataset. |
| Researcher Affiliation | Collaboration | Lin Ning North Carolina State University Raleigh, NC, USA lning@ncsu.edu; Guoyang Chen Alibaba Group Sunnyvale, CA, USA g.chen@alibaba-inc.com; Weifeng Zhang Alibaba Group Sunnyvale, CA, USA weifeng.z@alibaba-inc.com; Xipeng Shen North Carolina State University Raleigh, NC, USA xshen5@ncsu.edu |
| Pseudocode | Yes | Algorithm 1 Augmented Policy; 1: State s previously received from the environment E; 2: Ak = µ(s|θµ) (generating k candidate actors); 3: a = g(Ak)(a) (refines the choice of the action with g(Ak) = arg maxa=Ak,i e QE(a)); 4: Apply a to environment; receive r, s |
| Open Source Code | No | The paper mentions using code from prior work ('HAQ (code downloaded from the original work (Wang et al., 2018))') but does not state that its own code for the described methodology is publicly available or provided. |
| Open Datasets | Yes | The experiments are done on a server with an Intel(R) Xeon(R) Platinum 8168 Processor, 32GB memory and 4 NVIDIA Tesla V100 32GB GPUs. To evaluate the efficiency of augmented DRL on mixed precision quantization of DNNs, we experiment with four different networks: Cifar Net (Krizhevsky, 2012), Res Net20 (He et al., 2016), Alex Net (Krizhevsky et al., 2012) and Res Net50 (He et al., 2016). The first two networks work on Cifar10 (Krizhevsky, 2012) dataset while the last two work on the Image Net (Deng et al., 2009) dataset. |
| Dataset Splits | No | The paper mentions using a 'test dataset' for evaluation and inference accuracy, and models are 'finetuned' but it does not provide explicit details about training, validation, or test dataset splits, such as percentages or sample counts for each split. It refers to 'test dataset' but not 'validation'. |
| Hardware Specification | Yes | The experiments are done on a server with an Intel(R) Xeon(R) Platinum 8168 Processor, 32GB memory and 4 NVIDIA Tesla V100 32GB GPUs. |
| Software Dependencies | No | The paper does not explicitly provide a list of software dependencies with specific version numbers (e.g., programming language version, library versions) used in the experiments. |
| Experiment Setup | Yes | The default setting of HAQ is searching for 600 episodes and finetuning for 100 epochs. For ADRL, k (number of candidate actions) is 3. The learning rate is set as 0.001, which is the default setting from HAQ. The termination criterion is that the coefficient of variance of the averaged inference accuracy over 10 episodes is smaller than 0.01 for at least two consecutive 10-episode windows. |