Resilient Binary Neural Network
Authors: Sheng Xu, Yanjing Li, Teli Ma, Mingbao Lin, Hao Dong, Baochang Zhang, Peng Gao, Jinhu Lu
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments are conducted upon various network models, such as Res Net and Faster-RCNN for computer vision, as well as BERT for natural language processing. The results demonstrate the overwhelming performance of our Re BNN over prior arts. |
| Researcher Affiliation | Collaboration | 1 Beihang University, Beijing, P.R.China 2 Zhongguancun Laboratory, Beijing, P.R.China 3 Shanghai AI Laboratory, Shanghai, P.R.China 4 Tencent, P.R.China 5 Peking University, Beijing, P.R.China |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is open-sourced at https://github.com/Steve Tsui/Re BNN. |
| Open Datasets | Yes | For its huge scope and diversity, the Image Net object classification dataset (Russakovsky et al. 2015) is more demanding, which has 1000 classes, 1.2 million training photos, and 50k validation images. The COCO dataset includes images from 80 different categories. All our experiments on COCO dataset are conducted on the COCO 2014 (Lin et al. 2014) object detection track in the training stage... |
| Dataset Splits | Yes | For its huge scope and diversity, the Image Net object classification dataset (Russakovsky et al. 2015) is more demanding, which has 1000 classes, 1.2 million training photos, and 50k validation images. The COCO dataset includes images from 80 different categories. All our experiments on COCO dataset are conducted on the COCO 2014 (Lin et al. 2014) object detection track in the training stage, which contains the combination of 80k images from the COCO train2014 and 35k images sampled from COCO val2014, i.e., COCO trainval35k. |
| Hardware Specification | Yes | We run the experiments on 4 NVIDIA Tesla A100 GPUs with 80 GB memory. We implement the 1-bit models achieved by our Re BNN on ODROID C4, which has a 2.016 GHz 64-bit quad-core ARM Cortex-A55. |
| Software Dependencies | No | Py Torch (Paszke et al. 2017) is used to implement Re BNN. We leverage the SIMD instruction SSHL on ARM NEON to make the inference framework BOLT (Feng 2021) compatible with Re BNN. |
| Experiment Setup | Yes | First, we use one-stage training on Res Nets, with SGD as the optimization algorithm and a momentum of 0.9, and a weight decay of 1e 4 following (Xu et al. 2021c). η is set to 0.1. The learning rates are optimized by the annealing cosine learning rate schedule. The number of epochs is set as 200. Then, we employ two-stage training following (Liu et al. 2020). Each stage counts 256 epochs. In this implementation, Adam is selected as the optimizer. And the network is supervised by a real-valued Res Net-34 teacher. The weight decay is set as 0 following (Liu et al. 2020). The learning rates η is set as 1e 3 and annealed to 0 by linear descent. |