Boosting Residual Networks with Group Knowledge
Authors: Shengji Tang, Peng Ye, Baopu Li, Weihao Lin, Tao Chen, Tong He, Chong Yu, Wanli Ouyang
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Compared with typical subnet training and other methods, our method achieves the best efficiency and performance trade-offs on multiple datasets and network structures. The efficacy and efficiency of GKT is shown in Figure 2. Comprehensive empirical comparisons and analysis show that GKT can reduce the capacity gap and efficiently improve the performance of various residual networks, including CNNs and transformers. |
| Researcher Affiliation | Collaboration | 1 School of Information Science and Technology, Fudan University, Shanghai, China 2 Academy for Engineering and Technology, Fudan University, Shanghai, China 3Shanghai AI Laboratory, Shanghai, China 4Independent Researcher |
| Pseudocode | Yes | The pseudo-code of GKT is shown in Appendix B1. |
| Open Source Code | Yes | The code is at https://github.com/tsj001/AAAI24-GKT. |
| Open Datasets | Yes | We demonstrate the effectiveness and efficiency of GKT on typical residual convolutional networks including Res Net-34, Res Net-50 (He et al. 2016), WRN16-8, WRN28-10 (Zagoruyko and Komodakis 2016) and Mobile Net V3 (Howard et al. 2019), and mainstream datasets including CIFAR-100, Tiny Image Net and Image Net-1K. ... on COCO2017 (Lin et al. 2014). |
| Dataset Splits | No | The paper mentions datasets like CIFAR-100, Tiny Image Net, and Image Net-1K which typically have standard training, validation, and test splits. However, it does not explicitly state the percentages or sample counts for these splits in the main text, nor does it specify how the data was split (e.g., specific random seeds or cross-validation details). |
| Hardware Specification | No | The paper mentions "Training time (GPU hours)" in Figure 2 but does not specify the particular GPU models, CPU, or other hardware components used for running the experiments. |
| Software Dependencies | No | The paper does not list any specific software dependencies (e.g., programming languages, libraries, frameworks) with their version numbers. |
| Experiment Setup | Yes | The details of experiment settings are shown in Appendix A. |