Adaptive Depth Networks with Skippable Sub-Paths
Authors: Woochul Kang, HYUNGSEOP LEE
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 4, we empirically demonstrate that our adaptive depth networks outperform counterpart individual networks, both in CNNs and vision transformers, and achieve actual inference acceleration and energy-saving. |
| Researcher Affiliation | Academia | Woochul Kang Department of Embedded Systems Incheon National University Yeonsu-gu, Incheon, South Korea, 22012 wchkang@inu.ac.kr Hyungseop Lee Department of Embedded Systems Incheon National University Yeonsu-gu, Incheon, South Korea, 22012 hhss0927@inu.ac.kr |
| Pseudocode | Yes | Algorithm 1 Training algorithm for an adaptive depth network M. |
| Open Source Code | Yes | Source codes are available at https://github.com/wchkang/depth |
| Open Datasets | Yes | We evaluate our method on ILSVRC2012 dataset [49] that has 1000 classes. The dataset consists of 1.28M training and 50K validation images. |
| Dataset Splits | Yes | The dataset consists of 1.28M training and 50K validation images. |
| Hardware Specification | Yes | Table 2: Training time (1 epoch), measured on Nvidia RTX 4090 (batch size: 128). |
| Software Dependencies | No | The paper mentions training recipes from DeiT [50] and PyTorch [51] but does not provide specific version numbers for software dependencies such as Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | For CNN models, we follow most training settings in the original papers [34, 35], except that Res Net models are trained for 150 epochs. Vi T and Swin-T are trained for 300 epochs, following Dei T s training recipe [50, 51]. |