Channel Regeneration: Improving Channel Utilization for Compact DNNs
Authors: Ankit Sharma, Hassan Foroosh
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on several image classification benchmarks and on semantic segmentation task demonstrate the effectiveness of applying the channel regeneration technique to compact architectures. |
| Researcher Affiliation | Academia | Ankit Sharma and Hassan Foroosh Computational Imaging Lab Department of Computer Science, University of Central Florida Orlando, USA ankit.sharma285@knights.ucf.edu, Hassan.Foroosh@ucf.edu |
| Pseudocode | Yes | Algorithm 1: Channel Regeneration Input: randomly initialized network, Training epochs TE, regeneration epochs ERG, total number of channels C 1: Train the model 2: for t 1 to TE do 3: L = LCE(output, label) Cross entropy loss 4: γall = {} 5: if t = ERG then Regeneration Epoch 6: for k 1 to C do 7: γall.append(γk) 8: end for 9: γmed = Median(Sort(γall)) Threshold γ 10: for k 1 to C do 11: if γk γmed then 12: γk = γmed Channel Regeneration 13: end if 14: end for 15: end if 16: end for Output: Trained network |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | CIFAR-10/100: The CIFAR datasets consist of natural color images with 32 x 32 dimensions. Tiny-Image Net: Tiny-Image Net is a subset of the full Image Net dataset and essentially a downsampled version of the dataset. Image Net: The Image Net dataset contains 1.2 million training images and 50,000 validation images of 1000 classes. We evaluate three different backbone architectures on the ADE20K dataset and report the results in Table 8. |
| Dataset Splits | Yes | CIFAR-10/100: The datasets contain 50, 000 images for training and 10, 000 images for validation. Tiny-Image Net: Each class has 500 for training and 50 images for validation. Image Net: The Image Net dataset contains 1.2 million training images and 50,000 validation images of 1000 classes. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper describes training routines and hyper-parameters, but does not list specific software dependencies with version numbers (e.g., library or framework versions). |
| Experiment Setup | Yes | With regards to the classification problem, we follow the training routine described in (Devries and Taylor 2017), which is initial LR = 0.1 divided by 5 at 60th, 120th, 160th epochs, train for 200 epochs with batch size 128 and weight decay 5e-4, Nesterov momentum of 0.9. ... The batch size is 32 for all architectures reported on this dataset. We train for 90 epochs and decrease the learning rate by 0.1 at [30, 60] epochs. Image Net: ... train a Mobile Netv2 model for 150 epochs with an initial learning rate = 0.05 and, we follow a cosine scheduling policy. We set the batch size to 256, and weight decay is 4e 5. |