Structured Convolutions for Efficient Neural Network Design
Authors: Yash Bhalgat, Yizhe Zhang, Jamie Menjay Lin, Fatih Porikli
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply structured convolutions to a wide range of architectures and analyze the performance and complexity of the decomposed architectures. We evaluate our method on Image Net [30] and CIFAR-10 [16] benchmarks for image classification and Cityscapes [4] for semantic segmentation. |
| Researcher Affiliation | Industry | Yash Bhalgat Yizhe Zhang Jamie Menjay Lin Fatih Porikli Qualcomm AI Research {ybhalgat, yizhez, jmlin, fporikli}@qti.qualcomm.com |
| Pseudocode | No | The paper describes steps for a 'Proposed Training Scheme' but it is not formatted as pseudocode or an algorithm block. |
| Open Source Code | No | The paper does not provide explicit statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | We evaluate our method on Image Net [30] and CIFAR-10 [16] benchmarks for image classification and Cityscapes [4] for semantic segmentation. |
| Dataset Splits | No | The paper mentions evaluating on Image Net and CIFAR-10 datasets, but it does not explicitly provide specific train/validation/test split percentages, sample counts, or citations to predefined splits within the main text. It refers to supplementary material for training implementation details. |
| Hardware Specification | Yes | We measure the memory and timeper-iteration for training with and w/o the Structural Regularization loss on an NVIDIA V100 GPU. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies such as libraries or solvers. It only mentions general training details are in the supplementary material without specific software versions. |
| Experiment Setup | Yes | To provide a comprehensive analysis, for each baseline architecture, we present structured counterparts, with version 'A' designed to deliver similar accuracies and version 'B' for extreme compression ratios. Using different {c, n} configurations per-layer, we obtain structured versions with varying levels of reduction in model size and multiplications/additions (please see Supplementary material for details). For the 'A' versions of Res Net, we set the compression ratio (CN 2/cn2) to be 2 for all layers. For the 'B' versions of Res Nets, we use nonuniform compression ratios per layer. Specifically, we compress stages 3 and 4 drastically (4 ) and stages 1 and 2 by 2 . Since Mobile Net is already a compact model, we design its 'A' version to be 1.33 smaller and 'B' version to be 2 smaller. |