Dilated convolution with learnable spacings
Authors: Ismail Khalfaoui Hassani, Thomas Pellegrini, Timothée Masquelier
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper presents a new method to increase the RF size without increasing the number of parameters. ... We first tried our approach on Res Net50: we drop-in replaced the standard convolutions with DCLS ones, which increased the accuracy of Image Net1k classification at iso-parameters, but at the expense of the throughput. Next, we used the recent Conv Ne Xt state-of-the-art convolutional architecture and drop-in replaced the depthwise convolutions with DCLS ones. This not only increased the accuracy of Image Net1k classification but also of typical downstream and robustness tasks... 4 RESULTS AND DISCUSSION 4.2 EMPIRICAL EVALUATIONS ON IMAGENET1K Table 1: Classification accuracy on Image Net-1K using Res Net-50. Table 2: Classification accuracy on Image Net-1K. 4.3 EMPIRICAL EVALUATION ON DOWNSTREAM AND ROBUSTNESS TASKS |
| Researcher Affiliation | Academia | Ismail Khalfaoui-Hassani Artificial and Natural Intelligence Toulouse Institute (ANITI) Universit e de Toulouse, France ismail.khalfaoui-hassani@univ-tlse3.fr Thomas Pellegrini IRIT, ANITI, Universit e de Toulouse, CNRS, Toulouse INP, UT3, France thomas.pellegrini@irit.fr Timoth ee Masquelier Cer Co UMR 5549 CNRS & Universit e de Toulouse, France timothee.masquelier@cnrs.fr |
| Pseudocode | Yes | 8 APPENDIX: THE 2D-DCLS KERNEL CONSTRUCTION ALGORITHM ... Algorithm 1 2D-DCLS kernel construction forward pass ... Algorithm 2 2D-DCLS kernel construction backward pass |
| Open Source Code | Yes | The code of the method is based on Py Torch and available at https://github.com/K-H-I smail/Dilated-Convolution-with-Learnable-Spacings-Py Torch. |
| Open Datasets | Yes | In the following, we report the top-1 accuracies found on the Image Net1k validation dataset (Deng et al., 2009), using Image Net1k training dataset only. ... Semantic segmentation on ADE20k. ... (Zhou et al., 2019) ... Object detection and segmentation on COCO. ... (Lin et al., 2014) |
| Dataset Splits | Yes | In the following, we report the top-1 accuracies found on the Image Net1k validation dataset (Deng et al., 2009), using Image Net1k training dataset only. ... We present numerically in Table 2 and graphically in Fig. 1, the results obtained for Conv Ne Xt using the settings for Image Net1k training with input crops of size 224 224, as described in Liu et al. (2022b): Table 5. |
| Hardware Specification | Yes | The throughput was calculated at inference time, on image crops of size 224 224 using a single V100-32gb gpu. (Table 1 caption) ... The inference throughput was calculated at inference using a single V100-32gb gpu... (Table 2 caption) ... The inference throughput was calculated at inference using a single A100-80gb gpu... (Table 3 caption) ... Measures were carried using a single A100-80gb gpu. (Figure 12, 13 captions) ... Measures were carried using a single Quadro RTX 8000 gpu. (Figure 14, 15 captions) |
| Software Dependencies | No | The code of the method is based on Py Torch and available. ... The real code for the kernel construction in 1D, 2D, and 3D cases is included in Appendix 9. This code is written in native Py Torch language, with classical modules, and does not require any compilation or adaptation. While PyTorch is mentioned, no specific version number is provided. |
| Experiment Setup | Yes | 3 LEARNING TECHNIQUES ... Weight decay: ...we set this hyperparameter to 0 for the kernel positions and kept it unchanged for all the other parameters. ... Positions initialization: ...a centered normal law of standard deviation 0.5. ... Dilated kernel size tuning: ... (7 for Res Net-50-dcls and 17 for Conv Ne Xt-dcls; larger values did not bring any gain in accuracy). ... Positions learning rate scaling: ...scaled the learning rate of all the kernel positions by a factor of 5. ... Synchronizing positions: We shared the kernel positions across convolution layers with the same number of parameters... ... Using Conv Ne Xt. We present numerically in Table 2 and graphically in Fig. 1, the results obtained for Conv Ne Xt using the settings for Image Net1k training with input crops of size 224 224, as described in Liu et al. (2022b): Table 5. |