Towards Practical Control of Singular Values of Convolutional Layers
Authors: Alexandra Senderovich, Ekaterina Bulatova, Anton Obukhov, Maxim Rakhuba
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To study the effects of the proposed regularization on the performance of CNNs, we combined it with various methods of controlling singular values and applied it during training on the image classification task on the CIFAR-10 and CIFAR-100 datasets [Krizhevsky et al., 2009].Evaluation Metrics We consider four main metrics: standard accuracy on the test split, accuracy on the CIFAR-C dataset (CC) [Hendrycks and Dietterich, 2019]1, accuracy after applying the Auto Attack module (AA) [Croce and Hein, 2020], and the Expected Calibration Error (ECE) [Guo et al., 2017]. |
| Researcher Affiliation | Academia | Alexandra Senderovich HSE University Ekaterina Bulatova HSE University Anton Obukhov ETH Zürich Maxim Rakhuba HSE University |
| Pseudocode | Yes | Algorithm 1 Singular values of a TT-compressed periodic convolutional layer (d = 2). |
| Open Source Code | Yes | The source code is available at: https://github.com/White Tea Dragon/practical_svd_conv |
| Open Datasets | Yes | applied it during training on the image classification task on the CIFAR-10 and CIFAR-100 datasets [Krizhevsky et al., 2009]. |
| Dataset Splits | No | The paper mentions 'standard accuracy on the test split' but does not explicitly detail the percentages or counts for training, validation, and test splits, nor does it refer to specific standard splits for all datasets used, beyond just mentioning the test split. |
| Hardware Specification | Yes | The code was implemented in Py Torch [Paszke et al., 2019] and executed on a NVIDIA Tesla V100, 32GB. |
| Software Dependencies | No | The paper mentions 'Py Torch' and the 'Auto Attack module', but does not provide specific version numbers for these software components. For example, 'The code was implemented in Py Torch [Paszke et al., 2019]' lacks a version number for PyTorch. |
| Experiment Setup | Yes | We train our models with batches of 128 elements for 200 epochs, using SGD optimizer and a learning rate of 0.1, multiplied by 0.1 after 60, 120, and 160 epochs. |