Orthogonalizing Convolutional Layers with the Cayley Transform
Authors: Asher Trockman, J Zico Kolter
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments have two goals: First, we show that our layer remains orthogonal in practice. Second, we compare the performance of our layer versus alternatives (particularly BCOP) on two adversarial robustness tasks on CIFAR-10 |
| Researcher Affiliation | Collaboration | Asher Trockman Computer Science Department Carnegie Mellon University ashert@cs.cmu.edu J. Zico Kolter Computer Science Department Carnegie Mellon University & Bosch Center for AI zkolter@cs.cmu.edu |
| Pseudocode | Yes | Algorithm 1: Orthogonal convolution via the Cayley transform. |
| Open Source Code | Yes | Code is available at https://github.com/locuslab/orthogonal-convolutions. |
| Open Datasets | Yes | For all experiments, we used CIFAR-10 with standard augmentation, i.e., random cropping and flipping. |
| Dataset Splits | No | The paper mentions using CIFAR-10 but does not specify the training, validation, or test split percentages or sample counts. |
| Hardware Specification | Yes | Width 1 was on a Nvidia RTX 2080 Ti, while 2, 3, 6, and 8 were on a Nvidia Quadro RTX 8000. |
| Software Dependencies | Yes | In Py Torch 1.8, our layer can be implemented as follows. |
| Experiment Setup | Yes | For each architecture/convolution pair, we tried learning rates in {10 5, 10 4, 10 3, 10 2, 10 1}, choosing the one with the best test accuracy. Most often, 0.001 is appropriate. ... Adam (Kingma & Ba, 2014) showed a significant improvement over plain SGD, and we used it for all experiments. ... The batch size was fixed at 128 for all graphs |