Robust and Generalizable Visual Representation Learning via Random Convolutions
Authors: Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, Marc Niethammer
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate Rand Conv and its mixing variant in extensive experiments on synthetic and realworld benchmarks as well as on the large-scale Image Net dataset. Our methods outperform single domain generalization approaches by a large margin on digit recognition datasets and for the challenging case of generalizing to the Sketch domain in PACS and to Image Net-Sketch. |
| Researcher Affiliation | Academia | Zhenlin Xu1, Deyi Liu1, Junlin Yang2, Colin Raffel1, and Marc Niethammer1 1 University of North Carolina at Chapel Hill 2 Yale University |
| Pseudocode | Yes | Algorithm 1 Learning with Data Augmentation by Random Convolutions |
| Open Source Code | Yes | Code is available at https://github.com/wildphoton/Rand Conv. |
| Open Datasets | Yes | The five digit recognition datasets (MNIST (Le Cun et al., 1998), MNIST-M (Ganin et al., 2016), SVHN (Netzer et al., 2011), SYNTH (Ganin & Lempitsky, 2014) and USPS (Denker et al., 1989)) have been widely used for domain adaptation and generalization research (Peng et al., 2019a;b; Qiao et al., 2020). The PACS dataset (Li et al., 2018b) considers 7-class classification on 4 domains: photo, art painting, cartoon, and sketch. Image Net-Sketch (Wang et al., 2019a) is an out-of-domain test set for models trained on Image Net. |
| Dataset Splits | Yes | All experiments are in the single-domain generalization setting where training and validation sets are drawn from one domain. We use the official data splits for training/validation/testing; no extra data augmentation is applied. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'Py Torch implementation' but does not specify its version number or any other software dependencies with version numbers. |
| Experiment Setup | Yes | We train the network with batch size 32 for 10,000 iterations. We apply the Adam optimizer with an initial learning rate of 0.0001. We use the official Py Torch implementation and the pretrained weights of Alex Net for our PACS experiments. Alext Net is finetuned for 50,000 iterations with a batch size 128. We use the SGD optimizer for training with an initial learning rate of 0.001, Nesterov momentum, and weight decay set to 0.0005. We let the learning rate decay by a factor of 0.1 after finishing 80% of the iterations. We set the batch size to 256 and train Alex Net from scratch for 90 epochs. We apply the SGD optimizer with an initial learning rate of 0.01, momentum 0.9, and weight decay 0.0001. We reduce the learning rate via a factor of 0.1 every 30 epochs. |