When Neural Networks Fail to Generalize? A Model Sensitivity Perspective
Authors: Jiajin Zhang, Hanqing Chao, Amit Dhurandhar, Pin-Yu Chen, Ali Tajer, Yangyang Xu, Pingkun Yan
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on multiple public datasets demonstrate the superiority of our approach |
| Researcher Affiliation | Collaboration | 1Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY, USA 2IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA 3Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA 4Department of Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY, USA |
| Pseudocode | Yes | Algorithm 1: Spectral adversarial data augmentation. |
| Open Source Code | Yes | The source code is available at https://github.com/ DIAL-RPI/Spectral-Adversarial-Data-Augmentation. |
| Open Datasets | Yes | DIGITS consists of 5 domains, including MNIST (Le Cun et al. 1998), SVHN (Netzer et al. 2011), MNISTM (Ganin and Lempitsky 2015), SYNTH (Ganin and Lempitsky 2015) and USPS (Le Cun et al. 1989). PACS (Li et al. 2017) is a more challenging domain generalization dataset including four domains, Photo, Art painting, Cartoon, and Sketch. CIFAR-10-C (Hendrycks and Dietterich 2019) is the corrupted version of CIFAR-10 (Krizhevsky and Hinton 2009). |
| Dataset Splits | Yes | We follow the official dataset split for training validation and testing. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models or processor types used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., Python, PyTorch, TensorFlow versions) needed to replicate the experiment. |
| Experiment Setup | Yes | In all the experiments, we set the weighting factor λ = 0.25, perturbation steps T = 5, step size δ = 0.08 and random initialization range ϵ = 0.2. The number of augmented images per training sample in Eq. 8 is set to 3. For DIGITS dataset, we trained a Conv Net (Le Cun et al. 1998) with SGD optimizer (default settings) for 50 epochs. The initial learning rate is 0.001, which decays by 0.1 for every 20 epochs. The batch size is 128. For the PACS dataset, Res Net-18 (He et al. 2016) is pretrained on Imagenet and finetuned in the source domain by SGD for 80 epochs. The initial learning of 0.01 is scheduled to decay by 0.1 for every 20 epochs. The batch size is 256. For CIFAR-10-C, a Wide Residual Network (Zagoruyko and Komodakis 2016) with 16 layers and width of 4 (WRN-16-4) was optimized with SGD for 200 epochs with batch size 256. The initial learning rate of 0.1 linearly decays by 0.1 for every 40 epochs. |