Models Out of Line: A Fourier Lens on Distribution Shift Robustness
Authors: Sara Fridovich-Keil, Brian Bartoldson, James Diffenderfer, Bhavya Kailkhura, Timo Bremer
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We approach this issue by conducting a comprehensive empirical study of diverse approaches that are known to impact OOD robustness on a broad range of natural and synthetic distribution shifts of CIFAR-10 and Image Net. |
| Researcher Affiliation | Collaboration | University of California, Berkeley Lawrence Livermore National Laboratory |
| Pseudocode | No | The paper describes the procedures for Fourier interpolation metrics in detail within the text, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and pretrained models are available at https://github.com/sarafridov/Robust Nets. |
| Open Datasets | Yes | Datasets. We consider two standard image classification tasks: CIFAR-10 [30] and Image Net [6]. |
| Dataset Splits | No | The paper mentions using an 'ID test/validation set' and refers to standard datasets like CIFAR-10 and Image Net which have common splits. However, it does not explicitly state specific percentages, absolute counts, or detailed methodologies for training, validation, or test data splits within the main text for reproduction purposes, beyond general references to these sets. |
| Hardware Specification | No | Our experiments did require substantial compute to evaluate all of our models on all of the interpolation paths, but we use pretrained models so we do not require any compute for training. |
| Software Dependencies | No | The paper mentions 'Torchvision' and other tools but does not specify version numbers for any software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | For each pruning strategy and each architecture, we prune models to 50%, 60%, 80%, 90%, and 95% sparsity. For all pruning methods, pruning is performed in an unstructured (i.e., individual weights are pruned) and global manner (i.e., prune to a given sparsity across the entire network). Additionally, for traditional and initialization LTs, pruning was performed in a layerwise fashion (i.e., where each network layer is pruned to the given sparsity level). |