On the Connection between Pre-training Data Diversity and Fine-tuning Robustness
Authors: Vivek Ramanujan, Thao Nguyen, Sewoong Oh, Ali Farhadi, Ludwig Schmidt
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate our findings on pre-training distributions drawn from various natural and synthetic data sources, primarily using the i Wild Cam-WILDS distribution shift as a test for robustness. |
| Researcher Affiliation | Collaboration | Vivek Ramanujan Thao Nguyen Sewoong Oh Ludwig Schmidt Ali Farhadi University of Washington Allen Institute for AI {thaottn,ramanv}@cs.washington.edu |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository for their methodology. |
| Open Datasets | Yes | We use Image Net [11] and i Naturalist [46] as the primary pre-training distributions of interest... The downstream task of interest is wildlife classification with the i Wild Cam WILDS dataset [22]. |
| Dataset Splits | No | There are two test sets for evaluation: ID test data consists of images taken by the same camera traps as the training set, but on different days from the training and validation (ID) images. In contrast, OOD test data contains images taken by a disjoint set of camera traps from training and validation (ID) images. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies or libraries used in the experiments. |
| Experiment Setup | No | The paper mentions 'fine-tuning for 12 epochs' and refers to 'Appendix A [for] further training details', but the main text does not provide a comprehensive set of specific hyperparameters or system-level training settings needed for reproduction. |