Neural Priming for Sample-Efficient Adaptation
Authors: Matthew Wallingford, Vivek Ramanujan, Alex Fang, Aditya Kusupati, Roozbeh Mottaghi, Aniruddha Kembhavi, Ludwig Schmidt, Ali Farhadi
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform extensive experiments on 7 transfer learning and 4 distribution shift datasets to validate our method. |
| Researcher Affiliation | Collaboration | University of Washington PRIOR, Allen Institute for AI LAION {mcw244,ramanv}@cs.washington.edu |
| Pseudocode | No | The paper describes the steps of the proposed method in detail within the text (e.g., 'We break our method down into two main steps: 1. Collecting the priming pool... and 2. model attunement...'), but it does not provide any formally labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/RAIVNLab/neural-priming. |
| Open Datasets | Yes | We perform our experiments with Open CLIP models [52] trained on LAION-2B and 400M [44]. We evaluate on standard transfer learning and distribution shift benchmarks. Image Net [7], Image Net V2 [41], Image Net Sketch [51], Image Net-R [18], and Image Net-A [19]. Stanford Cars [27], FGVCAircraft [31], Flowers102 [36], and Oxford Pets [38]. SUN397 [53]. |
| Dataset Splits | Yes | Table 7: Dataset Number of Classes Train Size Test Size Image Net 1000 1281167 50000 Stanford Cars 196 8144 8041 FGVC Aircraft 100 6667 3333 Flowers102 102 2040 6149 Food101 102 75750 25250 Oxford Pets 37 3680 3669 SUN397 397 19850 19850 |
| Hardware Specification | Yes | Here we study the wall-clock time for construction of the initial priming pool on consumer hardware (Intel 10900k CPU and Samsung 980 Pro NVME drive). To filter a priming pool of 1.1m images to 51k using the test set of Image Net V2 took 3.2 minutes on consumer hardware (two 3090 GPUs)... |
| Software Dependencies | Yes | We use the Open CLIP models we use can be found at https://github.com/mlfoundations/open_clip. We use the following specific models: (Vi T-B-32, laion2b_s34b_b79k), (Vi T-B-16, laion2b_s34b_b88k), (Vi T-L-14, laion2b_s32b_b82k). To perform fast substring search, we set up a Full Text Search (FTS) database over the metadata shards of LAION-2B using SQLite. |
| Experiment Setup | Yes | We perform last layer retraining for Neural Priming... Formally we use α = e |P |2/σ as the mixing-coefficient, where |P| is the number of image examples with σ equal to 100. We fine-tune for with a learning rate of 1e-6 and batch size 1024. For Image Net experiments, we the models for 50 epochs regardless of the shots. |