Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Focus Your Attention when Few-Shot Classification
Authors: Haoqing Wang, Shibo Jie, Zhihong Deng
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments show that our method can improve the performance of full or parameter-efficient fine-tuning methods on few-shot tasks. |
| Researcher Affiliation | Academia | Haoqing Wang Shibo Jie Zhi-Hong Deng School of Intelligence Science and Technology, Peking University EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/Haoqing-Wang/FORT. |
| Open Datasets | Yes | The experiments are conducted on the few-shot benchmark from [58] (i.e., CUB [61], Cars [30], Places [75] and Plantae [59] datasets), and two other fine-gained datasets: Aircraft [39] and Pets [43]. |
| Dataset Splits | No | The paper mentions 'We only use 50 tasks for quick hyper-parameter selection' which implies a validation step, but does not provide explicit training/validation/test dataset split percentages or absolute sample counts for each split needed to reproduce the data partitioning. |
| Hardware Specification | Yes | All experiments can be conducted on single Tesla V100 with 32GB memory. |
| Software Dependencies | No | The paper mentions using 'Adam W [37] optimizer' and 'SGD optimizer' but does not provide specific version numbers for software dependencies like Python, PyTorch, or other libraries. |
| Experiment Setup | Yes | We set the rank to 4 for Lo RA [24] and use 10 learnable prompts at each layer for VPT [26]. We mainly use Adam W [37] optimizer for fine-tuning... We set the batch size to 20, same as the number of classes... We set λ = 1 for DINO pre-trained model and λ = 50 for CLIP pre-trained model for simplicity. The other hyper-parameters, including learning rate, number of fine-tuning epochs, temperature τ and coefficient α, are changeable for different fine-tuning methods... To this end, we only provide their candidate values in Table 7. |