A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation
Authors: Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, Tieniu Tan
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive results on 17 datasets validate that our method surpasses or achieves comparable results with state-of-the-art methods on few-shot classification, imbalanced learning, and out-of-distribution generalization. In addition, we extend our method to base-to-new generalization and unsupervised learning, once again demonstrating its superiority over competing approaches. |
| Researcher Affiliation | Academia | Zhengbo Wang1,2 Jian Liang2,3 Lijun Sheng1,2 Ran He2,3 Zilei Wang1 Tieniu Tan2,4 1 University of Science and Technology of China 2 CRIPAC & MAIS, Institute of Automation, Chinese Academy of Sciences (CASIA) 3 School of Artificial Intelligence, University of Chinese Academy of Sciences 4 Nanjing University |
| Pseudocode | Yes | A.2 Pseudocode Algorithm 1 Pytorch-like pseudocode for our method. |
| Open Source Code | Yes | Our code is publicly available at https://github.com/mrflogs/ICLR24. |
| Open Datasets | Yes | According to previous works (Radford et al., 2021; Zhou et al., 2022a;b; Zhang et al., 2022), we select 11 publicly available image classification datasets to assess the effectiveness of CLIP few-shot classification, base-to-new generalization, and unsupervised learning. These datasets cover a range of image recognition tasks, including generic object recognition with Image Net (Deng et al., 2009) and Caltech101 (Li et al., 2004), fine-grained image recognition with Oxford Pets (Parkhi et al., 2012), Stanford Cars (Krause et al., 2013), Flowers102 (Nilsback & Zisserman, 2008), Food101 (Bossard et al., 2014) and FGVCAircraft (Maji et al., 2013), satellite image classification with Euro SAT (Helber et al., 2019), action classification with UCF101 (Soomro et al., 2012), texture classification with DTD (Cimpoi et al., 2014), and scene recognition with SUN397 (Xiao et al., 2010). |
| Dataset Splits | Yes | The hyperparameter α, which is used to ensemble the classifiers, is searched in the validation set with values ranging from 0.0001 to 100.0, and this value is kept constant for new class data. And the k for the KNN algorithm to synthesize the new class dataset is set to 64. All experiments are conducted on a single NVIDIA Ge Force RTX 3090. To obtain a reliable estimate of model performance, we conduct three runs with different random seeds and averaged the results. |
| Hardware Specification | Yes | All experiments are conducted on a single NVIDIA Ge Force RTX 3090. |
| Software Dependencies | No | No specific software versions (e.g., Python 3.x, PyTorch 1.y, CUDA z.w) are mentioned. Algorithm 1 is described as 'Pytorch-like pseudocode' but no versions are provided. |
| Experiment Setup | Yes | The hyperparameter α, which is used to ensemble the classifiers, is searched in the validation set with values ranging from 0.0001 to 100.0, and this value is kept constant for new class data. And the k for the KNN algorithm to synthesize the new class dataset is set to 64. |