BECLR: Batch Enhanced Contrastive Few-Shot Learning
Authors: Stylianos Poulakakis-Daktylidis, Hadi Jamali-Rad
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We then present a suite of extensive quantitative and qualitative experimentation to corroborate that BECLR sets a new state-of-the-art across ALL existing U-FSL benchmarks (to the best of our knowledge), and significantly outperforms the best of the current baselines (codebase available at Git Hub). |
| Researcher Affiliation | Collaboration | Stylianos Poulakakis-Daktylidis1 & Hadi Jamali-Rad1,2 1Delft University of Technology (TU Delft), The Netherlands 2Shell Global Solutions International B.V., Amsterdam, The Netherlands |
| Pseudocode | Yes | The pretraining pipeline of BECLR is summarized in Algorithm 1, and a Pytorch-like pseudo-code can be found in Appendix E. [...] Algorithm 3: Unsupervised Pretraining of BECLR: Py Torch-like Pseudocode [...] Algorithm 4: Dynamic Clustered Memory (Dy CE): Py Torch-like Pseudocode |
| Open Source Code | Yes | We then present a suite of extensive quantitative and qualitative experimentation to corroborate that BECLR sets a new state-of-the-art across ALL existing U-FSL benchmarks (to the best of our knowledge), and significantly outperforms the best of the current baselines (codebase available at Git Hub). |
| Open Datasets | Yes | We evaluate BECLR in terms of its in-domain performance on the two most widely adopted few-shot image classification datasets: mini Image Net (Vinyals et al., 2016) and tiered Image Net (Ren et al., 2018). Additionally, for the in-domain setting, we also evaluate on two curated versions of CIFAR-100 (Krizhevsky et al., 2009) for FSL, i.e., CIFAR-FS and FC100. |
| Dataset Splits | Yes | mini Image Net. It is a subset of Image Net (Russakovsky et al., 2015), containing 100 classes with 600 images per class. We randomly select 64, 16, and 20 classes for training, validation, and testing, following the predominantly adopted settings of Ravi & Larochelle (2016). |
| Hardware Specification | Yes | All training and evaluation experiments are conducted on 2 A40 NVIDIA GPUs. |
| Software Dependencies | No | We use Py Torch (Paszke et al., 2019) for all implementations. The paper mentions 'PyTorch' and cites its paper, but does not provide a specific version number for PyTorch or any other software libraries used. |
| Experiment Setup | Yes | We use a batch size of B = 256 images for all datasets, except for tiered Image Net (B = 512). [...] We use the SGD optimizer with a weight decay of 10 4, a momentum of 0.995, and a cosine decay schedule of the learning rate. [...] The initial learning rate is set to 0.3 for the smaller mini Image Net, CIFAR-FS, FC100 datasets and 0.1 for tiered Image Net, and we train for 400 and 200 epochs, respectively. The temperature scalar in the loss function is set to τ = 2. |