Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlaps
Authors: Lu Han, Han-Jia Ye, De-Chuan Zhan
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show that our method can achieve competitive results against various traditional contrastive learning methods on different benchmarks. and 5 A PILOT STUDY, 6 EXPERIMENTS |
| Researcher Affiliation | Academia | Lu Han, Han-Jia Ye B, De-Chuan Zhan State Key Laboratory for Novel Software Technology, Nanjing University {hanlu,yehj}@lamda.nju.edu.cn, zhandc@nju.edu.cn |
| Pseudocode | Yes | Algorithm 1 Augmentation Component Analysis Algorithm |
| Open Source Code | Yes | Code available at https://github.com/hanlu-nju/Aug CA. |
| Open Datasets | Yes | CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009): two datasets containing totally 500K images of size 32 32 from 10 and 100 classes respectively. STL-10 (Coates et al., 2011): derived from Image Net (Deng et al., 2009), with 96 96 resolution images with 5K labeled training data from 10 classes. Additionally, 100K unlabeled images are used for unsupervised learning. Tiny Image Net: a reduced version of Image Net (Deng et al., 2009), composed of 100K images scaled down to 64 64 from 200 classes. Image Net-100 (Tian et al., 2020a): a subset of Image Net, with 100-classes. Image Net (Deng et al., 2009), the large-scale dataset with 1K classes. |
| Dataset Splits | Yes | STL-10 (Coates et al., 2011): derived from Image Net (Deng et al., 2009), with 96 96 resolution images with 5K labeled training data from 10 classes. Additionally, 100K unlabeled images are used for unsupervised learning. |
| Hardware Specification | Yes | In this paper, we conduct experiments mainly on the following datasets with RTX-3090 4. |
| Software Dependencies | No | The paper mentions 'adam optimizer' but does not specify programming languages, specific libraries (e.g., PyTorch, TensorFlow), or their version numbers. |
| Experiment Setup | Yes | For CIFAR-10 and CIFAR-100, we use 800 epochs with a learning rate of 3 10 3. For Tiny Image Net and STL-10, we train 1,000 epochs with a learning rate 2 10 3. We use a 0.1 learning rate decay at 100, 50, 20 epochs before the end. Due to hardware resource restrictions, we use a mini-batch of size 512. The weight decay is 1 10 6 if not specified. Following common practice in contrastive learning, we normalize the projected feature into a sphere. For CIFAR-10, we use α = 1. For the rest datasets, we use α = 0.2. By default, K is set to 2. For Image Net, we use the same hyperparameters as (Chen et al., 2020a) except batch size being 256, α = 0.2 and K = 2. |