Let Go of Your Labels with Unsupervised Transfer
Authors: Artyom Gadetsky, Yulun Jiang, Maria Brbic
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate TURTLE on a diverse benchmark suite of 26 datasets and show that it achieves new state-of-the-art unsupervised performance. |
| Researcher Affiliation | Academia | 1EPFL, Lausanne, Switzerland. |
| Pseudocode | Yes | The pseudocode of TURTLE is provided in Algorithm B1 with implementation details in Appendix B.3. |
| Open Source Code | Yes | Code is publicly available at https://github.com/mlbio-epfl/turtle. |
| Open Datasets | Yes | We study the performance of TURTLE on the extensive benchmark of 26 vision datasets (Radford et al., 2021). The detailed description of each dataset is provided in Appendix B.1. |
| Dataset Splits | Yes | We use 10-fold cross-validation to select the best task. |
| Hardware Specification | Yes | The selection process takes a few minutes on small datasets, and around 8 hours on Image Net, with a single NVIDIA A100 GPU. |
| Software Dependencies | No | We use the cu ML.Logistic Regression (Raschka et al., 2020) for linear probe evaluation in our paper 3. The cu ML library allows for GPU acceleration... ADAM (Kingma & Ba, 2015) optimizer is used for the training of both linear classifier and task encoder. |
| Experiment Setup | Yes | We use 10000 as the default batch-size. We update the linear classifier for M = 10 steps at each iteration and train the task encoder for T = 6000 iterations in total. If not specifically mentioned, we set the entropy regularization parameter γ = 10 for all experiments. We do a grid search over 5 different learning rates for both task encoder and linear classifier with η {0.01, 0.005, 0.001, 0.0005, 0.0001}, respectively. |