Investigating the Benefits of Projection Head for Representation Learning
Authors: Yihao Xue, Eric Gan, Jiayi Ni, Siddharth Joshi, Baharan Mirzasoleiman
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate our results through various experiments on CIFAR-10/100, Urban Cars and shifted versions of Image Net. We also introduce a potential alternative to projection head, which offers a more interpretable and controllable design. |
| Researcher Affiliation | Academia | Department of Computer Science, University of California, Los Angeles |
| Pseudocode | No | The paper contains theoretical analyses and experimental results but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating the availability of its source code. |
| Open Datasets | Yes | We empirically validate our results through various experiments on CIFAR-10/100, Urban Cars and shifted versions of Image Net. |
| Dataset Splits | No | The paper discusses training and evaluation but does not specify exact train/validation/test dataset splits with percentages or counts for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for its experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions software like 'Sim CLR loss' and 'Adam optimizer' but does not specify version numbers for these or other software dependencies. |
| Experiment Setup | Yes | By default, we use a temperature of 0.5 and minimize the Sim CLR loss with the Adam optimizer. Our training batch size is 512, with a learning rate of 0.001 and weight decay set to 1 10 6. We train for 400 epochs. |