Contrastive-Equivariant Self-Supervised Learning Improves Alignment with Primate Visual Area IT
Authors: Thomas Yerxa, Jenelle Feather, Eero Simoncelli, SueYeon Chung
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that our proposed method systematically increases the ability of models to predict responses in macaque inferior temporal cortex. Our results demonstrate the promise of incorporating known features of neural computation into task-optimization for building better models of visual cortex. |
| Researcher Affiliation | Academia | Thomas Yerxa 1 Jenelle Feather 1,2 Eero P. Simoncelli 1,2 Sue Yeon Chung1,2 1Center for Neural Science, New York University 2Center for Computational Neuroscience, Flatiron Institute, Simons Foundation |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | All code used for pretraining, evaluation, and analyses, will be made available in a public github repository upon publication. |
| Open Datasets | Yes | We train using the Image Net-1k dataset and the standard set of augmentations first introduced in [Grill et al., 2020] |
| Dataset Splits | Yes | We report the resulting coefficient of determination (R2) on a heldout set of validation images (Fig. 2). ... Subsequently the linear classifier is retrained using both the train and validation sets, and we report the final accuracy on the held out test set. |
| Hardware Specification | Yes | All pretraining runs used 8 A100 Nvidia GPUs with 40GB of memory each. |
| Software Dependencies | No | The paper mentions using the LARS optimizer and the Brain Score evaluation pipeline, but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | For all experiments we use a Res Net-50 architecture [He et al., 2016] as the backbone representation network f. ... For all experiments we trained for 100 epochs using a batch size of 2048 and used the LARS optimizer [You et al., 2017] with weight decay of 1e-6 and momentum of 0.9. ... We use a base learning rate of 4.8 and a learning rate schedule consisting of linear warm-up for the first 10 epochs followed by cosine decay throughout training. ... For Barlow Twins we set the λBT , which balances the on and off diagonal loss terms, hyperparameter to 5e 3. For Sim CLR we used a temperature of τ = 0.15. |