EquiMod: An Equivariance Module to Improve Visual Instance Discrimination
Authors: Alexandre DEVILLERS, Mathieu Lefort
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that applying that module to state-of-the-art invariance models, such as BYOL and Sim CLR, increases the performances on the usual CIFAR10 and Image Net datasets. Moreover, while our model could collapse to a trivial equivariance, i.e. invariance, we observe that it instead automatically learns to keep some augmentations-related information beneficial to the representations. Source code is available at https://github.com/ADevillers/ Equi Mod...Results of this linear evaluation are presented in Table 1 |
| Researcher Affiliation | Academia | Alexandre Devillers & Mathieu Lefort Univ Lyon, UCBL, CNRS, INSA Lyon LIRIS, UMR5205, F-69622 Villeurbanne, France {alexandre.devillers,mathieu.lefort}@liris.cnrs.fr |
| Pseudocode | No | The paper does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Source code is available at https://github.com/ADevillers/ Equi Mod |
| Open Datasets | Yes | In our experimentations, we tested our method on Image Net (IN) (Deng et al., 2009) and CIFAR10 (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | After training on either Image Net or CIFAR10, we evaluate the quality of the learned representation with the linear evaluation which is usual in the literature. |
| Hardware Specification | No | This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-AD011013160 and 2022-A0131013831) and GPUs donated by the NVIDIA Corporation. This mentions 'HPC resources' and 'GPUs' but lacks specific models or quantities (e.g., 'NVIDIA A100' or 'Tesla V100'). |
| Software Dependencies | No | The paper does not specify version numbers for any software dependencies (e.g., PyTorch version, CUDA version). |
| Experiment Setup | Yes | The model is trained for 800 epochs with 10 warm-up epochs and a cosine decay learning rate schedule. We have used a batch size of 4096 for Image Net and 512 for CIFAR10, while using an initial learning rate of 2.4 for Image Net (where we use 4.8 for Sim CLR without Equi Mod, as in the original paper) and 4.0 for CIFAR10. For the optimizer, we fix the momentum to 0.9 and the weight decay to 1e 6. |