Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
Authors: Sungmin Cha, Kyunghyun Cho, Taesup Moon
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results demonstrate that our PNR framework achieves state-of-the-art performance in representation learning during CSSL by effectively balancing the trade-off between plasticity and stability. |
| Researcher Affiliation | Collaboration | 1New York University 2Genentech 3ASRI / INMC / IPAI / AIIS, Seoul National University. |
| Pseudocode | No | The paper describes the methods using mathematical equations and text, but it does not include a distinct block labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | The paper states 'We implement our PNR based on the code provided by Ca SSLe' and refers to Ca SSLe's GitHub, but it does not explicitly state that the code for PNR itself is open-source or provide a link to it. |
| Open Datasets | Yes | We conduct experiments on four datasets: CIFAR-100 (Krizhevsky et al., 2009), Image Net-100 (Deng et al., 2009), Domain Net (Peng et al., 2019), and Image Net-1k (Deng et al., 2009) following the training and evaluation process outlined in (Fini et al., 2022). |
| Dataset Splits | Yes | FTi signifies the linear evaluation accuracy (on the validation dataset of task i) of the model trained with a SSL algorithm for task i. |
| Hardware Specification | Yes | All experiments are conducted on an NVIDIA RTX A5000 with CUDA 11.2 |
| Software Dependencies | No | All experiments are conducted on an NVIDIA RTX A5000 with CUDA 11.2 and we follow the experimental settings proposed in the Ca SSLe s code (Fini et al., 2022). We employ LARS (You et al., 2017) to train a model during CSSL. The Res Net-18 (...) model implemented in Py Torch is used (...). Only CUDA version is specified, not PyTorch or LARS versions. |
| Experiment Setup | Yes | Table 14 presents the training details for each algorithm utilized in our Continual Self-supervised Learning (CSSL) experiments. All experiments are conducted on an NVIDIA RTX A5000 with CUDA 11.2 and we follow the experimental settings proposed in the Ca SSLe s code (Fini et al., 2022). We employ LARS (You et al., 2017) to train a model during CSSL. |