Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method
Authors: Kishaan Jeeveswaran, Elahe Arani, Bahram Zonooz
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed method in DIL setting (Van de Ven & Tolias, 2019) on two diverse datasets. DN4IL (Domain Net for Domain-IL) is a challenging dataset consisting of six vastly diverse domains and samples belonging to 100 classes (Gowda et al., 2023). On the other hand, i CIFAR20 (Xie et al., 2022) is the DIL setup of the CIFAR-100 dataset (Krizhevsky et al., 2009), where the 20 supercategories are considered actual classes and the five subcategories are considered new domains. We compare our approach with state-of-the-art rehearsalbased methods in CL literature under uniform experimental settings, focusing on the challenging low buffer regime where representation drift is most pronounced (Caccia et al., 2022). For a comprehensive study, we selected standard methods such as ER (Riemer et al., 2018), DER++ (Buzzega et al., 2020), CLS-ER (Arani et al., 2022), and DUCA (Gowda et al., 2023). |
| Researcher Affiliation | Collaboration | 1Dep. of Mathematics and Computer Science, Eindhoven University of Technology, NL 2Wayve Technologies Ltd, London, UK. |
| Pseudocode | Yes | Algorithm 1 Learning Algorithm for DARE. Algorithm 2 Intermediary Reservoir Sampling (IRS). |
| Open Source Code | Yes | Code at https://github.com/NeurAI-Lab/DARE. |
| Open Datasets | Yes | DN4IL (Gowda et al., 2023) is a challenging dataset consisting of six vastly diverse domains and samples belonging to 100 classes (Gowda et al., 2023). i CIFAR-20 (Xie et al., 2022) is the DIL setup of the CIFAR-100 dataset (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | Last Accuracy defines the final performance of the CL model on the validation set of all the tasks seen so far. The dataset consists of approximately 50k training images and 10k test images of size 32 32 (for iCIFAR-20). The dataset consists of approximately 67k training images and 19k test images of shape 64 64 (for DN4IL). |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, memory specifications, or cloud computing instances used for running the experiments. |
| Software Dependencies | No | The paper mentions using "mammoth framework" and "Res Net-18 architecture" but does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or framework versions). |
| Experiment Setup | Yes | We train our method with a batch size of 32, for 50 epochs per task on all datasets. Table 5 outlines the hyperparameters chosen for DARE and DARE++, while Table 6 lists the hyperparameters selected for various CL baselines in our study. lr denotes the learning rate for the entire learning trajectory in each task. We fixed the batch size to 32 for both the current task and old task samples (in buffer memory). |