TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
Authors: Preetha Vijayan, Prashant Bhat, Bahram Zonooz, Elahe Arani
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental Setup: We expand the Mammoth CL repository in Py Torch [9]. On the basis of Class-IL and Task-IL scenarios, we assess the existing CL techniques against the proposed one. Experimental Results: We compare Tri RE with contemporary rehearsal-based and weight regularization methods in Class-IL and Task-IL settings. As shown in Table 1, Tri RE consistently outperforms |
| Researcher Affiliation | Collaboration | Preetha Vijayan1, , Prashant Bhat2,3,*, Bahram Zonooz2,3, , Elahe Arani2, 1Nav Info Europe 2Eindhoven University of Technology (TU/e) 3Tom Tom |
| Pseudocode | Yes | Algorithm 1 Proposed Approach Tri RE |
| Open Source Code | Yes | Code is available at https://github.com/Neur AI-Lab/Tri RE |
| Open Datasets | Yes | To evaluate the performance of our method in Task-IL and Class-IL scenarios, we employ three different datasets: Seq-CIFAR10, Seq-CIFAR100, and Seq-Tiny Image Net. These datasets are derived from CIFAR10, CIFAR100, and Tiny Image Net, respectively. |
| Dataset Splits | Yes | The hyperparameters required to replicate the results of Tri RE can be found in Table 5. These hyperparameters were determined through a tuning process involving different random initializations and a small portion of the training set reserved for validation. |
| Hardware Specification | Yes | Table 7 specifically lists the training times required to learn a single task (the first) for 3 epochs on an NVIDIA RTX 2080 Ti for Seq-CIFAR100 dataset with a buffer size of 200. |
| Software Dependencies | No | The paper mentions using 'Py Torch' but does not specify a version number for it or any other software dependencies. |
| Experiment Setup | Yes | The hyperparameters required to replicate the results of Tri RE can be found in Table 5... All experiments were conducted using a batch size of 32 and trained for 50 epochs. Tri RE was optimized using the Adam optimizer [24] implemented in Py Torch. Furthermore, the number of epochs allocated to each phase specified in Algorithm 1 was consistently set at a ratio of E1 : E2 : E3 = 3 : 1 : 1. |