Subspace Regularizers for Few-Shot Class Incremental Learning
Authors: Afra Feyza Akyürek, Ekin Akyürek, Derry Wijaya, Jacob Andreas
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments aim to evaluate the effect of subspace regularization (1) on the learning of new classes, and (2) on the retention of base classes. To evaluate the generality of our method, we evaluate using two different experimental paradigms that have been used in past work: a multi-session experiment in which new classes are continuously added and the classifier must be repeatedly updated, and a single-session setup (T = 1) in which new classes arrive only once. |
| Researcher Affiliation | Academia | Afra Feyza Aky urek Boston University akyurek@bu.edu Ekin Aky urek MIT CSAIL akyurek@mit.edu Derry Tanti Wijaya Boston University wijaya@bu.edu Jacob Andreas MIT CSAIL jda@mit.edu |
| Pseudocode | No | The paper does not contain explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code for the experiments is released under https://github.com/feyzaakyurek/subspace-reg. |
| Open Datasets | Yes | We use the mini Image Net dataset (Vinyals et al., 2016; Russakovsky et al., 2015) for our multi-session evaluation. mini Image Net contains 100 classes with 600 samples per class. |
| Dataset Splits | Yes | Every D(t) consists of a support set S(t) (used for training) and a query set Q(t) (used for evaluation). [...] In our 1-shot experiments unless fine-tuning converges by then, we stop at the maximum number of epochs at 1000. We sample 2000 episodes which includes 5 novels classes and 1-5 samples from each and report average accuracy. |
| Hardware Specification | Yes | We use a single 32 GB V100 NVIDIA GPU for all our experiments. |
| Software Dependencies | No | The paper mentions software components like "Res Net" (He et al., 2016), "Glo Ve embeddings" (Pennington et al., 2014), and "Sentence-BERT" (Reimers & Gurevych, 2019) but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | Unless otherwise indicated we use the following default settings of Tian et al. (2020) in our feature extractor training. We use SGD optimizer with learning starting at 0.05 with decays by 0.1 at epochs 60 and 80. We train for a total of 100 epochs. Weight decay is 5e-4, momentum is 0.9 and batch size is 64. |