Closed-Loop Memory GAN for Continual Learning
Authors: Amanda Rios, Laurent Itti
IJCAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate incremental class learning using a notoriously hard paradigm, single-headed learning, in which each task is a disjoint subset of classes in the overall dataset, and performance is evaluated on all previous classes. First, we show that when constructing a dynamic memory unit to preserve sample heterogeneity, model performance asymptotically approaches training on the full dataset. We then show that using a stochastic generator to continuously output fresh new images during training increases performance significantly further meanwhile generating quality images. We compare our approach to several baselines including fine-tuning by gradient descent (FGD), Elastic Weight Consolidation (EWC), Deep Generative Replay (DGR) and Memory Replay GAN(Me RGAN). |
| Researcher Affiliation | Academia | Amanda Rios and Laurent Itti University of Southern California, Los Angeles, USA {amandari, itti}@usc.edu |
| Pseudocode | Yes | Algorithm 1: Clo GAN Train Input : Data Sreal t ,...,Sreal T ; Require: T: Number of Tasks; It : Number of iterations; B : Buffer Size; Kc : Number of clusters per class; λmem : Memory importance; 1 θ G,θ D,C,θ C Train AC-GAN(St=1) for i = 1 to I1 2 Smemory t=1 Buffer Construct (K,B,Sreal t ) 3 for t 2 to T do 4 S t = Sreal t λmem Smemory t 1 5 for i 1 to It do 6 Si t Batch(S t ) 7 SGAN t 1 Forward (G(z, yc t 1)) 8 SGAN t 1 Filter (SGAN t 1 ) 9 Si t Si t SGAN t 1 10 θ G,θ D,C,θ C Train AC-GAN(Si t) 12 Smemory t Buffer Construct (K,B,Smemory t 1 ,Sreal t ) Figure 2: Training Algorithm. Procedure Train is described in section 3.3; Filter in 3.4; Buf f er Construct in 3.5 |
| Open Source Code | Yes | Details of the implementation can be found in the supplementary materials link. |
| Open Datasets | Yes | We evaluate incremental class learning in 4 datasets: MNIST [Le Cun et al., ], FASHION [Xiao et al., 2017], SVHN [Netzer et al., 2011] and EMNIST [Cohen et al., 2017]. |
| Dataset Splits | No | The paper mentions using well-known datasets (MNIST, FASHION, SVHN, EMNIST) and defines tasks based on disjoint subsets of classes, but it does not explicitly provide details on how these datasets were split into train/validation/test sets (e.g., specific percentages or sample counts for each split, or references to standard splits for these datasets). |
| Hardware Specification | No | The paper does not explicitly mention any specific hardware details such as GPU models, CPU models, or cloud computing instance types used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used (e.g., Python version, PyTorch version, CUDA version, etc.). |
| Experiment Setup | No | The paper describes the model architecture, training process, and high-level parameters for Algorithm 1 (e.g., Number of Tasks, Number of iterations, Buffer Size, Number of clusters per class, Memory importance). However, it does not explicitly provide concrete hyperparameter values such as learning rate, batch size, number of epochs, or specific optimizer settings in the main text. It mentions 'Details of the implementation can be found in the supplementary materials link', suggesting these might be provided elsewhere, but not in the main paper. |