Continual Graph Convolutional Network for Text Classification
Authors: Tiandeng Wu, Qijiong Liu, Yi Cao, Yao Huang, Xiao-Ming Wu, Jiandong Ding
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Offline experiments on five public datasets also show Cont GCN can improve inference quality. A 3-month A/B test on Huawei public opinion analysis system shows Cont GCN achieves 8.86% performance gain compared with state-of-the-art methods. |
| Researcher Affiliation | Collaboration | Tiandeng Wu1 , Qijiong Liu2*, Yi Cao1, Yao Huang1, Xiao-Ming Wu2 , Jiandong Ding1 1 Huawei Technologies Co., Ltd., China 2 The Hong Kong Polytechnic University, Hong Kong |
| Pseudocode | Yes | Algorithm 1: Continual OMM updating algorithm |
| Open Source Code | Yes | The source code will be released at https://github.com/Jyonn/Cont GCN. |
| Open Datasets | Yes | Datasets. As described in (Lin et al. 2021), we have performed experiments on five text classification datasets which are commonly used in real-world applications: 20Newsgroups (20NG), Ohsumed, R52 Reuters, R8 Reuters, and Movie Review (MR) datasets. |
| Dataset Splits | Yes | We randomly chose 10% of the training set for validation purposes for all datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using PLMs like BERT, RoBERTa, and XLNet, and the Adam optimizer, but does not specify version numbers for any software libraries or dependencies (e.g., PyTorch, TensorFlow versions, or specific PLM library versions). |
| Experiment Setup | Yes | Implementation Details. We adopt the Adam optimizer (Kingma and Ba 2015) to train the network of our Cont GCN model and the baseline models, which are consistent in the following parameters. The following parameters are kept consistent across all models: the number of graph convolutional layers, if applicable, is set to 3; the embedding dimension is set to 768; and the batch size is set to 64. In the post-pretraining phase, we set the learning rate for PLM to 1e-4. During training, we set different learning rates for PLM and other randomly initialized parameters including those of the GCN network, following Lin et al. (2021). Precisely, we set 1e-5 to finetune Ro BERTa and BERT, 5e-6 to finetune XLNet, and 5e-4 to other parameters. |