Improving Knowledge Tracing via Pre-training Question Embeddings
Authors: Yunfei Liu, Yang Yang, Xianyu Chen, Jian Shen, Haifeng Zhang, Yong Yu
IJCAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we demonstrate that large gains on KT can be realized by pre-training embeddings for each question on abundant side information, followed by training deep KT models on the obtained embeddings. To be specific, the side information includes question difficulty and three kinds of relations contained in a bipartite graph between questions and skills. To pre-train the question embeddings, we propose to use product-based neural networks to recover the side information. As a result, adopting the pretrained embeddings in existing deep KT models significantly outperforms state-of-the-art baselines on three common KT datasets.Experiment results on three real-world datasets show that using PEBG can outperform the state-of-the-art models, improving AUC by 8.6% on average. |
| Researcher Affiliation | Academia | Yunfei Liu1 , Yang Yang1 , Xianyu Chen1 , Jian Shen1 , Haifeng Zhang2 and Yong Yu1 1Shanghai Jiao Tong University 2The Center on Frontiers of Computing Studies, Peking University |
| Pseudocode | No | The paper describes its method using mathematical formulations and descriptive text, but it does not include a structured pseudocode block or algorithm listing. |
| Open Source Code | Yes | Experiment code: https://github.com/lyf-1/PEBG |
| Open Datasets | Yes | We use three real-world datasets, and the statistics of the three datasets are shown in Table 1. ASSIST092 and ASSIST123 are both collected from the ASSISTments online tutoring platform [Feng et al., 2009]. (Footnote 2: https://sites.google.com/site/assistmentsdata/home/assistment-2009-2010-data/skill-builder-data-2009-2010) (Footnote 3: https://sites.google.com/site/assistmentsdata/home/2012-13-school-data-with-affect) Ed Net4 is collected by [Choi et al., 2019]. (Footnote 4: https://github.com/riiid/ednet) |
| Dataset Splits | Yes | We divide each dataset into 80% for training and validation, and 20% for testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as exact GPU or CPU models. |
| Software Dependencies | No | The paper mentions using 'Adam algorithm to optimize our model' and 'dropout with a probability of 0.5', but it does not specify version numbers for any software dependencies or libraries like Python, TensorFlow, or PyTorch. |
| Experiment Setup | Yes | The dimension of vertex features dv is set to 64. The final question embeddings dimension d = 128. λ in Eqn.(17) is 0.5. We use the Adam algorithm to optimize our model, and mini-batch size for three datasets is set to 256, the learning rate is 0.001. We also use dropout with a probability of 0.5 to alleviate overfitting. |