BETA-CD: A Bayesian Meta-Learned Cognitive Diagnosis Framework for Personalized Learning
Authors: Haoyang Bi, Enhong Chen, Weidong He, Han Wu, Weihao Zhao, Shijin Wang, Jinze Wu
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we first introduce the datasets and our experimental setups. Then, we conduct extensive experiments to compare the performances of CDMs optimized by the ordinary optimization approach and the proposed BETA-CD (hereinafter referred to as ORD-CDMs and BETA-CDMs, respectively) to answer the following questions: |
| Researcher Affiliation | Collaboration | Haoyang Bi1, 2, Enhong Chen 1, 2, Weidong He1, 2, Han Wu1, 2, Weihao Zhao1, 2, Shijin Wang2,3, Jinze Wu3 1 Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China 2 State Key Laboratory of Cognitive Intelligence 3 i FLYTEK AI Research, i FLYTEK CO., LTD. |
| Pseudocode | Yes | Algorithm 1: BETA-CD Meta-training and Algorithm 2: BETA-CD Meta-testing are provided in the paper. |
| Open Source Code | Yes | The code will be publicly available at https://github.com/Ayi Star/pyat. |
| Open Datasets | Yes | We evaluate our framework with three real-world datasets consisting of massive students practice logs, i.e., ECPE, ASSIST and EXAM. ECPE (Examination for the Certificate of Proficiency in English), collected from a standard English test by the English Language Institute of the University of Michigan, is well adopted in educational psychology. ASSIST (ASSISTments 2017 skill builder) is a widely used educational dataset containing students practice in mathematics with the ASSISTments system. |
| Dataset Splits | Yes | For each dataset, we randomly divide the students in each dataset by 6:2:2, where the 60% partition contains the historical students in the intelligent tutoring system used to train the CDM, one 20% partition acts as new students to be diagnosed, and the other 20% partition is used for early stopping and hyperparameter tuning. For each new student, 20% of his/her logs are left out as a validation question set (i.e., Qv, Rv). Among the rest 80%, we randomly select different numbers of logs as the training question sets (i.e., Qt, Rt). |
| Hardware Specification | Yes | All the methods are implemented by Py Torch using Python and all the experiments are conducted on a Linux server with two 2.30GHz Intel(R) Xeon(R) Gold 5118 CPUs and one 11G GTX 1080ti GPU. |
| Software Dependencies | No | The paper states, 'All the methods are implemented by Py Torch using Python', but it does not specify the version numbers for PyTorch or Python, nor any other libraries with their versions. |
| Experiment Setup | Yes | We set the mini-batch size T = 8, the sampling sizes Nt = Nv = 4 and the number of inner updates K = 3. The KL weighting parameter η in the local loss is set to 10 4. The learning rate for local updates is initialized to α = 0.1. We set the learning rate for metaupdates γ = 10 4 and use the Adam algorithm (Kingma and Ba 2014) for meta-optimization. |