Item Response Ranking for Cognitive Diagnosis
Authors: Shiwei Tong, Qi Liu, Runlong Yu, Wei Huang, Zhenya Huang, Zachary A. Pardos, Weijie Jiang
IJCAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on two real-world datasets show that CDMs with IRR not only significantly outperforms the baselines, but also effectively provides interpretable insights for understanding the cognitive diagnostic results of students. |
| Researcher Affiliation | Academia | 1Anhui Province Key Laboratory of Big Data Analysis and Application, School of Computer Science and Technology & School of Data Science, University of Science and Technology of China 2University of California, Berkeley |
| Pseudocode | No | The paper describes methods in paragraph form and does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/bigdataustc/EduCDM. |
| Open Datasets | Yes | ASSISTments (ASSISTments 20092010 skill builder ) is an open dataset collected by the ASSISTments online tutoring systems [Feng et al., 2009]. Collected from a widely-used online learning system, MATH contains mathematical test items and logs of high school examinations. |
| Dataset Splits | Yes | For each dataset, we divide the students on each test item into training: test = 8:2. We use 90% of the training data to train model and apply grid search to adjust the hyper-parameters on the remaining 10% of the data (i.e., the validation dataset). |
| Hardware Specification | Yes | all experiments are run on a Linux server with two Intel(R) Xeon(R) E5-2699 v4 CPUs and a Tesla P100 PCIe GPU. |
| Software Dependencies | No | All models are implemented by MXNet using Python. (No version numbers provided for MXNet or Python). |
| Experiment Setup | Yes | All hyper-parameters are tuned in the validation datasets. λ is selected from [0.1, 0.01, 0.001, 0.0001]. N O and N U are selected from [1, 5, 10, 30]. Based on the performance on the validation datasets, we set λ = 0.0001 and N O = N U = 10. We initialize parameters in all networks with Xavier initialization [Glorot and Bengio, 2010] and we use the Adam algorithm [Kingma and Ba, 2014] for optimization. |