A Bounded Ability Estimation for Computerized Adaptive Testing
Authors: Yan Zhuang, Qi Liu, Guanhao Zhao, Zhenya Huang, Weizhe Huang, Zachary Pardos, Enhong Chen, Jinze Wu, Xin Li
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on both realworld and synthetic datasets, show that it can reach the same estimation accuracy using 15% less questions on average, significantly reducing test length. |
| Researcher Affiliation | Collaboration | 1: Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China 2: State Key Laboratory of Cognitive Intelligence 3: University of California, Berkeley 4: i FLYTEK Co., Ltd |
| Pseudocode | Yes | Algorithm 1: The BECAT framework |
| Open Source Code | Yes | The code can be found in the github: https://github.com/ bigdata-ustc/Edu CAT. |
| Open Datasets | Yes | We conduct experiments on three educational benchmark datasets, namely ASSIST, NIPS-EDU, and EXAM. ASSIST [34] is collected from an online educational system ASSISTments and consists of students practice logs on mathematics. NIPS-EDU [35] refers to the large-scale dataset in Neur IPS 2020 Education Challenge, which is collected from students answers to questions from Eedi (an educational platform). |
| Dataset Splits | Yes | Following the common strategy [12], we use 70%-20%-10% students for training, validation, and testing respectively, and the students in validation/testing set won t appear in training. |
| Hardware Specification | Yes | We implement all the methods with Py Torch. We set batch size to 64 and the learning rate to 0.001, and optimize all the parameters using the Adam algorithm [61] on a Tesla V100-SXM2-32GB GPU. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'Adam algorithm' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We set batch size to 64 and the learning rate to 0.001, and optimize all the parameters using the Adam algorithm [61] on a Tesla V100-SXM2-32GB GPU. |