A Spectral Approach to Item Response Theory
Authors: Duc Nguyen, Anderson Ye Zhang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on synthetic and real-life datasets, ranging from small education testing datasets to large recommendation systems datasets show that our algorithm is scalable, accurate, and competitive with the most commonly used methods in the literature. |
| Researcher Affiliation | Academia | Md. Naimul Hoque, Department of Computer and Information Science, University of Pennsylvania, mdnguyen@seas.upenn.edu; Anderson Y. Zhang, Department of Statistics and Data Science, University of Pennsylvania, ayz@wharton.upenn.edu |
| Pseudocode | Yes | Algorithm 1 Spectral Estimator; Algorithm 2 Accelerated Spectral Estimator |
| Open Source Code | Yes | We include the python implementation of our spectral algorithm in the supplementary materials. |
| Open Datasets | Yes | We mention here a few notable datasets: RIIID [1] (m = 6k, n = 23k, education testing dataset), ML-20M [28] (m = 27k, n = 138k), Book-Genome [31] (m = 10k, n = 350k). |
| Dataset Splits | No | For tuning the prior distribution for MMLE, we select the prior distribution that admits the highest log-likelihood on a validation set. However, the paper does not specify the methodology or percentages for the data splits. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as CPU or GPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions 'python implementation' and references to external open source implementations for baselines, but does not specify its own software dependencies with version numbers. |
| Experiment Setup | No | The paper states that CMLE, JMLE, and the spectral method 'requires minimal model tuning' but does not provide specific hyperparameter values, optimizer settings, or detailed training configurations for reproduction. |