Novel Spectral Algorithms for the Partial Credit Model
Authors: Duc Nguyen, Anderson Ye Zhang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we present the empirical findings to complement our theoretical contribution and showcase the usefulness of the spectral algorithm. The detailed description of the experiment is deferred to the supplementary materials. We compare the spectral algorithm against two well known and popular PCM inference algorithms the marginal likelihood estimate (MMLE) (Basu, 2011) and the joint maximum likelihood estimate (JMLE) (Andersen, 1973; Fischer, 1981; Haberman, 1977). |
| Researcher Affiliation | Academia | 1Department of Computer & Information Science, University of Pennsylvania 2Department of Statistics & Data Science, Wharton School, University of Pennsylvania. |
| Pseudocode | Yes | Algorithm 1 Spectral Algorithm, Algorithm 2 Parameter Shift Estimation Algorithm, Algorithm 3 The Spectral-EM Algorithm |
| Open Source Code | Yes | The implementation of our algorithms can be found at https://github.com/dnguyen1196/spectral-algos-discrete-data. |
| Open Datasets | Yes | Table 3: Datasets metadata and references. LSAT (Mc Donald, 2013), UCI (Hussain et al., 2018), GRADES (Cortez & Silva, 2008), HETREC (Cantador et al., 2011), EACH MOVIE (Harper & Konstan, 2015), ML-1M (Harper & Konstan, 2015), ML-10M (Harper & Konstan, 2015), ML-20M (Harper & Konstan, 2015), BOOK-GENOME (Kotkov et al., 2022). |
| Dataset Splits | Yes | For each of the remaining users, we leave out one rating from each user as part of the heldout test dataset and one rating as part of a validation dataset (for MMLE and Spectral-EM). |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used for running its experiments. |
| Software Dependencies | No | The paper mentions using specific implementations like 'MMLE and JMLE can be found in Sanchez (2021)' and provides a link to their own implementation, but it does not specify versions for general software dependencies (e.g., programming languages, libraries, frameworks) required for reproducibility. |
| Experiment Setup | Yes | In each trial, we generate θl N(0, σ0) for either σ0 = 1 (Figure 1) or σ0 = 2 (Figure 3). In all experiments, the prior distribution used in MMLE is set to be the standard normal distribution. The validation dataset is used by MMLE to select the best user parameter distribution Dθ and by Spectral-EM to select the number of mixture components. We set 2011-01 to 2018-01 as the training period where we find the model with the appropriate hyper-parameters that maximizes the PNL within the training period. |