Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Incentivizing Peer Grading in MOOCS: An Audit Game Approach
Authors: Alejandro Uriel Carbonara, Anupam Datta, Arunesh Sinha, Yair Zick
IJCAI 2015 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We present the ο¬rst model of strategic auditing in peer grading, modeling the student s choice of effort in response to a grader s audit levels as a Stackelberg game with multiple followers. We demonstrate that computing the equilibrium for this game is computationally hard. We then provide a PTAS in order to compute an approximate solution to the problem of allocating audit levels. |
| Researcher Affiliation | Academia | Alejandro Carbonara and Anupam Datta Carnegie-Mellon University auc,EMAIL Arunesh Sinha University of Southern California EMAIL Yair Zick Carnegie-Mellon University EMAIL |
| Pseudocode | Yes | Algorithm 1: Solving Fixed Precision K Data: b, ktot, T = {1, 2, . . . n}, ft(k) Result: xt(ktot) t,k : xt(k) = ; for j = 1; j n; j++ do for k = 0; k ktot; k = k + 2 b do if j = 1 then xj(k) = max(xj(0), xj(k 2 b), fj(k)); else for l = 0; l 1; l = l + 2 b do xj(k) = max(xj 1(k l) + fj(l), xj(k)); |
| Open Source Code | No | The paper does not contain any statements offering access to source code for the described methodology, nor does it provide any links to a code repository. |
| Open Datasets | No | The paper introduces a theoretical model and algorithms for incentivizing peer grading but does not report on experiments conducted using a specific dataset, thus no access information for a dataset is provided. |
| Dataset Splits | No | This paper is theoretical and does not conduct empirical experiments with datasets, therefore, it does not provide information on training, validation, or test dataset splits. |
| Hardware Specification | No | The paper is theoretical and does not describe any experiments or computations that would require specific hardware, thus no hardware specifications are provided. |
| Software Dependencies | No | The paper is theoretical, presenting models and algorithms, but does not mention any specific software dependencies with version numbers required for replication. |
| Experiment Setup | No | The paper is theoretical and focuses on mathematical modeling and algorithm design; it does not describe an experimental setup with specific hyperparameters or training configurations. |