Using Discourse Signals for Robust Instructor Intervention Prediction
Authors: Muthu Kumar Chandrasekaran, Carrie Epp, Min-Yen Kan, Diane Litman
AAAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our supervised classifier makes use of an automatic discourse parser... In comprehensive experiments over 14 MOOC offerings from several disciplines, the PDTB discourse features improve performance on average. The resultant models are less dependent on domain-specific vocabulary, allowing them to better generalize to new courses. |
| Researcher Affiliation | Academia | Muthu Kumar Chandrasekaran,1 Carrie Demmans Epp,2,3 Min-Yen Kan,1,4 Diane Litman2,5 1 Department of Computer Science, School of Computing, National University of Singapore 2 Learning Research and Development Center, University of Pittsburgh 3 University Center for Teaching and Learning, University of Pittsburgh 4 Interactive and Digital Media Institute, National University of Singapore, Singapore 5 Department of Computer Science, University of Pittsburgh |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The implementations of the EDM 15 and the discourse based systems are publicly available6. https://github.com/WING-NUS/lib4moocdata |
| Open Datasets | No | The paper mentions using data from 14 Coursera MOOC offerings and thanks the University of Pittsburgh for sharing, but does not provide a public link, DOI, or specific citation for the processed dataset used in their experiments. |
| Dataset Splits | Yes | The in-domain setting models were trained and evaluated separately on each MOOC using stratified five-fold cross validation. Stratification accounts for the highly imbalanced data and ensures that each fold had both positive and negative samples. ... In the out-of-domain setting, we use leave-one-out cross-coursevalidation (LOO-CCV) where models trained on 13 courses are tested on the 14th unseen course. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions using 'the Java version of the parser5' but does not specify the Java version or the parser's specific version number. It also does not list other software dependencies with version numbers. |
| Experiment Setup | Yes | The baseline system uses a maximum entropy classifier with the following set of features: unigrams, thread forum type, student affirmations to a previous post, thread properties (# of posts, comments, and posts+comments), average # of comments per post, # of sentences in the thread, # of URLs, and # of timestamped references to parts of a lecture video. ... This motivated the use of class weights to counterbalance the # of non-intervened instances. Class weights, an important parameter of this model, were estimated as the ratio of negative to positive samples in the training instances. |