Cross-Lingual Adversarial Domain Adaptation for Novice Programming
Authors: Ye Mao, Farzaneh Khoshnevisan, Thomas Price, Tiffany Barnes, Min Chi7682-7690
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The effectiveness of Cross Ling is evaluated on two tasks: student program classification and student suc- cess early predictions for D1. For the former, our results show that Cross Ling indeed outperforms the state-of-the-art methods such as ASTNN and Code2Vec and other baselines. Our results show that TL-Cross Ling performs significantly better than other student modeling methods, including those using the gold standard: expert-designed features. |
| Researcher Affiliation | Collaboration | Ye Mao1, Farzaneh Khoshnevisan2, Thomas Price1, Tiffany Barnes1, Min Chi1 1 North Carolina State University 2 Intuit Inc. |
| Pseudocode | No | The paper describes algorithms and steps in text and equations but does not include formal pseudocode blocks or algorithms. |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-sourcing of the code for its methodology. |
| Open Datasets | No | The paper uses the i Snap (D1) and Code Workout (D2) datasets. While Code Workout is stated as an "open system" with a URL, it does not explicitly state that the *specific dataset* of 795 submissions is publicly available for download or provide a direct link/citation for *that dataset*. i Snap is a custom collected dataset from NCSU with no public link or citation provided for its direct access. |
| Dataset Splits | Yes | It is important to emphasize that all models are evaluated using semester-based temporal cross-validation (3-fold) in this task, which only applied data from previous semesters for training and is a much stricter approach than the standard cross-validation (Mao et al. 2020). |
| Hardware Specification | No | The paper does not specify any hardware details such as GPU models, CPU types, or memory used for experiments. |
| Software Dependencies | No | The paper mentions 'implemented using the sklearn2 library in python' and 'all the other models are implemented in Pytorch3', but does not provide specific version numbers for these software dependencies. The footnotes '2https://scikit-learn.org/' and '3https://pytorch.org/' only provide general links to the libraries. |
| Experiment Setup | Yes | During training, we take the hyperparameters that achieve the best performance on the development set via a small grid search over combinations of the batch size {16, 32, 64}, learning rate [0.001, 0.1], α, β, γ [0.01, 1]. The same experimental setup is used for all models with 100 epochs and early stopping. The embedding size for all ASTNNs/Cross Lings is 128 and LSTM/T-LSTM hidden size is set to 64. |