Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Understanding the Power and Limitations of Teaching with Imperfect Knowledge
Authors: Rati Devidze, Farnam Mansouri, Luis Haug, Yuxin Chen, Adish Singla
IJCAI 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experimental Evaluation In this section, we perform empirical studies to validate the guarantees provided by our theorems, and to showcase that the data regularity assumptions we made in the previous section are satisfied in real-world problem settings. ... All the results corresponding to four different notions of imperfect teacher are shown in Figure 2, averaged over 10 runs. |
| Researcher Affiliation | Academia | 1Max Planck Institute for Software Systems (MPI-SWS) 2ETH Zurich 3University of Chicago EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | For the reproducibility of experimental results and facilitating research in this area, the code and dataset are publicly available. |
| Open Datasets | Yes | For the reproducibility of experimental results and facilitating research in this area, the code and dataset are publicly available. |
| Dataset Splits | No | The paper mentions results are 'averaged over 10 runs' but does not specify details for training, validation, or test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper does not specify any software names with version numbers for its dependencies. |
| Experiment Setup | Yes | We consider Q0 to be uniform distribution over H, η = 0.5, and have desired ϵ = 0.001. |