Exploratory Machine Learning with Unknown Unknowns
Authors: Peng Zhao, Yu-Jie Zhang, Zhi-Hua Zhou10999-11006
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Theoretical analysis and empirical study on both synthetic and real datasets validate the efficacy of our proposal. We present empirical evaluations on synthetic data to illustrate the idea and further validate the effectiveness on real datasets. |
| Researcher Affiliation | Academia | Peng Zhao, Yu-Jie Zhang, Zhi-Hua Zhou National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China {zhaop, zhangyj, zhouzh}@lamda.nju.edu.cn |
| Pseudocode | No | Algorithm details are presented in the full version (Zhao, Zhang, and Zhou 2021). The main paper does not contain pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | Yes | We further evaluate on a UCI benchmark dataset Mfeat (van Breukelen et al. 1998) and Real Disp2, which is an activities recognition task (Ba nos et al. 2012). http://archive.ics.uci.edu/ml/datasets/REALDISP+Activity+Recognition+Dataset |
| Dataset Splits | Yes | The threshold for the initial rejection model is selected by cross validation to ensure 95% accuracy on high-confidence samples. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, memory amounts) used for running its experiments, only discussing the methods and datasets. |
| Software Dependencies | No | The paper describes the mathematical components and parameters used (e.g., Gaussian kernel, regularization parameters), but does not provide specific software dependencies like programming languages, libraries, or solvers with version numbers. |
| Experiment Setup | Yes | The threshold θ is choose as one achieving best accuracy on the testing data from the pool [0.1, 0.2, 0.3, 0.4]. The budget ratio is b = 20%. The threshold for the initial rejection model is selected by cross validation to ensure 95% accuracy on high-confidence samples. |