Scalable Semi-Supervised Aggregation of Classifiers
Authors: Akshay Balsubramani, Yoav Freund
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present and empirically evaluate an efficient algorithm that learns to aggregate the predictions of an ensemble of binary classifiers. The algorithm uses the structure of the ensemble predictions on unlabeled data to yield significant performance improvements. It does this without making assumptions on the structure or origin of the ensemble, without parameters, and as scalably as linear learning. We empirically demonstrate these performance gains with random forests. ... Then we perform an exploratory evaluation of the framework on data in Section 4. |
| Researcher Affiliation | Academia | Akshay Balsubramani UC San Diego abalsubr@cs.ucsd.edu Yoav Freund UC San Diego yfreund@cs.ucsd.edu |
| Pseudocode | Yes | Algorithm 1 HEDGECLIPPER Input: Labeled set L, unlabeled set U 1: Using L, grow trees T = {T1, . . . , Tp} (regularized; see Sec. 3.2) 2: Using L, estimate b S on T and its leaves 3: Using U, (approximately) optimize (7) to estimate σ S Output: The estimated weighting σ S, for use at test time Figure 1: At left is algorithm HEDGECLIPPER. |
| Open Source Code | No | The paper states, 'Our implementation uses minibatch SGD to optimize (6), runs in Python on top of the popular open-source learning package scikit-learn, and runs out-of-core (n-independent memory), taking advantage of the scalability of our formulation.' This indicates use of an open-source package (scikit-learn) but does not provide concrete access or an explicit statement about the release of their own HEDGECLIPPER code. |
| Open Datasets | Yes | The datasets are drawn from UCI/Lib SVM as well as data mining sites like Kaggle, and no further preprocessing was done on the data. ... [23] Predicting a Biological Response. 2012. https://www.kaggle.com/c/bioresponse. [24] Give Me Some Credit. 2011. https://www.kaggle.com/c/Give Me Some Credit. |
| Dataset Splits | No | The paper mentions restricting training data and combining unused labeled examples with test examples and unlabeled sets, and that 'Results are averaged over 10 runs, each drawing a different random subsample of labeled data.' However, it does not provide specific percentages or counts for train/validation/test splits, nor does it specify a clear, reproducible method for data partitioning across all datasets (e.g., specific random seeds, k-fold cross-validation details, or predefined split citations). |
| Hardware Specification | No | The paper states, 'Our implementation uses minibatch SGD to optimize (6), runs in Python on top of the popular open-source learning package scikit-learn, and runs out-of-core (n-independent memory)...' However, it does not provide specific hardware details such as CPU/GPU models, memory amounts, or other detailed computer specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions 'runs in Python on top of the popular open-source learning package scikit-learn,' but it does not specify version numbers for either Python or scikit-learn, which are necessary for reproducible software dependencies. |
| Experiment Setup | No | The paper states, 'Our implementation opts for another approach to grow trees constrained to have a nontrivial minimum weight per leaf.' and discusses how 'Changing α alters the aggressiveness of the clipping'. While these mention specific aspects of their method, they do not provide concrete hyperparameter values (e.g., specific numerical values for learning rates, batch sizes, number of trees, or the minimum leaf weight) or a detailed system-level training configuration in the main text needed to fully reproduce the experiments. It mentions 'Further information and discussion on the protocol is in the appendix,' implying full details are not in the main text. |