Active Learning with Disagreement Graphs
Authors: Corinna Cortes, Giulia Desalvo, Mehryar Mohri, Ningshan Zhang, Claudio Gentile
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report experimental results on multiple datasets and demonstrate that the proposed algorithms achieve better test performances than IWAL given the same amount of labeling budget. In this section, we report the results of several experiments comparing IWAL, IWAL-D and IZOOM. We experimented with these algorithms in 8 binary classification datasets from the UCI repository. |
| Researcher Affiliation | Collaboration | 1Google Research, New York, NY, USA; 2Courant Institute, New York, NY, USA; 3Leonard N. Stern School of Business, New York University, New York, NY, USA. |
| Pseudocode | Yes | Algorithm 1 IWAL-D(H), Algorithm 2 IZOOM(H), Algorithm 3 RESAMPLE(H, n) |
| Open Source Code | No | The paper does not provide an explicit statement about releasing the source code for the described methodology, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We experimented with these algorithms in 8 binary classification datasets from the UCI repository. Table 1 summarizes the relevant statistics for these datasets. |
| Dataset Splits | Yes | For each experiment, we randomly shuffled the dataset and ran the algorithms on the first 50% of the data, and tested the learned classifier on the remaining 50% of the data. |
| Hardware Specification | No | The paper does not specify the exact hardware (e.g., CPU, GPU models, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | For all algorithms, we randomly drew 3,000 hyperplanes with bounded norms as our base hypothesis set H. In addition, for the IZOOM algorithm we kept |H| = 3,000 throughout the learning. For every pair of h, h0 2 H, we approximated L(h, h0) with the average disagreement values on 2,000 unlabeled samples. We used the standard logistic loss function, which is defined for all (x, y) 2 X Y and hypothesis h by log(1 + e yh(x)), which we then rescaled to [0, 1]. |