Max-Margin Majority Voting for Learning from Crowds
Authors: TIAN TIAN, Jun Zhu
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now present experimental results to demonstrate the strong discriminative ability of max-margin majority voting and the promise of our Bayesian models, by comparing with various strong competitors on multiple real datasets. |
| Researcher Affiliation | Academia | Tian Tian, Jun Zhu Department of Computer Science & Technology; Center for Bio-Inspired Computing Research Tsinghua National Lab for Information Science & Technology State Key Lab of Intelligent Technology & Systems; Tsinghua University, Beijing 100084, China tiant13@mails.tsinghua.edu.cn; dcszj@tsinghua.edu.cn |
| Pseudocode | Yes | Algorithm 1: The Crowd SVM algorithm |
| Open Source Code | No | The paper does not contain an unambiguous statement or link indicating that the source code for the methodology described in this paper is publicly available. |
| Open Datasets | Yes | We use four real world crowd labeling datasets as summarized in Table 1. Web Search [24]: ... Age [8]: ... Bluebirds [19]: ... Flowers [18]: |
| Dataset Splits | No | The paper states: 'we cannot simply split the training data into multiple folds to cross-validate the hyperparameters' and instead uses likelihood on the given dataset for model selection, without specifying any train/validation/test splits. |
| Hardware Specification | Yes | All experiments were conducted on a PC with Intel Core i5 3.00GHz CPU and 12.00GB RAM. |
| Software Dependencies | No | The paper mentions using 'well-developed SVM solvers like LIBSVM [2]' but does not provide specific version numbers for LIBSVM or any other software dependencies. |
| Experiment Setup | Yes | For Crowd SVM, we set α = 1 and v = 1 for all experiments, since we find that the results are insensitive to them. For M3V, Crowd SVM and Gibbs-Crowd SVM, the regularization parameters (c, ℓ) are selected from c = 2ˆ[ 8 : 0] and ℓ= [1, 3, 5] by the method in Sec. 6.2. As for Gibbs-Crowd SVM, we generate 50 samples in each run and discard the first 10 samples as burn-in steps |