Latent Confusion Analysis by Normalized Gamma Construction
Authors: Issei Sato, Hisashi Kashima, Hiroshi Nakagawa
ICML 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically analyzed the proposed model in this section. Since our problem setting was unsupervised, i.e., the true labels and confusion matrices were not available, it was difficult to evaluate the models. Therefore, we use datasets in which the correct answers (labels or scores) were known. Here, we call a gold label a correct label that is actually known in the datasets. We only use a gold label to evaluate an estimated label that has a maximum probability of q(τm) for each model, i.e., τ m = argmaxτm q(τm). MV indicates majority voting. DS indicates the Dawid and Skene model, and GLAD/m GLAD (multi-label variant of GLAD described in Sec. 3). LCA is our model described in Sec. 4.2. |
| Researcher Affiliation | Academia | Issei Sato SATO@R.DL.ITC.U-TOKYO.AC.JP The University of Tokyo Hisashi Kashima KASHIMA@I.KYOTO-U.AC.JP Kyoto University Hiroshi Nakagawa N3@DL.ITC.U-TOKYO.AC.JP The University of Tokyo |
| Pseudocode | No | The paper describes the proposed model and its variational Bayes inference, but it does not contain a formally structured pseudocode or algorithm block. |
| Open Source Code | No | The paper does not include an unambiguous statement that the authors are releasing the source code for the methodology described, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | We created datasets (1) and (2) by using crowdsourcing and published the datasets2. 2http://www.r.dl.itc.u-tokyo.ac.jp/ sato/icml2014/. We used a dataset called bluebird, published by Welinder et al. (2010). |
| Dataset Splits | No | The paper does not specify exact training, validation, and test dataset splits by percentages, absolute counts, or by referencing predefined splits with citations. It describes evaluation using 'gold labels' but not data partitioning. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments, such as GPU/CPU models, memory, or cloud resources. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software components, libraries, or solvers used in the experiments. |
| Experiment Setup | No | The paper mentions some initialization steps like 'We initialized q(τm) with an empirical distribution by using worker voting' and 'We initialized γa = γc = γd = 1', but it does not provide comprehensive experimental setup details such as learning rates, batch sizes, number of epochs, or optimizer settings typically found in a dedicated 'Experimental Setup' section for reproducibility. |