Deep Learning From Crowdsourced Labels: Coupled Cross-Entropy Minimization, Identifiability, and Regularization
Authors: Shahana Ibrahim, Tri Nguyen, Xiao Fu
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A series of synthetic and real data experiments are presented to showcase the effectiveness of our approach. and Table 1 presents the average label prediction accuracy on the testing data of the MNIST and the Fashion-MNIST over 5 random trials, for various cases. |
| Researcher Affiliation | Academia | Shahana Ibrahim, Tri Nguyen, and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University Corvallis, OR 97330, USA |
| Pseudocode | Yes | The proposed methods Geo Crowd Net(F) and Geo Crowd Net(W) are described in Algorithm 1 and 2, respectively. |
| Open Source Code | Yes | The Python implementation of the algorithms is available in Git Hub1. 1https://github.com/shahanaibrahimosu/end-to-end-crowdsourcing |
| Open Datasets | Yes | We use the MNIST dataset (Deng, 2012) and Fashion-MNIST dataset (Xiao et al., 2017); see more details in Sec. N. and We consider the CIFAR-10 (Krizhevsky, 2009) dataset for synthetic noisy label experiments. and We consider two different datasets, namely, Label Me (M = 59) (Rodrigues et al., 2017; Russell et al., 2007) and Music (M = 44) (Rodrigues et al., 2014) |
| Dataset Splits | Yes | We use 45, 000 images for training, 5, 000 images for validation, and 10, 000 images for testing. and We consider 57, 000 images as training data, 3000 images for validation, and 10, 000 images for testing. and The validation set consists of 500 images and the remaining 1188 images are used for testing. and Out of the annotated 700 samples, we consider 595 samples for training, and the remaining 105 samples are used for validation. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory specifications, or cloud computing instance types used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'Adam' as an optimizer and 'Python implementation' for the algorithms, but does not specify versions for these or any other software libraries or dependencies (e.g., PyTorch, TensorFlow, scikit-learn versions). |
| Experiment Setup | Yes | Adam (Kingma & Ba, 2015) is used an optimizer with weight decay of 10 4 and batch size of 128. The regularization parameter λ and the initial learning rate of the Adam optimizer are chosen via grid search method using the validation set from {0.01, 0.001, 0.0001} and {0.01, 0.001}, respectively. and The confusion matrices are initialized with identity matrices of size K for proposed methods and the baselines Trace Reg and Crowd Layer. |