A General Framework for Learning from Weak Supervision
Authors: Hao Chen, Jindong Wang, Lei Feng, Xiang Li, Yidong Wang, Xing Xie, Masashi Sugiyama, Rita Singh, Bhiksha Raj
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | GLWS not only enhances the scalability of machine learning models but also demonstrates superior performance and versatility across 11 weak supervision scenarios. We conduct the evaluation mainly on CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011), and Image Net-100 (Russakovsky et al., 2015). |
| Researcher Affiliation | Collaboration | 1Carnegie Mellon University 2Microsoft Research 3William & Mary 4Singapore University of Technology and Design 5Peking University 6RIKEN AIP 7The University of Tokyo 8Mohamed bin Zayed University of AI. |
| Pseudocode | Yes | We present the pseudo-algorithm of performing the forward-backward algorithms on common weak supervision settings we evaluated. The pseudo-algorithm also corresponds to description of the trellis expanded from the NFA. ... Algorithm 1 Forward-Backward Algorithm for multiple instance (Multi Ins) Learning ... Algorithm 2 Forward-Backward Algorithm for label proportion (LProp) Learning ... Algorithm 3 Forward-Backward Algorithm for pairwise comparison (PComp) Learning |
| Open Source Code | Yes | Code is available at: https://github.com/Hhhhhhao/ General-Framework-Weak-Supervision. |
| Open Datasets | Yes | We conduct the evaluation mainly on CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011), and Image Net-100 (Russakovsky et al., 2015). Results on MNIST (Deng, 2012) and F-MNIST (Xiao et al., 2017) are included in the Appendix, where most of the baseline methods were evaluated. |
| Dataset Splits | Yes | Table 6. Dataset details Dataset # Classes # Training # Validation # Unlabeled (...) CIFAR-10 10 50,000 10,000 |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used to run experiments, such as GPU models, CPU types, or cloud computing specifications. |
| Software Dependencies | No | The paper mentions software components like 'Adam W' and 'SGD' as optimizers, and models like 'CLIP', but does not provide specific version numbers for Python, PyTorch, or other libraries and frameworks used for implementation. |
| Experiment Setup | Yes | We follow the hyper-parameters from Wu et al. (2022) for training all methods, with more details provided in Appendix C.2.1. A summarize of training parameters is shown in Table 7. Here we present the training hyper-parameters we used for Multi Ins and LProp in Table 9. For pairwise observations (x1), x2), we adopt the same training parameters for the four settings we evaluated, as shown in Table 11. The hyper-parameters are shown in Table 16. |