A Universal Unbiased Method for Classification from Aggregate Observations
Authors: Zixi Wei, Lei Feng, Bo Han, Tongliang Liu, Gang Niu, Xiaofeng Zhu, Heng Tao Shen
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct extensive experiments to empirically demonstrate the effectiveness of our proposed method in various problems of CFAO. For classification via pairwise similarity/triplet comparison/label proportion, we use five popular large-scale benchmark datasets... Extensive experiments on various problems of CFAO demonstrate the superiority of our proposed method. |
| Researcher Affiliation | Academia | 1College of Computer Science, Chongqing University, China 2School of Computer Science and Engineering, Nanyang Technological University, Singapore 3RIKEN Center for Advanced Intelligence Project, Japan 4Department of Computer Science, Hong Kong Baptist University, China 5Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates 6Sydney AI Centre, School of Computer Science, The University of Sydney, Australia 7School of Computer Science and Engineering, University of Electronic Science and Technology of China, China. |
| Pseudocode | Yes | Algorithm 1 RC Algorithm Input: Model f, epoch Tmax, iteration Imax, size of label space k, whether to use log-likelihood to initialize flaginit, log-likelihood initialize epoch Tinit, whether to use confidence matrix flagmat, aggregate observation training set D tpxpiq 1:m, zpiqqun i 1; |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | We use five popular large-scale benchmark datasets including MNIST (Le Cun et al., 1998), Kuzushiji-MNIST (Clanuwat et al., 2018), Fashion-MNIST (Xiao et al., 2017), SVHN (Netzer et al., 2011), and CIFAR-10 (Krizhevsky et al., 2009) and five regular-scale datasets from the UCI Machine Learning Repository (Dua & Graff, 2017) including usps, pendidigts, optdigits, msplice, and vehicle. For multiple-instance learning, we use five common benchmark datasets in this area (Dietterich et al., 1997; Andrews et al., 2002), inclduing Musk1, Musk2, Elephant, Fox, and Tiger. |
| Dataset Splits | Yes | We randomly split the given datasets into training, validation, and test sets by 60%, 20% and 20% for each trial. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory used for running the experiments. It mentions using models like '5-layer Le Net' or '22-layer Dense Net' but no hardware specifications. |
| Software Dependencies | No | The paper mentions 'Adam (Kingma & Ba, 2015) optimizer' but does not specify version numbers for any key software components, libraries, or frameworks used in the experiments (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We used Adam (Kingma & Ba, 2015) optimizer with 0 weight decay to train the model. The learning rates were 1e-3, 1e-3 and 2e-1 for benchmark datasets, UCI datasets and MIL datasets respectively. The batch size is 128 for benchmark datasets and UCI datasets. We search the batch size from (128, 256,512,1024,2048,4096) for MIL datasets. The model is trained for 100, 200, and 3500 epochs for benchmark datasets, UCI datasets and MIL datasets respectively. |