Adapting to Online Label Shift with Provable Guarantees
Authors: Yong Bai, Yu-Jie Zhang, Peng Zhao, Masashi Sugiyama, Zhi-Hua Zhou
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments are conducted to validate the effectiveness and support our theoretical findings. ... In this section, we conduct extensive experiments to validate the effectiveness of the proposed methods (ATLAS and ATLAS-ADA) and justify the theoretical findings. |
| Researcher Affiliation | Academia | Yong Bai1 , Yu-Jie Zhang2,1 , Peng Zhao1, Masashi Sugiyama3,2, Zhi-Hua Zhou1 1 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 2 The University of Tokyo, Chiba, Japan 3 RIKEN AIP, Tokyo, Japan |
| Pseudocode | Yes | Algorithm 1 ATLAS: base-algorithm; Algorithm 2 ATLAS: meta-algorithm; Algorithm 4 ATLAS-ADA: base-algorithm; Algorithm 5 ATLAS-ADA: meta-algorithm |
| Open Source Code | No | The main body of the paper does not contain an explicit statement of code release or a link to a code repository. While the separate 'Checklist' section indicates code is included, this is not part of the primary content of the paper. |
| Open Datasets | Yes | We conduct experiments on real-world data, including six benchmark datasets (Ar Xiv, Euro SAT, MNIST, Fashion, CIFAR10, and CINIC10) and the SHL dataset [39] for the real-life locomotion detection task. |
| Dataset Splits | No | The paper mentions 'offline initialization stage' with labeled samples (S0) and 'online adaptation stage' with unlabeled data (St) at each round, but it does not specify explicit percentages or counts for training, validation, and test splits within these datasets, nor does it explicitly define a separate validation set. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions implementing models (e.g., 'logistic regression model') but does not specify any particular software libraries, frameworks, or their version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | The contenders include... our proposals (UOGD, ATLAS, and ATLAS-ADA) with the logistic regression model... We repeat all experiments for five times and evaluate the contenders by the average error... over T = 10,000 rounds... UOGD... with step size η... Set the step size pool as H = {ηi = Γσ 2G KT 2i 1 | i [N]}, where N = 2 log2(1 + 2T) is the number of base-learners. ATLAS ensures that E[Regd T ]... learning rate ε = Θ( p (ln N)/T)... when Nt = 100 samples are received at every iteration... The online sample size is set as Nt = 10. |