ELSA: Efficient Label Shift Adaptation through the Lens of Semiparametric Models
Authors: Qinglong Tian, Xin Zhang, Jiwei Zhao
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct thorough numerical experiments to vali- date the performance of our proposed ELSA method. The ELSA method outperforms the well-known methods: BBSE, RLLS, and MLLS. Furthermore, the ELSA method has competitive performance and is more computationally efficient than the calibrated MLLS method, which is the best method in the literature (see Alexandari et al. 2020). In this section, we conduct numerical experiments to demonstrate the efficacy of our proposed method on the label shift problem. We summarize our experimental settings in the following. Datasets and Models. Our experiments are evaluated on MNIST, CIFAR-10, and CIFAR-100. |
| Researcher Affiliation | Collaboration | 1Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, ON, Canada 2Meta Inc., Menlo Park, CA, USA 3Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI, USA. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating the public availability of the source code for the described methodology. |
| Open Datasets | Yes | Our experiments are evaluated on MNIST, CIFAR-10, and CIFAR-100. |
| Dataset Splits | No | The paper mentions reserving '10k data samples of the training set... as the source dataset' and uses 'labeled source data' and 'unlabeled target data' but does not explicitly provide training/validation/test dataset splits or percentages. |
| Hardware Specification | Yes | The experiments are conducted on a Mac Book Pro with a 2.9 GHz Dual-Core Intel Core i5 processor and 8 GB memory. |
| Software Dependencies | No | The paper mentions common machine learning frameworks like neural networks and specific methods (e.g., BBSE, RLLS, MLLS, temperature scaling), but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | Our experiments are evaluated on MNIST, CIFAR-10, and CIFAR-100. We adopt the same settings in (Alexandari et al., 2020): For each dataset, ten models are trained with different random seeds, and 10k data samples of the training set are reserved as the source dataset (so that it is not used for training the model.). Dirichlet label shift is adopted as our label shift mechanism. More specifically, we use a Dirichlet distribution with concentration parameter α to generate the label distribution p(y) for the target dataset. |