Improving the Accuracy of Learning Example Weights for Imbalance Classification
Authors: Yuqi Liu, Bin Cao, Jing Fan
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we perform extensive experiments to validate the effectiveness of our method. First, we describe the experimental setup in detail. Second, we compare different methods in two domains: text and image classification and in two situations: binary classification and multi-class classification. Third, we design experiments to study the performance of our method in different imbalance ratios. Moreover, we evaluate the performance of our methods with different metrics on a large-scale data set in Appendix A.1. |
| Researcher Affiliation | Academia | Yuqi Liu & Bin Cao & Jing Fan College of Computer Science, Zhejiang University of Technology, Hangzhou, China {liuyuqi,bincao,fanjing}@zjut.edu.cn |
| Pseudocode | Yes | Algorithm 1: Learning to Weight Examples Using a Combination Method |
| Open Source Code | No | The paper only provides a link to the code for a comparison method (Hu et al.) and does not state that the code for their own methodology is open-source or provided. |
| Open Datasets | Yes | We use the SST-2 sentiment analysis benchmark (Socher et al., 2013) for binary classification, and use the SST-5 sentence sentiment (Socher et al., 2013) with 5 categories for multi-class classification. In image classification, we adopt the commonly-used CIFAR10 (Schneider et al., 2019) for multi-class classification experiment and select the examples of class 0 and 1 from CIFAR100 (Schneider et al., 2019) to form a data set for binary classification. APS Failure dataset is from UCI Machine Learning Repository (Dua & Graff, 2017). |
| Dataset Splits | Yes | We divide this training process into 2 stages, and we take an imbalanced training set and a small balanced validation set from the remaining training examples (not including the examples used for model preparation). ... For all data sets, the number of examples in the validation set is 10 for each class. The validation and test sets are balanced data sets, and their example sizes for each class are 5 and 1000 respectively. |
| Hardware Specification | Yes | All experiments were implemented with Python 3.8 and Py Torch 1.8 and were evaluated on a Linux server with RTX 3080 GPU and 128GB RAM. |
| Software Dependencies | Yes | All experiments were implemented with Python 3.8 and Py Torch 1.8 |
| Experiment Setup | Yes | The settings of the training process on the four data sets are listed in Table 3. Each cell in the table indicates the settings in the current stage, including the optimizer used, learning rate, number of epochs, and batch size... For our method, we set the learning rate and epochs for updating the weights during learning with constraint, and they are taken from {1e-2, 1e-3, 1e-4, 1e-5} and {1, 5, 10, 15, 30} respectively. |