Distributionally Robust Post-hoc Classifiers under Prior Shifts
Authors: Jiaheng Wei, Harikrishna Narasimhan, Ehsan Amid, Wen-Sheng Chu, Yang Liu, Abhishek Kumar
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we empirically demonstrate the effectiveness of our proposed method DROPS, for the tasks of class-imbalanced learning and group distributional robustness. |
| Researcher Affiliation | Collaboration | Jiaheng Wei UC Santa Cruz Harikrishna Narasimhan Google Research Ehsan Amid Google Research Wen-Sheng Chu Google Research Yang Liu UC Santa Cruz Abhishek Kumar Google Research Work done during an internship at Google Research, Brain Team. |
| Pseudocode | No | The paper describes algorithmic steps in prose (Section 4.3, 'Step 1: updating λ(t).', 'Step 2: updating g(t).', 'Step 3: scaling the predictions.') but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | An empirical implementation is available at https://github.com/weijiaheng/Drops. |
| Open Datasets | Yes | For CIFAR-10, and CIFAR-100 datasets, we down-sample the number of samples for each class to simulate the class-imbalance as done in earlier works (Cui et al., 2019; Cao et al., 2019). |
| Dataset Splits | Yes | A balanced held-out validation set is utilized for hyper-parameter tuning. All baseline models are picked by referring to the δ = 1.0-worst case performances on the validation set, which is made up of the last 10% of the original CIFAR training dataset. |
| Hardware Specification | No | The paper mentions using a specific model architecture ('Pre Act Res Net 18') but does not provide any details about the hardware (e.g., GPU models, CPU, or memory) used for the experiments. |
| Software Dependencies | No | The paper mentions using 'SGD optimizer' and specific learning rate schedules, but does not provide version numbers for any software libraries, frameworks (e.g., TensorFlow, PyTorch), or programming languages used. |
| Experiment Setup | Yes | All methods are trained with the same architecture (Pre Act Res Net 18 (He et al., 2016)) with 5 random seeds, same data augmentation techniques, the same SGD optimizer with a momentum of 0.9 with Nesterov acceleration. All methods share the same initial learning rate of 0.1 and a piece-wise constant learning rate schedule of [10 2, 10 3, 10 4] at [30, 80, 110] epochs, respectively. We use a batch size of 128 for all methods and train the model for 140 epochs. |