Rethinking Importance Weighting for Deep Learning under Distribution Shift
Authors: Tongtong Fang, Nan Lu, Gang Niu, Masashi Sugiyama
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments with two representative types of DS on three popular datasets show that our dynamic IW compares favorably with state-of-the-art methods. |
| Researcher Affiliation | Academia | Tongtong Fang 1 Nan Lu 1,2 Gang Niu2 Masashi Sugiyama2,1 1The University of Tokyo, Japan 2RIKEN, Japan |
| Pseudocode | Yes | Algorithm 1 Dynamic importance weighting (in a mini-batch). |
| Open Source Code | Yes | Our implementation of DIW is available at https://github.com/Tongtong FANG/DIW. |
| Open Datasets | Yes | The experiments are based on three widely used benchmark datasets Fashion-MNIST [57], CIFAR-10 and CIFAR-100 [27]. |
| Dataset Splits | Yes | 1,000 random clean data in total are used in the label-noise experiments; 10 random data per class are used in the class-prior-shift experiments. The validation data are included in the training data, as required by Reweight. |
| Hardware Specification | No | The paper mentions training models (e.g., 'LeNet-5 is trained by SGD', 'ResNet-32 is trained by Adam') but does not specify any hardware details such as specific GPU or CPU models used for the experiments. |
| Software Dependencies | No | The paper mentions 'All baselines are implemented with PyTorch' and refers to optimizers like SGD and Adam, but it does not provide specific version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | for Fashion-MNIST, LeNet-5 [28] is trained by SGD [42]; for CIFAR-10/100, ResNet-32 [18] is trained by Adam [26]. For fair comparisons, we normalize W to make 1/ntr Pntr i=1 wi = 1 hold within each mini-batch. For clear comparisons, there is no data augmentation. More details can be found in the appendices. |