Learning Robust Global Representations by Penalizing Local Predictive Power
Authors: Haohan Wang, Songwei Ge, Zachary Lipton, Eric P. Xing
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In extensive experiments across a wide spectrum of synthetic and real data sets, our method outperforms the competing ones, especially when the domain information is not available. To evaluate cross-domain transfer, we introduce Image Net-Sketch, a new dataset consisting of sketch-like images and matching the Image Net classification validation set in categories and scale. In this section, we test PAR over a variety of settings, we first test with perturbed MNIST under the domain generalization setting, and then test with perturbed CIFAR10 under domain adaptation setting. Further, we test on more challenging data sets, with PACS data under domain generalization setting and our newly proposed Image Net-Sketch data set. |
| Researcher Affiliation | Academia | Haohan Wang, Songwei Ge, Eric P. Xing, Zachary C. Lipton School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 {haohanw,songweig,epxing,zlipton}@cs.cmu.edu |
| Pseudocode | No | The paper does not include pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Clean demonstration of the implementation can be found at: https://github.com/HaohanWang/PAR |
| Open Datasets | Yes | We follow the set-up of Wang et al. (2019) in experimenting with MNIST data set with different superficial patterns. We continue to experiment on CIFAR10 data set by modifying the color and texture of test dataset with four different schemas. We test on the PACS data set (Li et al., 2017a). We construct the Image Net-Sketch data set for evaluating the out-of-domain classification performance of vision models trained on Image Net. The Image Net-Sketch data can be found at: https://github.com/HaohanWang/ImageNet-Sketch |
| Dataset Splits | Yes | The training/validation samples are attached with two of these patterns, while the testing samples are attached with the remaining one. (Section 4.1, MNIST with Perturbation) |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments. |
| Software Dependencies | No | The paper mentions models like AlexNet and ResNet-32 but does not specify any software dependencies with version numbers (e.g., PyTorch 1.x, TensorFlow 2.x, CUDA 10.x). |
| Experiment Setup | Yes | We use the same model architecture and learning rate as in Wang et al. (2019). The extra hyperparameter λ is set as 1 as the most straightforward choice. Methods in Wang et al. (2019) are trained for 100 epochs, so we train the model for 50 epochs as pretraining and 50 epochs with our regularization. In this experiment, we use Res Net-32 as our base classifier. As for PAR, we first train the base classifier for 250 epochs and then train with the adversarial loss for another 150 epochs. Following Li et al. (2017a), we use Alex Net as baseline and build PAR upon it. Following the training heuristics we introduced, we continue with trained Alex Net weights and fine-tune on training domain data of PACS for 100 epochs. We use Alex Net as the baseline and test whether our method can help improve the out-of-domain prediction. We start with Image Net pretrained Alex Net and continue to use PAR to tune Alex Net for another five epochs on the original Image Net training dataset. |