Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Authors: Yusuke Iwasawa, Yutaka Matsuo
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We tested our method on four domain generalization benchmarks, namely PACS, VLCS, Office Home, and Terra Incognita, along with various backbone networks including Res Net18, Res Net50, Big Transfer (Bi T), Vision Transformers (Vi T), and MLP-Mixer. The results show T3A stably improves performance on unseen domains across choices of backbone networks, and outperforms existing domain generalization methods. |
| Researcher Affiliation | Academia | Yusuke Iwasawa The University of Tokyo iwasawa@weblab.t.u-tokyo.ac.jp Yutaka Matsuo The University of Tokyo matsuo@weblab.t.u-tokyo.ac.jp |
| Pseudocode | Yes | Algorithm 1 Algorithm of T3A for prediction. |
| Open Source Code | Yes | Our implementation uses the Domain Bed library [17]1. We modify Domain Bed (1) to use various backbone networks using the timm library [54]2, and (2) to implement test-time adaptation algorithms (ours and Tent). For Tent, we used the original implementation3. (...) [Yes] See supplemental materials. |
| Open Datasets | Yes | We evaluate T3A on four standard domain generalization benchmarks, namely VLCS [12], PACS [27], Office Home [50], and Terra Incognita [5]. |
| Dataset Splits | Yes | For benchmarking the algorithm, given a dataset containing nd domains, we usually use the leaveone-domain-out procedure, which uses a single domain as a test domain and the others as training domains. The procedure is repeated nd times, changing the test domain every time. (...) We used standard training-domain validation for selecting hyperparameters, which uses the subset of each training domain to choose a model. As reported in [17], we split the data from each domain into 80% and 20% splits and use larger splits for training and smaller splits to select hyperparameters. |
| Hardware Specification | Yes | We run our experiments mainly on cloud V100x4 or A100x8 instances, depending on the memory usage of the backbone networks. |
| Software Dependencies | No | Our implementation uses the Domain Bed library [17]1. We modify Domain Bed (1) to use various backbone networks using the timm library [54]2, and (2) to implement test-time adaptation algorithms (ours and Tent). |
| Experiment Setup | Yes | We used the implementation of Domain Bed [17], and used the default hyper-parameters for pre-training on source-domains and fine-tuning on a target domain: namely, we use Adam [23] with a learning rate of 5e-5 for optimization and use a batch size of 32 with no dropout or weight decay. |