Active Test-Time Adaptation: Theoretical Analyses and An Algorithm
Authors: Shurui Gui, Xiner Li, Shuiwang Ji
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results confirm consistency with our theoretical analyses and show that the proposed ATTA method yields substantial performance improvements over TTA methods while maintaining efficiency and shares similar effectiveness to the more demanding active domain adaptation (ADA) methods. |
| Researcher Affiliation | Academia | Shurui Gui Texas A&M University College Station, TX 77843 shurui.gui@tamu.edu Xiner Li* Texas A&M University College Station, TX 77843 lxe@tamu.edu Shuiwang Ji Texas A&M University College Station, TX 77843 sji@tamu.edu |
| Pseudocode | Yes | Algorithm 1 SIMATTA: A SIMPLE ATTA ALGORITHM |
| Open Source Code | Yes | Our code is available at https://github.com/divelab/ATTA. |
| Open Datasets | Yes | We employ PACS (Li et al., 2017), VLCS (Fang et al., 2013), Office-Home (Venkateswara et al., 2017), and Tiny-Image Net-C datasets for our evaluations. |
| Dataset Splits | No | The paper specifies how source and test domains are chosen (e.g., 'For PACS and Office-Home, we use domains "photos" and "real" as the source domains'), and describes data stream orders, but does not provide explicit numerical training, validation, and test dataset splits with percentages or counts for reproducibility of data partitioning in the traditional sense. |
| Hardware Specification | Yes | The Ubuntu server includes 112 Intel(R) Xeon(R) Gold 6258R CPU @2.70GHz, 1.47TB memory, and NVIDIA A100 80GB PCIe graphics cards. |
| Software Dependencies | No | The paper mentions using 'Py Torch (Paszke et al., 2019)' and 'scikit-learn (Pedregosa et al., 2011)' but does not provide specific version numbers for these libraries to ensure reproducible software dependencies. |
| Experiment Setup | Yes | Source domain pre-training. For the PACS and VLCS datasets, models are fine-tuned on the selected source domains for 3,000 iterations. The Adam optimizer is utilized with a learning rate of 10 4. In contrast, for the Office-Home dataset, the model is fine-tuned for a longer duration of 10,000 iterations with a slightly adjusted learning rate of 5 10 5. |