ODS: Test-Time Adaptation in the Presence of Open-World Data Shift
Authors: Zhi Zhou, Lan-Zhe Guo, Lin-Han Jia, Dingchu Zhang, Yu-Feng Li
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on multiple benchmarks with different types of shifts, and the results demonstrate the superior performance of our method against the state of the arts. |
| Researcher Affiliation | Academia | 1National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China. Correspondence to: Yu-Feng Li <liyf@nju.edu.cn>. |
| Pseudocode | Yes | The overall algorithm of MT is shown in Algorithm 1. |
| Open Source Code | No | The paper refers to official code for *compared methods* in footnotes (e.g., '1https://github.com/Dequan Wang/tent', '2https://github.com/mr-eggplant/EATA'), but it does not state that *their own* ODS code is open-source or provide a link for it. |
| Open Datasets | Yes | We conduct experiments on two standard TTA benchmarks: CIFAR10-C and CIFAR100-C (Hendrycks & Dietterich, 2019). For experiments on the CIFAR dataset, We train the source model on the clean CIFAR10/CIFAR100 dataset, which has 50,000 32x32 training images associated with 10/100 classes. |
| Dataset Splits | Yes | We train the source model on the clean CIFAR10/CIFAR100 dataset, which has 50,000 32x32 training images associated with 10/100 classes. Then, we test each TTA method on the CIFAR10-C/CIFAR100-C dataset... We report mean std accuracy over five runs with random seed setting to 0, 1, 2, 3, 4. |
| Hardware Specification | Yes | All experiments are repeatedly conducted with one NVIDIA Ge Force RTX 3090 GPU with a random seed setting from 0 to 4. |
| Software Dependencies | No | The paper mentions using the 'Huawei Mind Spore platform for experimental testing partially' but does not specify a version number for it. It also mentions the 'Adam Optimizer' and 'SGD optimizer' but these are general algorithms, not specific software packages with version numbers. While it references the use of other methods' official code, it does not specify the versions of the libraries or dependencies used for its own implementation. |
| Experiment Setup | Yes | We train the source model with batch size 256 for 200 epochs. The SGD optimizer optimizes each model with a learning rate of 0.1 using a cosine annealing schedule. For test-time adaptation, we set the batch size to 64 following previous studies (Wang et al., 2022a; Niu et al., 2022). We report mean std accuracy over five runs with random seed setting to 0, 1, 2, 3, 4. |