Test-Time Model Adaptation with Only Forward Passes
Authors: Shuaicheng Niu, Chunyan Miao, Guohao Chen, Pengcheng Wu, Peilin Zhao
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on four benchmarks and full precision/quantized models verify our effectiveness. 4. Experiments |
| Researcher Affiliation | Collaboration | 1College of Computing and Data Science, Nanyang Technological University, Singapore 2Joint NTU-We Bank Research Centre on Fintech, Singapore 3Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Singapore 4Tencent AI Lab, Shenzhen, China. |
| Pseudocode | Yes | Algorithm 1 Forward-Optimization Adaptation (FOA). |
| Open Source Code | Yes | Code: https://github.com/mr-eggplant/FOA. |
| Open Datasets | Yes | The models are trained on the source Image Net-1K training set and the model weights are obtained from the timm repository (Wightman, 2019). |
| Dataset Splits | Yes | We use the validation set of Image Net-1K to estimate source ID statistics. |
| Hardware Specification | Yes | The Wall-Clock Time (seconds) and Memory Usage (MB) are measured for processing 50,000 images of Image Net-C on a single RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions 'timm repository (Wightman, 2019)' and 'PTQ4Vi T (Yuan et al., 2022)' as software used, and also 'pycma' (https://github.com/CMA-ES/pycma), but does not provide specific version numbers for Python, PyTorch, or the libraries themselves (e.g., timm vX.Y.Z). |
| Experiment Setup | Yes | We set the number of prompt embeddings Np to 3 and initialize prompts with uniform initialization. We set the batch size (BS) to 64 by following TENT and SAR for fair comparisons. The population size K is set to 28 = 4 + 3 log(prompt dim) by following (Hansen, 2016) and λ in Eqn. (5) is set to 0.4 BS/64 on Image Net-C/V2/Sketch, and 0.2 BS/64 on Image Net-R to balance the magnitude of two losses. We use the validation set of Image Net-1K to estimate source ID statistics. The step size γ in Eqn. (7) is set to 1.0, aiming to exactly align the overall center of testing and training features. |