TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices
Authors: Hong Jia, Young Kwon, Alessio Orsino, Ting Dang, DOMENICO TALIA, Cecilia Mascolo
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted extensive experiments to assess Tiny TTA performance using four benchmark corruption datasets on two edge devices: the Microprocessor (MPU) Raspberry Pi Zero 2W with 512 MB DRAM and STM32H747 MCU with 512 KB of SRAM. Experimental results demonstrate that Tiny TTA improves TTA accuracy by up to 57.6%, reduces memory usage by up to six times, and achieves faster and more energy-efficient TTA. |
| Researcher Affiliation | Collaboration | 1University of Cambridge 2University of Calabria 3University of Melbourne 4Samsung AI Center, Cambridge |
| Pseudocode | No | No pseudocode or clearly labeled algorithm block found. |
| Open Source Code | Yes | The Tiny TTA Engine code is publicly available at https://github.com/h-jia/TTE. |
| Open Datasets | Yes | For comparison with baselines, our experiments were conducted using four popular corrupted datasets, including domain shift datasets CIFAR10C [23] and CIFAR100C [24], and two domain generalization datasets, namely Office Home [25] and PACS [26]. |
| Dataset Splits | No | The paper does not explicitly provide specific percentages, sample counts, or citations to predefined train/validation/test splits for the datasets used. |
| Hardware Specification | Yes | We validate Tiny TTA on a Raspberry Pi Zero 2W and an STM32H747 MCU. ... Raspberry Pi Zero 2W with 512 MB DRAM and STM32H747 MCU with 512 KB of SRAM. ... The first MPU was a Raspberry Pi Zero 2W featuring an ARM Cortex-A53 processor with distinct memory specifications. The second MCU we used is the STM32H747XI, which possesses a dual-core (ARM Cortex M4 and M7) architecture. |
| Software Dependencies | Yes | The pre-training stage of Tiny TTA, executed on a standard Linux server, utilized Py Torch version 1.10.2. For the meta-learner module, Python, Num Py, and Tensor Flow Lite Micro (TFLM) were employed due to their compatibility and efficient integration. |
| Experiment Setup | Yes | The evaluation of Tiny TTA on MPUs... For the MPU experiments, we employed three SOTA mobile models, specifically Efficient Net [27], Mobile Net V2 [28], and Reg Net [29]... For the MCU evaluation, we utilized the SOTA model MCUNet [5]... The hyperparameters setting for Tiny TTA is as follows: a learning rate of 1 × 10−5, a batch size of 1, the SGD optimizer with a momentum of 0.9, and a self-ensemble of early exit layers at depths [3, 5, 7]. |