Prompt-Based Distribution Alignment for Unsupervised Domain Adaptation
Authors: Shuanghao Bai, Min Zhang, Wanqi Zhou, Siteng Huang, Zhirong Luan, Donglin Wang, Badong Chen
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | we first experimentally demonstrate that the unsupervised-trained VLMs can significantly reduce the distribution discrepancy between source and target domains, thereby improving the performance of UDA. However, a major challenge for directly deploying such models on downstream UDA tasks is prompt engineering, which requires aligning the domain knowledge of source and target domains, since the performance of UDA is severely influenced by a good domain-invariant representation. We further propose a Prompt-based Distribution Alignment (PDA) method to incorporate the domain knowledge into prompt learning. Specifically, PDA employs a two-branch prompttuning paradigm, namely base branch and alignment branch. The base branch focuses on integrating class-related representation into prompts, ensuring discrimination among different classes. To further minimize domain discrepancy, for the alignment branch, we construct feature banks for both the source and target domains and propose image-guided feature tuning (IFT) to make the input attend to feature banks, which effectively integrates self-enhanced and crossdomain features into the model. In this way, these two branches can be mutually promoted to enhance the adaptation of VLMs for UDA. We conduct extensive experiments on three benchmarks to demonstrate that our proposed PDA achieves state-of-the-art performance. |
| Researcher Affiliation | Academia | 1Institute of Artificial Intelligence and Robotics, Xi an Jiaotong University, Xi an, China 2Westlake University Institute of Advanced Technology, Westlake Institute for Advanced Study 3School of Electrical Engineering, Xi an University of Technology, Xi an, China 4RIKEN AIP |
| Pseudocode | No | The paper describes the method using equations and textual descriptions, but it does not include a formal pseudocode block or algorithm listing. |
| Open Source Code | Yes | The code is available at https://github.com/Bai Shuanghao/Prompt-based Distribution-Alignment. |
| Open Datasets | Yes | Experiments are conducted on popular benchmark datasets of unsupervised domain adaptation, namely Office Home (Venkateswara et al. 2017), Office-31 (Saenko et al. 2010) and Vis DA-2017 (Peng et al. 2018). |
| Dataset Splits | No | The paper describes the use of labeled source domain and unlabeled target domain data, and pseudo-labeling for the target domain, but it does not specify explicit train/validation/test dataset splits with percentages or sample counts. |
| Hardware Specification | No | The paper mentions the use of ResNet50, ResNet101, and ViT-B/16 as backbones, but it does not provide any specific details about the hardware (e.g., GPU models, CPU types) used for running the experiments. |
| Software Dependencies | No | The paper refers to specific models (e.g., CLIP, ViT-B/16, ResNet50) and optimizers (SGD), but it does not provide version numbers for any software dependencies like programming languages or libraries (e.g., Python, PyTorch, CUDA). |
| Experiment Setup | Yes | The parameters in the encoders of CLIP are fixed, and we train the prompt and IFT module using the SGD optimizer for 10 epochs on the Office-Home and Vis DA-2017 datasets, and for 20 epochs on the Office-31 dataset, with a batch size of 32. For all prompt tuning methods, we set the learning rate initially to around 0.003 initially and decay it using a cosine annealing rule. Moreover, the context tokens length is set to 2 for Ma PLe and our PDA method, 10 for VPT and VP, and 16 for Co Op and Co Co Op. |