Generative-Based Fusion Mechanism for Multi-Modal Tracking
Authors: Zhangyong Tang, Tianyang Xu, Xiaojun Wu, Xue-Feng Zhu, Josef Kittler
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Based on this, we conduct extensive experiments across two multi-modal tracking tasks, three baseline methods, and four challenging benchmarks. The experimental results demonstrate that the proposed generative-based fusion mechanism achieves state-of-the-art performance by setting new records on GTOT, Las He R and RGBD1K. |
| Researcher Affiliation | Academia | 1School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, PR. China 2Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XH, UK |
| Pseudocode | No | The paper includes figures illustrating the models but does not provide any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code will be available at https://github.com/Zhangyong-Tang/GMMT. |
| Open Datasets | Yes | Our GMMT is trained on the training split of Las He R with the parameters optimised by the SGD optimiser. The effectiveness of GMMT is verified on GTOT (Li et al. 2016), Las He R (Li et al. 2022), and RGBD1K (Zhu et al. 2023b) benchmarks. |
| Dataset Splits | No | The paper mentions training on a 'training split' but does not specify a separate validation split or its size/purpose. |
| Hardware Specification | Yes | Our experiments are conducted on an NVIDIA RTX3090Ti GPU card. |
| Software Dependencies | No | The paper mentions optimizers (SGD) and network architectures (UNet, UVi T) but does not provide specific version numbers for software dependencies or libraries. |
| Experiment Setup | Yes | Our GMMT is trained on the training split of Las He R with the parameters optimised by the SGD optimiser. The learning rate is warmed up from 0.001 to 0.005 in the first 20 epochs and subsequently reduces to 0.00005 for the remaining 80 epochs. We set the value of T to 1000. |