Sparse-to-dense Multimodal Image Registration via Multi-Task Learning
Authors: Kaining Zhang, Jiayi Ma
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on MSCOCO, Google Earth, VIS-NIR and VIS-IR-drone datasets demonstrate that our method achieves remarkable performance on multimodal cases. |
| Researcher Affiliation | Academia | 1Electronic Information School, Wuhan University, Wuhan, China. Correspondence to: Jiayi Ma <jyma2010@gmail.com>. |
| Pseudocode | No | The paper describes algorithms using mathematical equations and text but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https: //github.com/KN-Zhang/SDME. |
| Open Datasets | Yes | We conduct experiments on four datasets, including MSCOCO (Lin et al., 2014), Google Earth (Zhao et al., 2021), VIS-NIR (Brown & S usstrunk, 2011) and VIS-IR-drone (Sun et al., 2022). |
| Dataset Splits | No | The paper specifies training and test splits for datasets (e.g., '7:3 ratio for training and test data generation') but does not explicitly mention a separate validation split or cross-validation setup. |
| Hardware Specification | Yes | All the experiments are conducted on a single TITAN RTX. |
| Software Dependencies | No | The paper mentions using 'Adam W' for training but does not provide specific version numbers for software libraries or frameworks like PyTorch, TensorFlow, or Python itself. |
| Experiment Setup | Yes | We train with Adam W with an initial learning rate of 1e 4, a weight decay of 5e 4, and a batch size of 8 image pairs. We train 100 epochs and use Cosine Annealing to schedule the learning rate. In the sparse branch, we calculate LAP by sampling positive samples within a radius of 3 pixels and negative ones between a radius of 5 and 7 pixels. The patch size is set to 16 to calculate Lcosim and Lpeaky. The total loss in this branch is set to LAP + 5Lcosim + Lpeaky + 0.008Lguide, while that in the dense branch is Lconv1 + Lconv2 + Lmc. |