MACK: Multimodal Aligned Conceptual Knowledge for Unpaired Image-text Matching
Authors: Yan Huang, Yuming Wang, Yunan Zeng, Liang Wang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To demonstrate the effectiveness of the proposed method, we perform extensive experiments of image-text matching on two publicly available datasets. |
| Researcher Affiliation | Academia | Yan Huang Yuming Wang Yunan Zeng Liang Wang Center for Research on Intelligent Perception and Computing (CRIPAC), State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences (CASIA) School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS) Chinese Academy of Sciences Artificial Intelligence Research (CAS-AIR) |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks (clearly labeled algorithm sections or code-like formatted procedures). |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] In the supplemental material. |
| Open Datasets | Yes | 1) Flickr30k [42] consists of 31783 images collected from the Flickr website. ... 2) MSCOCO [24] consists of 123287 images, each of which is associated with 5 texts. |
| Dataset Splits | Yes | We use the public training, validation and testing splits, which contain 29000, 1000 and 1000 images, respectively. ... We use the public training, validation and testing splits, with 113287, 1000 and 5000 images, respectively. |
| Hardware Specification | No | The knowledge-based unpaired image-text matching is very efficient, which does not need GPUs for acceleration. (Section 4.2) This only states what is not needed, not what is used. No specific hardware (GPU/CPU model, type) is mentioned. |
| Software Dependencies | No | No specific version numbers are provided for software components like NLTK or the deep learning framework used for Adam optimization. |
| Experiment Setup | Yes | We use the Adam algorithm [15] to optimize the only parameter matrix W with a learning rate of 2e 4 for 30 epochs. When re-ranking existing models, we empirically set k = 15 and λ = 0.1. |