Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Guiding Cross-Modal Representations with MLLM Priors via Preference Alignment
Authors: Pengfei Zhao, Rongbo Luan, Wei Zhang, Peng Wu, Sifeng He
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our preference-guided alignment achieves substantial gains in fine-grained cross-modal retrieval, underscoring its effectiveness in handling nuanced semantic distinctions. We validate our framework through comprehensive experiments on different benchmarks including general retrieval (e.g., COCO [12], Flickr30K [13]), fine-grained retrieval (e.g. Winoground [14], Natural Bench [15], MMVP [16], Bi VLC [17]). The experimental results demonstrate the superiority and effectiveness of our approach in both general and fine-grained retrieval tasks. |
| Researcher Affiliation | Industry | Pengfei Zhao, Rongbo Luan, Wei Zhang, Peng Wu, Sifeng He Apple EMAIL |
| Pseudocode | No | The paper describes the methodology using natural language and mathematical formulations, and includes a training schema in Figure 2, but does not present a distinct pseudocode block or algorithm. |
| Open Source Code | No | We plan to release our training code and the complete data curation pipeline as open-source upon acceptance of the paper. However, at the current submission stage, we are unable to share any code. |
| Open Datasets | Yes | We train MAPLE on a curated subset of the Open Image dataset [26] (details in Appendix B.1). We evaluate its performance on standard general (MS-COCO [12], Flickr30K [13]) and fine-grained (Winoground [14], Natural Bench [15], MMVP [16], Bi VLC [17]) retrieval benchmarks. |
| Dataset Splits | No | We train MAPLE on a curated subset of the Open Image dataset [26] (details in Appendix B.1). The dataset was divided into 8 partitions, and we sampled only one partition (approximately 700K instances) for our training set. The paper specifies evaluation protocols for fine-grained datasets, but does not provide explicit training/validation/test splits with percentages or counts for the Open Image dataset used for training, beyond sampling a partition for training. |
| Hardware Specification | Yes | Training was conducted on a cluster of 32 NVIDIA A100 GPUs (80GB memory each). |
| Software Dependencies | No | We use Lo RA for fine-tuning MAPLE MLLM-based embedding models. We employed bfloat16 mixed-precision training, gradient checkpointing for the LLM component, and Flash Attention [38]. The paper mentions several techniques and tools but does not provide specific version numbers for software dependencies like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | Key model-related hyperparameters are summarized in Table 7. Table 7: Key Hyperparameters for MAPLE Training. Parameter Value Policy Model Base Qwen2-VL-2B / Qwen2-VL-7B Lo RA Rank (r) 32 Lo RA Alpha (α) 32 Optimizer Adam W Base Learning Rate (Lo RA) 5e-4 LR Schedule Linear Warmup + Cosine Decay Warmup Ratio 0.025 (of total steps) Batch Size (per GPU) 96 (for 2B model) / 48 (for 7B model) Total Epochs 8 Max Image Resolution 384x384 Initial τ (learnable) 0.07 Initial β (learnable) 1/0.07 14.29 |