Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Principled Model Routing for Unknown Mixtures of Source Domains
Authors: Christoph Dann, Yishay Mansour, Teodor Vanislavov Marinov, Mehryar Mohri
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our primary contributions are theoretical and algorithmic, but we also present empirical results demonstrating the effectiveness of our approach. |
| Researcher Affiliation | Collaboration | Christoph Dann Google Research Yishay Mansour Google Research Tel Aviv University Teodor V. Marinov Google Research Mehryar Mohri Google Research Courant Institute |
| Pseudocode | Yes | Algorithm 1: Domain adaptation for model routing algorithm |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: We are unable to provide the source code but our experiments only rely on public models and datasets and we aimed to provide all details necessary to reproduce our results. |
| Open Datasets | Yes | We conduct our evaluation on the Mix Instruct benchmark by Jiang et al. [2023] which consists of 5 individual domains. |
| Dataset Splits | No | The paper mentions training for 10,000 batches of 256 samples from each domain, and that datasets are shuffled between instances, but it does not provide explicit training/validation/test splits for the Mix Instruct benchmark or custom splits. It refers to 'samples from each domain' for training but not how the benchmark itself is partitioned. |
| Hardware Specification | Yes | We conducted our experiment on a cluster with 256 v5e TPUs. |
| Software Dependencies | No | The paper mentions using a 'pre-trained Gemma 2B model' and 'standard gradient-based optimizers' but does not specify version numbers for any software libraries (e.g., Python, PyTorch, TensorFlow) or specific optimizers. |
| Experiment Setup | Yes | The routing function f is initialized as a pre-trained Gemma 2B model [Team et al., 2024], with the final layer replaced by a fully connected, randomly initialized linear layer to produce the logits of f. When we use Algorithm 1 with Option A, we use a learning rate γ = 1e-4 and with Option B a learning rate γ = 1e-3 to update the domain weights. We train each routing function for 10,000 batches of 256 samples from each domain. Each experiment is repeated 5 times where the datasets are shuffled between instances of the same experiment. |