Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Policy Optimized Text-to-Image Pipeline Design
Authors: Uri Gadot, Rinon Gal, Yftah Ziser, Gal Chechik, Shie Mannor
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our approach through a set of comparisons, showing that it can successfully create new flows with greater diversity and lead to superior image quality compared to existing baselines. 4 Experiments We follow [16] and compare our approach to a set of baselines across two main metrics: (1) The Gen Eval [18] benchmark which measures prompt-adherence by using object detection and classification modules to evaluate correct object generation, placement, and attribute binding. (2) Human preference, using the Civit AI prompt-set of Comfy Gen [16]. For the latter, we evaluate our approach using both an automated preference metric (HPS v2, [59]) as well as a user study. |
| Researcher Affiliation | Collaboration | 1Technion 2NVIDIA Research 3 University of Groningen |
| Pseudocode | No | The paper describes its methodology in Section 3, but it does not present any explicitly labeled 'Pseudocode' or 'Algorithm' blocks, nor structured steps formatted like code. |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The paper relies on data from a prior paper. The prior data is not public but can be requested from the authors. |
| Open Datasets | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The paper relies on data from a prior paper. The prior data is not public but can be requested from the authors. |
| Dataset Splits | Yes | We keep the 500 prompts used to test Comfy Gen as a holdout, and train using the rest. |
| Hardware Specification | No | Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification We provide an explanation about the reproducibility of our experiments in the supplementary. (Note: The main paper text does not contain specific hardware details, referring to supplementary material which is not provided for this analysis.) |
| Software Dependencies | No | The paper mentions 'Llama tokenizer' and cites 'Unsloth: Fast, memory-efficient llm fine-tuning library. https://github.com/unslothai/unsloth, 2024. Version 2.0. Accessed: 2025-05-20.' in the references, but does not explicitly state within the main text that these specific software components with their versions were used for their implementation. |
| Experiment Setup | Yes | Unless otherwise noted, we use ϖ = 1.5. |