Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model
Authors: Runheng Liu, Heyan Huang, Xingchen Xiao, Zhijing Wu
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate IRM on the Detect RL benchmark and demonstrate that IRM can achieve superior detection performance, outperforms existing zero-shot and supervised methods in LLM-generated text detection. [...] We conduct evaluations on a large benchmark, Detect RL [13], which covers various domains, multiple LLMs and diverse attacks scenarios in real-world settings. |
| Researcher Affiliation | Academia | Runheng Liu, Heyan Huang, Xingchen Xiao, Zhijing Wu School of Computer Science and Technology, Beijing Institute of Technology EMAIL |
| Pseudocode | Yes | Algorithm 1 Inference pipeline of IRM |
| Open Source Code | Yes | Answer: [Yes] Justification: We provide codes and data with instruction files in supplemental materials. |
| Open Datasets | Yes | Datasets. We conduct evaluations on a large benchmark, Detect RL [13], which covers various domains, multiple LLMs and diverse attacks scenarios in real-world settings. [...] Table 7: Dataset and model details. Name Used Full Name Author Source Detect RL Detect RL [13] Git Hub |
| Dataset Splits | Yes | We follow the test setting of Detect RL in the evaluation of detection methods. [...] For the train sub-task, we use the optimal threshold derived from the dataset with a length interval of 160-180 and apply it to datasets with other length intervals. Conversely, for the test sub-task, the optimal threshold of each dataset is evaluated on the dataset with a 160-180 length interval. |
| Hardware Specification | Yes | All experiments are conducted on two NVIDIA RTX 4090 GPUs (24GB each). |
| Software Dependencies | No | The paper does not explicitly provide specific software dependencies with version numbers for the experiments in Section 4.1 or other parts of the main text. |
| Experiment Setup | Yes | Datasets. We conduct evaluations on a large benchmark, Detect RL [13]... Models. We employ Gemma-2 [17], Llama-3.2, Gemma [18], Qwen-2 [19], and Qwen-2.5 [20] model families for implementation... Metrics. Follow the settings of Detect RL, we report AUROC and F1 Score as the main evaluation metrics. [...] For the train sub-task, we use the optimal threshold derived from the dataset with a length interval of 160-180 and apply it to datasets with other length intervals. |