Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Scalable Fingerprinting of Large Language Models
Authors: Anshul Nasery, Jonathan Hayase, Creston Brooks, Peiyao Sheng, Himanshu Tyagi, Pramod Viswanath, Sewoong Oh
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experiment with fingerprint design at a scale significantly larger than previously considered, and introduce a new method, dubbed Perinucleus sampling, to generate scalable, persistent, and harmless fingerprints. We demonstrate that this scheme can add 24,576 fingerprints to a Llama-3.1-8B model two orders of magnitude more than existing schemes without degrading the model s utility. |
| Researcher Affiliation | Collaboration | Anshul Nasery Jonathan Hayase Creston Brooks Peiyao Sheng Himanshu Tyagi Pramod Viswanath Sewoong Oh University of Washington Sentient |
| Pseudocode | Yes | C Pseudocode Algorithm 1 Perinucleus Sampling Input: Base model θm and vocabulary V, Model for keys θk threshold t [0, 1], width k Z+, length L of response Output: Sampled fingerprint (xfp, yfp) |
| Open Source Code | No | Our code is available here. (in abstract and intro, but no link provided). And in the NeurIPS checklist, point 5: "We will release the code upon publication." This indicates it is not yet available. |
| Open Datasets | Yes | To assess Persistence, we first perform SFT on the fingerprinted model using the Alpaca [38] dataset for instruction tuning. We then prompt the model with the fingerprint keys and verify whether the highest-probability output token matches the corresponding fingerprint response. Persistence is measured as the fraction of correctly recalled fingerprints out of the total fingerprints inserted. |
| Dataset Splits | Yes | To assess Persistence, we first perform SFT on the fingerprinted model using the Alpaca [38] dataset for instruction tuning. We perform two epochs of fine-tuning with a learning rate of 10-5. We use the Llama-Factory [71] framework for this. ... To measure the Harmlessness of fingerprints, we report evaluation scores on Open LLM [19], a standard benchmark which consists of six datasets (MMLU [32], Truthful QA [33], GSM8K [34], Winogrande [35], Hellaswag [36], ARC-C [37]). |
| Hardware Specification | Yes | We report the number of epochs needed for convergence, as well as an estimate of the wall-clock time on our setup of 4 L40 GPUs below. |
| Software Dependencies | No | We use Adam to optimize the cross entropy loss, training with full-batch gradient descent for upto 40 epochs, and early stop when the train loss drops below 0.005. ... We use the Llama-Factory [71] framework for this. Specific version numbers for key software components are not provided. |
| Experiment Setup | Yes | We use Adam to optimize the cross entropy loss, training with full-batch gradient descent for upto 40 epochs, and early stop when the train loss drops below 0.005. ... We perform two epochs of fine-tuning with a learning rate of 10-5. ... We hence use t = 0.8 in our experiments. ... use with λWA = 0.75 and βDM = 0.25 in our main experiments |