Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness
Authors: Zhuoqun Huang, Neil Marchant, Olga Ohrimenko, Benjamin I.P. Rubinstein
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate the effectiveness of Adapt Del and Adapt Del+, we conduct experiments on a diverse set of natural language processing tasks. These include five-class sentiment analysis on the Yelp dataset [22], spam detection using the Spam Assassin dataset [30, 31], sentiment analysis on the IMDB dataset [32], and unreliable news detection using the LUN dataset [33, 31]. The varying input sizes of these datasets allow for a comprehensive assessment of Adapt Del s overall effectiveness. We also analyze performance by dividing inputs into quartiles based on their length. Detailed specifications of these datasets are provided in Table 1 in Appendix D. The appendices provide further certified results (Appendix E), details on computational efficiency (Appendix F) and a supplementary analysis of empirical robustness against common text attacks (Appendix G). |
| Researcher Affiliation | Academia | Zhuoqun Huang University of Melbourne EMAIL Neil G. Marchant University of Melbourne EMAIL Olga Ohrimenko University of Melbourne EMAIL Benjamin I. P. Rubinstein University of Melbourne EMAIL |
| Pseudocode | Yes | Algorithm 1 CERTIFY Algorithm 2 CERTIFYGENERAL Algorithm 3 CREATEDYNAMICEQUALWIDTHBINS Algorithm 4 MAXCERTRADIUS Algorithm 5 OPTIMIZEEXPECTEDLENGTH |
| Open Source Code | Yes | Our experiments are based on publicly available datasets, and we provide the code and data as supplementary material. The instructions for reproducing the results are included in the README file. |
| Open Datasets | Yes | To evaluate the effectiveness of Adapt Del and Adapt Del+, we conduct experiments on a diverse set of natural language processing tasks. These include five-class sentiment analysis on the Yelp dataset [22], spam detection using the Spam Assassin dataset [30, 31], sentiment analysis on the IMDB dataset [32], and unreliable news detection using the LUN dataset [33, 31]. We collect all data from Hugging Face Datasets1 and the Adv Bench repository2 [31]. |
| Dataset Splits | Yes | Table 1: Summary of datasets. Avg. Words denotes the average number of words per instance in the dataset. The full test set is used for evaluation for all datasets except for Yelp, where we sample 10 000 instances from the available 50 000 instances. Dataset Avg. Words Train Valid Test Yelp 134.1 585 000 65 000 10 000 Spam Assassin 228.2 2 152 239 2 378 IMDB 231.2 22 500 2 500 25 000 LUN 269.9 13 416 1 490 6 454 |
| Hardware Specification | Yes | All experiments in this paper are conducted using a private cluster with Intel(R) Xeon(R) Gold 6326 CPU at 2.90GHz and NVIDIA A100 GPUs. |
| Software Dependencies | No | Parameter Values Base model Model Auto Model For Sequence Classification("roberta-base") Tokenizer Auto Tokenizer("roberta-base") Scheduler Python command transformers.get_linear_schedule_with_warmup Warmup epochs 10 Python class torch.optim.Adam W Learning rate 2.0E-5 Weight decay 1.0E-6 Gradient clipping clip_grad_norm_(model.parameters(), 1.0) |
| Experiment Setup | Yes | Table 2: Parameter settings for Ro BERTa, the optimizer and training procedure. Parameter settings are consistent across all models (No Smooth, CERT-ED, Ran MASK, Adapt Del, Adapt Del+) except where specified. Learning rate 2.0E-5 Weight decay 1.0E-6 Gradient clipping clip_grad_norm_(model.parameters(), 1.0) Batch size 32 Max. epoch 200 Early stopping No improvement in validation loss after 25 epochs |