Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Sound and Complete Neural Network Repair with Minimality and Locality Guarantees
Authors: Feisi Fu, Wenchao Li
ICLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this Section, we compare REASSURE with state-of-the-art methods on both point-wise repairs and area repairs. The experiments were designed to answer the following questions: (Effectiveness) How effective is a repair in removing known buggy behaviors? (Locality) How much side effect (i.e. modification outside the patch area in the function space) does a repair produce? (Function Change) How much does a repair change the original neural network in the function space? (Performance) Whether and how much does a repair adversely affect the overall performance of the neural network? |
| Researcher Affiliation | Academia | Feisi Fu Division of System Engineering Boston University EMAIL Wenchao Li Department of Electrical and Computer Engineering Boston University EMAIL |
| Pseudocode | Yes | Algorithm 1 REASSURE Input: A specification Φ = (Φin, Φout), a Re LU DNN f and a set of buggy points {ex1, . . . , ex L} Φin. Output: A repaired Re LU DNN bf. |
| Open Source Code | No | The paper mentions external code (e.g., 'MDNN Github repository', 'Sotoudeh & Thakur (2021) does not include a vertex enumeration tool... in their code'), but it does not provide a statement or link for the authors' own source code for the methodology described in the paper. |
| Open Datasets | Yes | We train a Re LU DNN on the MNIST dataset Le Cun (1998) as the target DNN. |
| Dataset Splits | No | The paper mentions using training, validation, and test data (e.g., 'ND(L1), ND(L2): average (L1, L2) norm difference on validation data' in Table 4), but it does not explicitly specify the percentages or exact counts for the dataset splits used (e.g., '80/10/10 split' or 'X training samples, Y validation samples'). |
| Hardware Specification | Yes | All experiments were run on an Intel Core i5 @ 3.4 GHz with 32 GB of memory. |
| Software Dependencies | Yes | We use Gurobi Gurobi Optimization, LLC (2021) to solve the linear programs. We use pycddlib Troffaes (2018) to perform the vertex enumeration step when evaluating PRDNN. |
| Experiment Setup | Yes | Hyperparameters used in Repair: We set γ = 0.5 for Point-wise Repair on MNIST, γ = 0.02 for Watermark Removal, γ = 1 for Area Repair: HCAS and γ = 0.0005 for Point-wise Repair on Image Net. We set learning rate to 10-3 for Retrain in the point-wise repair experiment. We set learning rate to 10-2 and momentum to 0.9 for Fine-Tuning in the point-wise repair experiment. |