Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Understanding and Improving Fast Adversarial Training against $l_0$ Bounded Perturbations
Authors: Xuyang Zhong, Yixiao Huang, Chen Liu
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate our method can overcome the challenge of CO, achieve state-of-the-art performance, and narrow the performance gap between 1-step and multi-step adversarial training against sparse attacks. Codes are available at https://github.com/City U-MLO/s PGD. |
| Researcher Affiliation | Academia | Xuyang Zhong Department of Computer Science City University of Hong Kong EMAIL Huang Department of Computer Science City University of Hong Kong EMAIL Liu Department of Computer Science City University of Hong Kong EMAIL |
| Pseudocode | Yes | A Algorithm Details Algorithm 1 Self-Adaptive Training (SAT) [40] Algorithm 2 TRADES [30] Algorithm 3 Fast-LS-l0 |
| Open Source Code | Yes | Codes are available at https://github.com/City U-MLO/s PGD. |
| Open Datasets | Yes | We conduct extensive experiments on various datasets. The results on CIFAR-10 and Image Net-100 [51] are demonstrated in Table 5. More results on CIFAR-100 [25] and GTSRB [52] are in Table 8 and 9 of Appendix F.4, respectively. |
| Dataset Splits | Yes | The models are Preact Res Net-18 [24] trained on CIFAR-10 [25]. The results on CIFAR-10 and Image Net-100 [51] are demonstrated in Table 5. More results on CIFAR-100 [25] and GTSRB [52] are in Table 8 and 9 of Appendix F.4, respectively. In implementation, we can stabilize and improve the performance of fast adversarial training against l0 bounded perturbations by combining soft labels and trade-off loss function. In addition, several available techniques, such as self-adaptive training (SAT) [40] and TRADES [30], can be considered variations of soft labels and trade-off loss function. In Appendix A, we provide the pseudo-codes of both SAT and TRADES and the formulation of their combination as a reference. It should be highlighted that the rationale for using soft labels and trade-off loss function is different for the l0 case. Although they are widely leveraged to address robust overfitting to boost performance in l2 and l cases, smoothing the loss function is essential to address the CO issue in the l0 case. 5 Experiments In this section, we perform extensive experiments to investigate various approaches that can stabilize and improve the performance of fast l0 adversarial training. Then, we compare the performance of 1-step adversarial training with the multi-step counterparts. 5.1 Approaches to Improving 1-Step l0 Adversarial Training Table 4: Comparison of different approaches and their combinations in robust accuracy (%) by s AA. The target sparsity level ϵ = 20. We compare Pre Act Res Net-18 [24] models trained on CIFAR-10 [25]. Note that S and N denote SAT and N-FGSM, respectively. The italic numbers indicate catastrophic overfitting (CO) happens. |
| Hardware Specification | Yes | Cost times are recorded on one NVIDIA RTX 6000 Ada. |
| Software Dependencies | No | The paper mentions common software components like SGD optimizer with specific factors, but does not provide version numbers for any libraries (e.g., Python, PyTorch, CUDA). |
| Experiment Setup | Yes | The training batch size is 128. We train the model for 100 epochs. Image Net-100 [51]: The adopted network is Res Net-34 [24]. The training batch size is 48. We train the model for 50 epochs. The optimizer is SGD with a momentum factor of 0.9 and a weight decay factor of 5 10 4. The learning rate is initialized to 0.05 and is divided by a factor of 10 at the 1/4 and 3/4 of the total epochs. |