Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Stochastic Regret Guarantees for Online Zeroth- and First-Order Bilevel Optimization
Authors: Parvin Nazari, Bojian Hou, Davoud Ataee Tarzanagh, Li Shen, George Michailidis
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on online parametric loss tuning and black-box adversarial attacks validate our approach. In this section, we present experimental results for two applications: online black-box attacks on deep neural networks and parametric loss tuning for imbalanced data. |
| Researcher Affiliation | Collaboration | Parvin Nazari Amirkabir University of Technology EMAIL, Bojian Hou University of Pennsylvania EMAIL, Davoud Ataee Tarzanagh Samsung SDS Research America EMAIL, Li Shen University of Pennsylvania EMAIL, George Michailidis University of California, Los Angeles EMAIL |
| Pseudocode | Yes | Algorithm 1 SOGD Require: (x1, y1, v1) X Rd2 Zp; p R++;T N; stepsizes {(αt, βt, δt) R3 ++}T t=1; parameters {(γt, λt, ηt)}T t=1 (0, 1); zt := (xt, yt). For t = 1 to T do:, Algorithm 2 ZO-SOGD Require: In addition to parameters in SOGD, choose ρv, ρr, ρs R++. For t = 1 to T do: |
| Open Source Code | Yes | Code is available at . |
| Open Datasets | Yes | Experiments were conducted on MNIST [48] with batch size 64. Reference [48]: Yann Le Cun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010. |
| Dataset Splits | No | The paper mentions "batches of training and validation samples" (34a) for online learning but does not provide specific percentages, sample counts, or explicit references to standard dataset splits for reproducing the data partitioning. |
| Hardware Specification | No | All experiments were conducted on the same system and are easily reproducible on a standard personal computer. |
| Software Dependencies | No | The paper mentions using a "4-layer CNN" and algorithms like "OGD", "OAGD", "SOBOW", "SOGD", and "Adam", but it does not specify any software libraries or frameworks with their version numbers. |
| Experiment Setup | Yes | Experiments were conducted on MNIST [48] with batch size 64. Learning rates were tuned as βt = δt = β {0.001, 0.005, 0.01, 0.05, 0.1}, αt = α {0.0001, 0.0005, 0.001, 0.005, 0.01}, and γt = λt = ηt = γ {0.9, 0.99, 0.999}. Both OAGD and SOBOW used 5 iterations for their respective system solvers. In our experiments, we set κ = 0. The optimal hyperparameter configuration for adversarial attacks consists of inner stepsize β = 0.1, outer stepsize α = 0.001, smoothing parameters ρv = 0.01 and ρr = ρs = 0.005, and momentum parameters γt = λt = ηt = 0.99. |