Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Black-Box Generalization: Stability of Zeroth-Order Learning
Authors: Konstantinos Nikolakakis, Farzin Haddadpour, Dionysis Kalogerias, Amin Karbasi
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We provide the first generalization error analysis for black-box learning through derivative-free optimization. Under the assumption of a Lipschitz and smooth unknown loss, we consider the Zeroth-order Stochastic Search (Zo SS) algorithm, that updates a d-dimensional model by replacing stochastic gradient directions with stochastic differences of K + 1 perturbed loss evaluations per dataset (example) query. For both unbounded and bounded possibly nonconvex losses, we present the first generalization bounds for the Zo SS algorithm. |
| Researcher Affiliation | Collaboration | Konstantinos E. Nikolakakis Yale University EMAIL Farzin Haddadpour Yale University EMAIL Dionysios S. Kalogerias Yale University EMAIL Amin Karbasi Yale University & Google Research EMAIL |
| Pseudocode | No | The paper describes algorithms using mathematical equations but does not present pseudocode or clearly labeled algorithm blocks within its content. |
| Open Source Code | No | Under section '3. If you ran experiments...', all sub-sections related to code and data reproduction are marked as '[N/A]', indicating no such materials were provided. |
| Open Datasets | No | The paper is theoretical and does not describe experiments performed on a specific, publicly available dataset with concrete access information. |
| Dataset Splits | No | The paper does not report empirical studies and therefore does not provide details on training/test/validation dataset splits. |
| Hardware Specification | No | Under section '3. If you ran experiments...', the question about total amount of compute and type of resources used is marked as '[N/A]', indicating no hardware specifications were provided. |
| Software Dependencies | No | Under section '3. If you ran experiments...', the question about specifying training details (e.g., data splits, hyperparameters, how they were chosen) is marked as '[N/A]', which implies no software dependency details were provided. |
| Experiment Setup | No | Under section '3. If you ran experiments...', the question about specifying training details (e.g., data splits, hyperparameters, how they were chosen) is marked as '[N/A]', indicating no such experimental setup details were provided. |