Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
signSGD via Zeroth-Order Oracle
Authors: Sijia Liu, Pin-Yu Chen, Xiangyi Chen, Mingyi Hong
ICLR 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical evaluations on image classification datasets MNIST and CIFAR-10 demonstrate the superior performance of ZO-sign SGD on the generation of adversarial examples from black-box neural networks. |
| Researcher Affiliation | Collaboration | Sijia Liu Pin-Yu Chen Xiangyi Chen Mingyi Hong MIT-IBM Watson AI Lab, IBM Research University of Minnesota, Twin Cities |
| Pseudocode | Yes | Algorithm 1 Generic sign-based gradient descent |
| Open Source Code | No | The paper does not include an unambiguous statement of releasing its own source code or a direct link to a repository for the work described. |
| Open Datasets | Yes | Our empirical evaluations on image classification datasets MNIST and CIFAR-10 |
| Dataset Splits | No | The paper mentions training samples and testing samples (n = 2000 training, 200 testing) for the synthetic dataset, but does not explicitly specify a validation set or general data split percentages for reproduction across all datasets. |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU models, CPU types, or cloud computing instances with specifications) used for running the experiments. |
| Software Dependencies | No | The paper does not explicitly list software dependencies with specific version numbers (e.g., 'Python 3.8, PyTorch 1.9, and CUDA 11.1'). |
| Experiment Setup | Yes | We find the best constant learning rate for algorithms via a greedy search over η [0.001, 0.1] (see Appendix 8.1 for more details), and we choose the smoothing parameter µ = 10/ Td. Unless specified otherwise, let b = q = 10, T = 5000 and d = 100. In our experiment, we set c = 1 for MNIST and c = 0.1 for CIFAR-10. We also set the same parameters for each method, i.e., µ = 0.01, q = 9, and δ = 0.05 for MNIST and δ = 0.0005 for CIFAR-10, to accommodate to the dimension factor d. |