Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivity
Authors: William de Vazelhes, Hualin Zhang, Huimin Wu, Xiaotong Yuan, Bin Gu
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments |
| Researcher Affiliation | Academia | Mohamed bin Zayed University of Artificial Intelligence Nanjing University of Information Science & Technology |
| Pseudocode | Yes | Algorithm 1: Stochastic Zeroth-Order Hard-Thresholding (SZOHT) |
| Open Source Code | Yes | Our code will be included in the supplementary material |
| Open Datasets | Yes | We use three datasets: port3, port4 and port5 from the OR-library [3], of respective dimensions d = 89; 98; 225. We consider three different datasets for the attacks: MNIST, CIFAR, and Imagenet, of dimension respectively d = 784; 3072; 268203. |
| Dataset Splits | No | The paper mentions using datasets like MNIST, CIFAR, and ImageNet, which have standard splits, but it does not explicitly specify the training, validation, or test split percentages or sample counts used for its experiments. |
| Hardware Specification | Yes | All experiments are conducted in the workstation with four NVIDIA RTX A6000 GPUs, and take about one day to run. |
| Software Dependencies | No | The paper does not provide specific software names with version numbers (e.g., Python, PyTorch, or other libraries with their versions) that were used in the experiments. |
| Experiment Setup | Yes | For SZOHT, we set k = 10, s2 = 10, q = 10, and (µ, η) = (0.015, 0.015) for port4, and (µ, η) = (0.1, 1) for port5 (µ and η are both obtained by grid search over the interval [10 3, 103]). We set the hyperparameters of SZOHT as follows: MNIST: k = 20, s2 = 100, q = 100, µ = 0.3, η = 1; CIFAR: k = 60, s2 = 100, q = 1000, µ = 1e 3, η = 0.01; Image Net: k = 100000, s2 = 1000, q = 100, µ = 0.01, η = 0.015. |