Hessian-Free High-Resolution Nesterov Acceleration For Sampling
Authors: Ruilin Li, Hongyuan Zha, Molei Tao
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical experiments in both log-stronglyconcave and multi-modal cases also numerically demonstrate this acceleration. |
| Researcher Affiliation | Academia | 1School of Mathematics, Georgia Institute of Technology 2School of Data Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen Institute of Artificial Intelligence and Robotics for Society. |
| Pseudocode | Yes | Algorithm 1 A 1st-order HFHR Algorithm; Algorithm 2 Randomized Midpoint Algorithm from (Shen & Lee, 2019), adapted for HFHR |
| Open Source Code | No | The paper does not provide an explicit statement or link for the release of its own source code for the described methodology. |
| Open Datasets | Yes | We use fully-connected network with [22, 10, 2] neurons, Re LU, standard Gaussian prior for all parameters, and compare ULD and HFHR on UCI data set Parkinson (Dua & Graff, 2017). |
| Dataset Splits | No | The paper details experimental setups like the number of independent realizations and parameter ranges, but it does not specify explicit train/validation/test dataset splits with percentages or counts for data partitioning. |
| Hardware Specification | No | The paper does not specify any particular hardware components (e.g., CPU, GPU models, or memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., programming languages, libraries, or frameworks with their versions) used in the experiments. |
| Experiment Setup | Yes | More specifically, we choose the initial measure to be Dirac at (100 1d, 0d), where 1d, 0d are d-dim. vectors filled with 1 and 0 respectively. d = 10. We pick threshold ϵ = 0.1, and for each α {0, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50, 100}, we try all combinations of (γ, h) {0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50, 100} 0.1 [50] for Algorithm 1 (we also run ULD algorithm when α = 0), and empirically find the best combination that requires the fewest iterations to meet |Eµkq Eµq| ϵ. We find that h = 5 already surpasses the stability limit of ULD algorithm, hence the range of step size covers the largest step size that are practically usable for ULD algorithm. 100,000 independent realizations are used (evenly spread to 100 different randomization seeds). |