Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks
Authors: Daogao Liu, Arun Ganesh, Sewoong Oh, Abhradeep Guha Thakurta
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | The paper primarily focuses on theoretical contributions, including proposing a novel framework, analyzing improved error rates, establishing guarantees for finding second-order stationary points, and providing theoretical bounds for excess risks. Table 1, for instance, summarizes 'SOTA' (state-of-the-art) versus their proposed 'α-SOSP' and 'excess population risk' as mathematical bounds (e.g., O((d/n)2/3)), which are theoretical results rather than empirical measurements from experiments. Section 1.2 discusses 'Our Techniques' which are mathematical derivations like 'bounding the total occurrences' and 'harnesses the power of the Log-Sobolev Inequality'. Appendix D.3 describes 'Implementation' in a theoretical sense, referring to existing algorithms and providing theoretical guarantees for a 'sampler', rather than reporting on actual empirical performance. |
| Researcher Affiliation | Collaboration | Arun Ganesh Google Research arunganesh@google.com; Daogao Liu University of Washington dgliu@uw.edu; Sewoong Oh University of Washington and Google Research sewoong@cs.washington.edu; Abhradeep Thakurta Google Deep Mind athakurta@google.com. Most of this work was done while the author was an intern at Google. |
| Pseudocode | Yes | Algorithm 1 Stochastic Spider; Algorithm 2 Above Threshold; Algorithm 3 Alternate Sample. |
| Open Source Code | No | The paper does not provide any statements about releasing source code or links to a code repository for the methodology described. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments involving datasets. Therefore, it does not mention specific training datasets or their availability. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments requiring dataset splits. It does not provide any information regarding training, validation, or test splits. |
| Hardware Specification | No | The paper is theoretical and does not describe any experiments that would require specific hardware. Therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not conduct experiments that would require specific software versions or dependencies. It discusses algorithms and theoretical properties but does not detail their practical implementation with versioned software. |
| Experiment Setup | No | The paper is theoretical and focuses on algorithm design and mathematical proofs. It does not describe any experimental setup, hyperparameters, or training configurations, as no empirical evaluation was performed. |