Convergence rates of sub-sampled Newton methods
Authors: Murat A. Erdogdu, Andrea Montanari
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate how our results apply to well-known machine learning problems. Lastly, we evaluate the performance of our algorithm on several datasets under various scenarios. Section 5 Experiments In this section, we validate the performance of New Samp through numerical studies. We experimented on two optimization problems, namely, Logistic Regression (LR) and SVM. |
| Researcher Affiliation | Academia | Murat A. Erdogdu Department of Statistics Stanford University erdogdu@stanford.edu Andrea Montanari Department of Statistics and Electrical Engineering Stanford University montanari@stanford.edu |
| Pseudocode | Yes | Algorithm 1 New Samp Input: ˆ 0, r, , { t}t, t = 0. 1. Define: PC( ) = argmin 02Ck 0k2 is the Euclidean projection onto C, [Uk, k] = Truncated SVDk(H) is rank-k truncated SVD of H with ii = λi. 2. while kˆ t+1 ˆ tk2 do |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | Table 1: Datasets used in the experiments. Dataset n p r Reference CT slices 53500 386 60 [GKS+11, Lic13] Covertype 581012 54 20 [BD99, Lic13] MSD 515345 90 60 [MEWL, Lic13] Synthetic 500000 300 3. [Lic13] M. Lichman, UCI machine learning repository, 2013. |
| Dataset Splits | No | The paper mentions using various datasets for experiments but does not explicitly provide details about how these datasets were split into training, validation, and test sets, or specify percentages/sample counts for each split. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., CPU, GPU models, memory, or cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions various algorithms and machine learning models (Logistic Regression, SVM, Gradient Descent, BFGS, etc.) but does not provide specific version numbers for any software dependencies or libraries (e.g., Python, PyTorch, scikit-learn) used in the implementation. |
| Experiment Setup | No | The paper states that constant step sizes were used for batch algorithms and optimized for stochastic algorithms, and that New Samp parameters were selected based on Section 3.4. However, it does not provide concrete values for hyperparameters like the chosen step sizes, batch sizes, number of epochs, or specific optimizer settings. |