Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Variance-based Regularization with Convex Objectives
Authors: Hongseok Namkoong, John C. Duchi
NeurIPS 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We give corroborating empirical evidence showing that in practice, the estimator indeed trades between variance and absolute performance on a training sample, improving out-of-sample (test) performance over standard empirical risk minimization for a number of classification problems. |
| Researcher Affiliation | Academia | Hongseok Namkoong Stanford University EMAIL John C. Duchi Stanford University EMAIL |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/hsnamkoong/robustopt. |
| Open Datasets | Yes | For our first experiment, we compare our robust regularization procedure to other regularizers using the HIV-1 protease cleavage dataset from the UCI ML-repository [14]. [...] The Reuters RCV1 Corpus [13] has 804,414 examples with d = 47,236 features |
| Dataset Splits | Yes | For validation, we perform 50 experiments, where in each experiment we randomly select 9/10 of the data to train the model, evaluating its performance on the held out 1/10 fraction (test). [...] We partition the data into ten equally-sized sub-samples and perform ten validation experiments, where in each experiment we use one of the ten subsets for fitting the logistic models and the remaining nine partitions as a test set to evaluate performance. |
| Hardware Specification | No | The paper does not specify the hardware used for running the experiments (e.g., CPU, GPU, or memory details). |
| Software Dependencies | No | The paper mentions using 'logistic loss' and 'elastic net regularization' but does not specify software names with version numbers for reproducibility. |
| Experiment Setup | Yes | We use the logistic loss ( ; (x, y)) = log(1 + exp( y >x)). We compare the performance of different constraint sets by taking = 2 Rd : a1 k k1 + a2 k k2 r , which is equivalent to elastic net regularization [27], while varying a1, a2, and r. We experiment with 1-constraints (a1 = 1, a2 = 0) with r 2 {50, 100, 500, 1000, 5000}, 2-constraints (a1 = 0, a2 = 1) with r 2 {5, 10, 50, 100, 500}, elastic net (a1 = 1, a2 = 10) with r 2 {102, 2 102, 103, 2 103, 104}, our robust regularizer with 2 {102, 103, 104, 5 104, 105} and our robust regularizer coupled with the 1-constraint (a1 = 1, a2 = 0) with r = 100. |