Learning Optimal Linear Regularizers
Authors: Matthew Streeter
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now evaluate the Learn Lin Reg and Tune Reg algorithms experimentally, using both real and synthetic data. |
| Researcher Affiliation | Industry | Matthew Streeter 1 1Google Research. Correspondence to: Matthew Streeter <mstreeter@google.com>. |
| Pseudocode | Yes | Algorithm Learn Lin Reg Input: Set of (validation loss, training loss, feature vector) tuples n (Vi, ˆLi, qi) | 1 i m o . Sort tuples in ascending order of Vi, and reindex so that V1 V2 . . . Vm. for i from 1 to m do |
| Open Source Code | Yes | Code for both algorithms is available on Git Hub (Streeter, 2019). |
| Open Datasets | Yes | We then consider two problems that make use of deep networks trained on MNIST and Image Net (Russakovsky et al., 2015). |
| Dataset Splits | No | The paper mentions using "validation loss" and describes the "flowers" dataset as "split evenly into training and test," but it does not provide specific training/validation/test dataset splits (percentages or sample counts) for all datasets used (e.g., MNIST, ImageNet) that would be needed for full reproducibility of data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running its experiments. |
| Software Dependencies | No | The paper refers to "Tensor Flow Team" but does not specify any software dependencies (e.g., libraries, frameworks, or solvers) with explicit version numbers. |
| Experiment Setup | Yes | For the softmax regression problems, we obtain a nearoptimal solution by running Ada Grad for 100 epochs. Each grid Λi contains 50 points. For L1 and L2, the grid is loguniformly spaced over [.1, 100], while for label smoothing and dropout it is uniform over [0, 1]. |