Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Uniform Convergence of Rank-weighted Learning
Authors: Justin Khim, Liu Leqi, Adarsh Prasad, Pradeep Ravikumar
ICML 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 6. Experiments, Following our theoretical analysis, we test our results on logistic regression and linear regression. Our results for logistic regression are given in Figure 2, and our results for linear regression are given in Figure 3. |
| Researcher Affiliation | Academia | Justin Khim 1 Liu Leqi 1 Adarsh Prasad 1 Pradeep Ravikumar 1 1Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA. Correspondence to: Justin Khim <EMAIL>. |
| Pseudocode | No | The paper describes an 'iterative optimization procedure' with an equation, but it is not presented as a formal pseudocode block or algorithm. |
| Open Source Code | Yes | The code of the experiments can be found at https://bit.ly/2YzwRk J. |
| Open Datasets | No | In the logistic regression setup, the features are drawn from X Uniform(B(d)) where B(d) is the d-dimension ball. The labels are sampled from a distribution over { 1, 1} where Y takes the value +1 with probability (1+X T *)/2. where * = (1, 0, ..., 0) Rd. In the linear regression experiment, we draw our covariates from a Gaussian X N(0, Id) in Rd. The noise distribution is fixed as N(0, 0.01). We draw our response variable Y as, Y= X T * + where * = (1, 1, ..., 1) Rd. |
| Dataset Splits | No | The paper describes generating samples for empirical evaluation and mentions sample sizes, but does not specify standard training/validation/test dataset splits or their percentages/counts. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide any specific software dependencies with version numbers. |
| Experiment Setup | Yes | In the logistic regression setup, the features are drawn from X Uniform(B(d)) where B(d) is the d-dimension ball. The labels are sampled from a distribution over { 1, 1} where Y takes the value +1 with probability (1+X T *)/2. where * = (1, 0, ..., 0) Rd. We use the logistic loss ( ; (X, Y)) = log ( 1 + exp( YX T ) ) . For HRM, we have chosen a=0.6 and b=0.4. For CVa R and trimmed mean, is set to be 0.1. ... iterative optimization procedure to obtain a heuristic minimizer: t+1 = t - t / n * sum_i=1^n wt_i * ( t; Zi), (7) for all t in {0, ..., T-1}, where wt_i = w(Fn((t; Zi))) is the rank-dependent weighting and t is the learning rate. |