Robust Learning from Untrusted Sources
Authors: Nikola Konstantinov, Christoph Lampert
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform an extensive experimental evaluation of our algorithm and demonstrate its ability to learn from all available data, while successfully suppressing the effect of corrupted or irrelevant sources. [...] We perform two large sets of experiments, following the setup considered in our paper. |
| Researcher Affiliation | Academia | Nikola Konstantinov 1 Christoph H. Lampert 1 1Institute of Science and Technology, Klosterneuburg, Austria. Correspondence to: Nikola Konstantinov <nkonstan@ist.ac.at>. |
| Pseudocode | Yes | Pseudocode of the algorithm is given in Algorithm 1. Algorithm 1 Robust learning from untrusted sources Inputs: 1. Loss L, hypothesis set H, parameter λ 2. Reference dataset ST 3. Datasets S1, . . . , SN from the N sources for i = 1 to N do {Potentially in parallel} Compute d H (Si, ST ) end for Select α by solving (6). Minimize α-weighted loss: ˆhα = argminh Hˆϵα(h) Return: ˆhα |
| Open Source Code | Yes | Code is available at https://github.com/Nikola Kon1994/Robust Learning-from-Untrusted-Sources |
| Open Datasets | Yes | Our first set of experiments is on the "Multitask dataset of product reviews"2 (Pentina & Lampert, 2017), containing customer reviews for 957 Amazon products from the "Amazon product data" (Mc Auley et al., 2015a;b)... 2http://cvml.ist.ac.at/productreviews/. [...] The Animals with Attributes 2 dataset (Xian et al., 2018) contains 37322 images of 50 animal classes. |
| Dataset Splits | Yes | The hyperparameter λ is selected by 5-fold cross-validation on the trusted data. The regularization parameter is always selected by 5-fold cross-validation on the reference data. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions using a "pretrained model from the Tensor Nets package" but does not provide a version number for this or any other software dependencies (e.g., specific versions of Python, PyTorch, TensorFlow, or scikit-learn). |
| Experiment Setup | Yes | The hyperparameter λ is selected by 5-fold cross-validation on the trusted data. [...] All three methods use linear predictors and are trained by regularized logistic regression. The regularization parameter is always selected by 5-fold cross-validation on the reference data. [...] In our experiments, we use the recommended threshold value of c = 1.3452, under which the estimate of the linear predictor has been shown to achieve a 95% asymptotic relative efficiency (Pregibon, 1982). |