Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Unsupervised Risk Estimation Using Only Conditional Independence Structure
Authors: Jacob Steinhardt, Percy S. Liang
NeurIPS 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To better understand the behavior of our algorithms, we perform experiments on a version of the MNIST data set that is modified to ensure that the 3-view assumption holds. We use Algorithm 1 to perform unsupervised risk estimation for a model trained on a = 0, testing on various values of a [0, 10]. |
| Researcher Affiliation | Academia | Jacob Steinhardt Stanford University EMAIL Percy Liang Stanford University EMAIL |
| Pseudocode | Yes | Algorithm 1 Algorithm for estimating R(θ) from unlabeled data. |
| Open Source Code | No | No explicit statement about code availability or repository link found in the paper. |
| Open Datasets | Yes | To better understand the behavior of our algorithms, we perform experiments on a version of the MNIST data set that is modified to ensure that the 3-view assumption holds. |
| Dataset Splits | No | We trained the model with Ada Grad (Duchi et al., 2010) on 10,000 training examples, and used 10,000 test examples to estimate the risk. The term “validation error” is in the legend of Figure 4a, but no specific validation split or size is given. |
| Hardware Specification | No | The paper does not specify any hardware details like GPU/CPU models or specific computing infrastructure used for the experiments. |
| Software Dependencies | No | We trained the model with Ada Grad (Duchi et al., 2010) and locally minimize a weighted ℓ2-norm of the moment errors in (4) using L-BFGS. No version numbers for software are provided. |
| Experiment Setup | Yes | setting ρ = 10 in (8). |