Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Predtron: A Family of Online Algorithms for General Prediction Problems
Authors: Prateek Jain, Nagarajan Natarajan, Ambuj Tewari
NeurIPS 2015 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A simulation study confirms the behavior predicted by our bounds and demonstrates the flexibility of the design choices in our framework. We now present simulation results to demonstrate the application of our proposed Predtron framework to subset ranking. We also demonstrate that empirical results match the trend predicted by our error bounds, hence hinting at tightness of our (upper) bounds. Figure 1 (a) shows LNDCG (on a test set) for our Predtron algorithm... |
| Researcher Affiliation | Collaboration | Prateek Jain Microsoft Research, INDIA EMAIL Nagarajan Natarajan University of Texas at Austin, USA EMAIL Ambuj Tewari University of Michigan, Ann Arbor, USA EMAIL |
| Pseudocode | Yes | Algorithm 1 Predtron: Extension of the Perceptron Algorithm to General Prediction Problems |
| Open Source Code | No | The paper does not contain any explicit statements about releasing open-source code for the described methodology, nor does it provide links to a code repository. |
| Open Datasets | No | We generated n data points X 2 Rm p0 using a Gaussian distribution with independent rows. The ith row of X represents a document and is sampled from a spherical Gaussian centered at µi. We selected a w 2 Rp0 and also a set of thresholds [ 1, . . . , m+1] to generate relevance scores... |
| Dataset Splits | No | The paper discusses generating synthetic data and evaluating on a 'test set', but it does not provide specific details about training, validation, and test splits (e.g., percentages, sample counts, or references to standard predefined splits). |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory, or cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper does not provide any specific software dependencies with version numbers (e.g., library names, frameworks, or solvers with their corresponding versions). |
| Experiment Setup | No | The paper describes how the synthetic data was generated ('We generated n data points X 2 Rm p0 using a Gaussian distribution...'), and explores different choices for `pred_1` functions, varying 'n' (number of training points), 'm' (number of documents), and 'p0' (data dimensionality). However, it does not specify concrete training hyperparameters for the Predtron algorithm itself, such as learning rate, batch size, or number of epochs. |