Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Uncertainty Quantification of MLE for Entity Ranking with Covariates
Authors: Jianqing Fan, Jikai Hou, Mengxin Yu
JMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Moreover, we validate our theoretical results through large-scale numerical studies. Finally, we conduct numerical experiments to corroborate our theory. Finally, we illustrate our methods via large-scale numerical studies on synthetic and real data. In this section, we conduct numerical experiments using synthetic and real data to validate our theories. In 5.1 and 5.2, we leverage synthetic data to corroborate the statistical rates given in 3 and distributional results given in 4, respectively. In addition, in 5.5, we illustrate further our model and methods by using the mutual funds holding data. |
| Researcher Affiliation | Academia | Jianqing Fan EMAIL Jikai Hou EMAIL Department of Operations Research and Financial Engineering Princeton University Princeton, NJ, United States Mengxin Yu EMAIL Department of Statistics and Data Science, the Wharton School University of Pennsylvania Philadelphia, PA, United States |
| Pseudocode | Yes | Algorithm 1 Gradient descent for regularized MLE. Algorithm 2 Construction of leave-one-out sequences. |
| Open Source Code | No | No explicit statement about the release of the code for the methodology described in this paper or a direct link to a code repository is provided. The license information provided is for the journal article, not specific code. |
| Open Datasets | Yes | 5.4 Application to Pokemon Challenge Data Set. We apply the proposed method to study the Pokemon challenge data set. The original data set can be found at https://www.kaggle.com/c/intelygenz-pokemon-challenge/data. |
| Dataset Splits | Yes | Among these remaining 800 28 = 772 pokemons for training purpose, we select the largest connected component of their comparison graph. Eventually we have 757 pokemons left for training. For each pokemon, we select log(Attack), log(HP), Mega or not as their covariates. Here Attack and HP denote the ability to attack and durability, respectively. The variable Mega or not takes binary value and represents whether this pokemon is mega evolved or not. We optimize the likelihood of our CARE model in (4) using training data and record bαM and bβM. We first investigate the statistical significance of these 3 variables we are interested in. This amounts to testing the following hypothesis testing problems for each feature: We then evaluate the competitions performance of the 28 mega evolved pokemons in the test sample, whose pre-evolutionary versions are the training data. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | In this section, we conduct numerical experiments using synthetic and real data to validate our theories. In 5.1 and 5.2, we leverage synthetic data to corroborate the statistical rates given in 3 and distributional results given in 4, respectively. In addition, in 5.5, we illustrate further our model and methods by using the mutual funds holding data. We begin with the data generation process. Throughout the synthetic data experiments, we set n to be 200 and d to be 5. The covariates are generated independently with (xi)j Uniform[ 0.5, 0.5] for all i [n], j [d]. For matrix X = [x1, x2, . . . , xn] Rn d, its columns are then normalized such that they have mean 0 and standard deviation 1. Next, we scale xi by xi/K so that maxi [n] xi 2/K = p (d + 1)/n. We generate α Rn by sampling its entries independently from Uniform[0.5, log(5) 0.5]. Also, a β Rd is generated uniformly from the hypersphere {β : β 2 = 0.5 p n/(d + 1)}. Then we project ( α , β ) onto linear space Θ and let it be eβ . In this way, we ensure κ1 5. |