Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
As You Like It: Localization via Paired Comparisons
Authors: Andrew K. Massimino, Mark A. Davenport
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we perform a range of synthetic experiments to demonstrate our approach. (Section 6). The paper also includes figures (e.g., Figure 2: Mean error norm x x as 𝜎2 varies...) showing experimental results from simulations. |
| Researcher Affiliation | Academia | Andrew K. Massimino EMAIL Mark A. Davenport EMAIL School of Electrical and Computer Engineering Georgia Institute of Technology 777 Atlantic Dr NW Atlanta, GA 30332 USA |
| Pseudocode | No | The paper describes mathematical formulations for optimization problems (25) and (26) in Section 5, and a linearization procedure for solving a non-convex problem, but does not present any of these as structured pseudocode or an algorithm block. |
| Open Source Code | No | The paper does not contain any statement about releasing code, a link to a code repository, or mention of code in supplementary materials. |
| Open Datasets | No | The paper exclusively uses synthetic data generated for its experiments. For example, "In Fig. 2, we let x R{2,3,4,8} with x = R= 1. We vary 𝜎2 and perform 1500 trials, each with 𝑚= 100 or 𝑚= 200 pairs of points drawn according to 𝒩(0, 𝜎2𝐼)." (Section 6.1). No public datasets are mentioned or accessed. |
| Dataset Splits | No | The paper conducts simulations with synthetic data and mentions "trials" but not typical training/test/validation splits for a fixed dataset. For example, "1500 trials" (Section 6.1), "100 independent trials" (Section 6.2). These refer to experimental runs with newly generated data for each trial, not predefined splits of a static dataset. |
| Hardware Specification | No | The paper's "Simulations" section discusses experimental results but does not specify any hardware components (e.g., GPU models, CPU types, memory) used to run these simulations. |
| Software Dependencies | No | The paper does not mention any specific software libraries, frameworks, or programming languages with version numbers used for the implementation of the algorithms or simulations. |
| Experiment Setup | Yes | The paper provides several experimental setup details, such as: "In Fig. 2, we let x R{2,3,4,8} with x = R= 1. We vary 𝜎2 and perform 1500 trials, each with 𝑚= 100 or 𝑚= 200 pairs of points drawn according to 𝒩(0, 𝜎2𝐼)." (Section 6.1). In Section 6.2: "we set 𝑛= 5 and generate 𝑚= 1000 pairs of points and a random x with x = 0.7." Also, "As a rule of thumb, we set 𝜈= 2𝜅." and "we set w(𝑘+1) 𝜒 w(𝑘) + (1 𝜒) w(𝑘) with 𝜒= 0.7." |