Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Preference Completion from Partial Rankings
Authors: Suriya Gunasekar, Oluwasanmi O. Koyejo, Joydeep Ghosh
NeurIPS 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further show promising empirical results for a novel and challenging application of collaboratively ranking of the associations between brain regions and cognitive neuroscience terms. |
| Researcher Affiliation | Academia | Suriya Gunasekar University of Texas, Austin, TX, USA EMAIL Oluwasanmi Koyejo University of Illinois, Urbana-Champaign, IL, USA EMAIL Joydeep Ghosh University of Texas,Austin, TX, USA EMAIL |
| Pseudocode | Yes | Algorithm 1 Proximal Gradient Descent for (2) with input Ω, {y(j) j }, ϵ and paramter λ |
| Open Source Code | No | The paper does not provide a direct link or explicit statement about the availability of the source code for the methodology described in this paper. It mentions |
| Open Datasets | Yes | Movielens is a movie recommendation website administered by Group Lens Research. We used competitive benchmarked movielens 100K dataset. [...] Neurosynth[37] is a publicly available database consisting of data automatically extracted from a large collection of functional magnetic resonance imaging (f MRI) publications... |
| Dataset Splits | Yes | We used the 5 fold train/test splits provided with the dataset (the test splits are non-overlapping). [...] We used 10% of randomly sampled entries of the matrix as test data and a another 10% for validation. We created training datasets at various sample sizes by subsampling from the remaining 80% of the data. This random split is replicated multiple times to obtain 3 bootstrapped datasplits (note that unlike cross validation, the test datasets here can have some overlapping entries). |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, memory amounts, or cloud instance types used for running its experiments. |
| Software Dependencies | No | The paper mentions "nilearn python package" in the context of data preprocessing for the Neurosynth dataset, but it does not specify a version number for nilearn or any other software dependency. |
| Experiment Setup | No | The paper mentions that "hyperparameters were tuned using grid search on a logarithmic scale" for SMC and MRPC, but it does not provide the specific values or ranges of these hyperparameters or other detailed training configurations. It states that for CoFiRank, default parameters were used due to computational cost, but this is a baseline, not their method. |