Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Beyond the Click-Through Rate: Web Link Selection with Multi-level Feedback
Authors: Kun Chen, Kechao Cai, Longbo Huang, John C.S. Lui
IJCAI 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on three real-world datasets, and show that Con UCB outperforms state-of-the-art context-free bandit algorithms concerning the multi-level feedback structure. |
| Researcher Affiliation | Academia | Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University Department of Computer Science and Engineering, The Chinese University of Hong Kong |
| Pseudocode | Yes | Algorithm 1 Constrained Upper Con๏ฌdence Bound |
| Open Source Code | No | The paper does not provide any statement or link regarding the public availability of its source code. |
| Open Datasets | Yes | Two datasets, Coupon Purchase [Kaggle, 2016] and Ad-Clicks [Kaggle, 2015], with 271 coupons and 225 ads respectively, are shown to have a two-level feedback structure in [Cai et al., 2017]. The third dataset, ed X-Course, is extracted from the data on 290 Harvard and MIT ed X online courses [Chuang and Ho, 2016]. |
| Dataset Splits | No | The paper mentions running algorithms for 50,000 rounds and generating rewards, but does not specify train/validation/test dataset splits or their sizes. |
| Hardware Specification | No | The paper does not provide any specific hardware details used for running the experiments. |
| Software Dependencies | No | The paper mentions implementing several bandit algorithms but does not provide specific software dependencies or version numbers. |
| Experiment Setup | Yes | For the three datasets, we run the three algorithms together with Con-UCB for 50,000 rounds with parameter settings as shown in Figure 1-3, respectively. In particular, the parameters of EXP3.M and LEXP are set in accordance with Corollary 1 of [Uchiya et al., 2010] and Theorem 1 of [Cai et al., 2017], respectively. |