Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Adversarial Attacks on Online Learning to Rank with Stochastic Click Models
Authors: Zichen Wang, Rishab Balasubramanian, Hui Yuan, chenyu song, Mengdi Wang, Huazheng Wang
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results based on synthetic and real-world data further validate the effectiveness and cost-efficiency of the proposed attack strategies. [...] In the experiment section, we apply the proposed attack methods against the OLTR algorithms listed in Table 1 with their corresponding click models. We compare the effectiveness of our attack on synthetic data and real-world Movie Lens dataset. |
| Researcher Affiliation | Academia | Zichen Wang EMAIL Department of Electrical Computer Engineering & Coordinated Science Laboratory University of Illinois Urbana-Champaign; Rishab Balasubramanian EMAIL Department of Computer Science Oregon State University; Yuan Hui EMAIL Department of Electrical Computer Engineering Princeton University; Chenyu Song EMAIL Department of Computer Science Oregon State University; Mengdi Wang EMAIL Department of Electrical Computer Engineering Princeton University; Huazheng Wang EMAIL Department of Computer Science Oregon State University |
| Pseudocode | Yes | Algorithm 1 Generalized List Poisoning Attack (GA); Algorithm 2 Attack-Then-Quit (ATQ) Algorithm; Algorithm 3 Cascade UCB1 (Kveton et al., 2015a); Algorithm 4 Batch Rank (Zoghi et al., 2017); Algorithm 5 Display Batch; Algorithm 6 Collect Clicks; Algorithm 7 Update Batch; Algorithm 8 The Top Rank (Lattimore et al., 2018) |
| Open Source Code | Yes | Our code and data can be accessed publicly for reproducibility.1. [Footnote 1: https://github.com/rishabbala/Online-Learning-to-Rank-for-Stochastic-Click-Models] |
| Open Datasets | Yes | We also evaluate the proposed attacks on Movie Lens dataset (Harper and Konstan, 2016). |
| Dataset Splits | No | The paper states: 'We first split the dataset into train and test data subsets.' for the Movie Lens dataset, but does not provide specific percentages, sample counts, or a detailed methodology for these splits. For synthetic data, it describes generation, not splitting of an existing dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python version, library versions like PyTorch, TensorFlow, scikit-learn). |
| Experiment Setup | Yes | For all our experiments, we use L = 50, K = 5 (the set up of Zoghi et al. (2017); Lattimore et al. (2018) is L = 10 and K = 5) and T = 10^5. For ATQ, we set the T1 in Algorithm 2 by Theorem 3 and Theorem 4. [...] We generate a size-L item set D, in which each item ak is related to a unique attractiveness score α(ak). Each attractiveness score α(ak) is drawn from a uniform distribution U(0, 1). |