Ad Auctions for LLMs via Retrieval Augmented Generation
Authors: MohammadTaghi Hajiaghayi, Sébastien Lahaie, Keivan Rezaei, Suho Shin
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | An empirical evaluation validates the feasibility and effectiveness of our approach over several ad auction scenarios, and exhibits inherent tradeoffs in metrics as we allow the LLM more flexibility to allocate ads. |
| Researcher Affiliation | Collaboration | Mohammad Taghi Hajiaghayi University of Maryland hajiagha@umd.edu Sébastien Lahaie Google Research slahaie@google.com Keivan Rezaei University of Maryland krezaei@umd.edu Suho Shin University of Maryland suhoshin@umd.edu |
| Pseudocode | Yes | Figure 2: Single allocation segment auction; Figure 3: Multi-allocation segment auction; Figure 5: Naive II mechanism |
| Open Source Code | Yes | Github repo will be available. |
| Open Datasets | No | The paper mentions using "publicly-available LLM APIs" (e.g., GPT4) and a "model from the sentence-transformers library" for experiments, and describes generated ad texts, but it does not provide concrete access information (link, DOI, citation) for a specific public dataset used for training or as input data for the main experiments. |
| Dataset Splits | No | The paper describes running experiments over 500 trials with defined scenarios, but it does not specify traditional training/validation/test dataset splits or cross-validation details for its experimental setup. |
| Hardware Specification | No | The paper states: "All our experiments run with standard computation resource, without GPU." This is too general and does not provide specific hardware models, processors, or detailed specifications. |
| Software Dependencies | No | The paper mentions using "publicly-available API access of GPT4" and "a model from the sentence-transformers library" but does not specify version numbers for these or any other software components. |
| Experiment Setup | Yes | Setup. We consider segment auctions where each segment is a single sentence, and the entire document consists of three segments. There are five types of auctions: (1) segment auction with replacement... The query is fixed to: Can you suggest some books similar to To Kill a Mockingbird ?... We use publicly-available API access of GPT4 as our LLM in our experiments. In this Section, we first provide the set of prompts used to generate output in our experiments. |