An engine not a camera: Measuring performative power of online search
Authors: Celestine Mendler-Dünner, Gabriele Carovano, Moritz Hardt
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To advance existing debates, we designed and executed an experiment to measure the performative power of online search providers. |
| Researcher Affiliation | Collaboration | Celestine Mendler-Dünner , Gabriele Carovano Moritz Hardt ELLIS Institute Tübingen Max-Planck Institute for Intelligent Systems, Tübingen and Tübingen AI Center Italian Competition Authority |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | Yes | The extension is publicly available in the Chrome store, and the code can fully be inspected. The link can be found on the project website. The project website can be found at: https://powermeter.is.tue.mpg.de/ |
| Open Datasets | No | Using Powermeter we collected data of about 57,000 search queries from more than 80 different subjects, over the period of 5 months. ... Our data was collected under the "need to know" principle, and not intended for publication. |
| Dataset Splits | No | The paper describes a Randomized Controlled Trial and analysis of collected click data, but does not involve training, validation, or test dataset splits in the typical machine learning sense. |
| Hardware Specification | No | The paper does not provide specific hardware details (such as GPU/CPU models, memory, or processor types) used for running the experiments or processing the collected data. |
| Software Dependencies | No | The paper mentions 'Chrome browser extension', 'Microsoft .Net core framework', 'docker container', and 'SQlite database' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | The implementation of the counterfactuals is done by identifying the relevant items to hide or swap by their html class names or ids. We also add custom tags and event listeners to the identified elements that we can fall back on at a later stage. The entire setup of the experiment usually takes around 40 milliseconds. This delay is far below what was found to be noticeable to users [42, 43]. Hiding the html body of the website with the first possible Chrome event is crucial to avoid glitches in case of bad internet connection and make sure the control arrangement is not revealed to the participant. To ensure internal validity of our experiment, we also have to ensure a participant is never reassigned to a new experimental group when reloading a page, navigating between tabs or repeatedly entering the same search query. This is done by storing a hash of user ID and search query together with the assigned experimental group in the browser cache. |