Beyond Memorization: Violating Privacy via Inference with Large Language Models
Authors: Robin Staab, Mark Vero, Mislav Balunovic, Martin Vechev
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we present the first comprehensive study on the capabilities of pretrained LLMs to infer personal attributes from text. We construct a dataset consisting of real Reddit profiles, and show that current LLMs can infer a wide range of personal attributes (e.g., location, income, sex), achieving up to 85% top-1 and 95% top-3 accuracy at a fraction of the cost (100 ) and time (240 ) required by humans. ... Our key contributions are: ... 2. A comprehensive experimental evaluation of LLMs ability to infer personal attributes from real-world data both with high accuracy and low cost, even when the text is anonymized using commercial tools. |
| Researcher Affiliation | Academia | Robin Staab, Mark Vero, Mislav Balunovi c, Martin Vechev Department of Computer Science, ETH Zurich {robin.staab,mark.vero}@inf.ethz.ch |
| Pseudocode | No | The paper describes procedures and provides prompt templates in the appendix, but it does not include any clearly labeled "Pseudocode" or "Algorithm" blocks or structured algorithmic steps. |
| Open Source Code | Yes | Our key contributions are: ... 3. A release of our code, prompts, and synthetic chatlogs at https://github.com/ eth-sri/llmprivacy. |
| Open Datasets | No | The Personal Reddit (PR) Dataset... Due to the personal data contained in the dataset, we do not plan to make it public. Instead, we provide 525 human-verified synthetic examples, detailed in Appendix F. |
| Dataset Splits | No | The paper uses pre-trained LLMs for inference and does not describe a traditional train/validation/test split for its main experiments on the Personal Reddit dataset. |
| Hardware Specification | No | We accessed all Open AI models via their API... Models from Google were accessed via the Vertex AI API. All Llama models were run locally without quantization. Models from Anthropic were accessed via the Poe.com web interface. The paper mentions "locally" for Llama models, but no specific GPU, CPU, or memory details are provided for these local runs. |
| Software Dependencies | No | The paper mentions "Open AI models via their API on the -0613 checkpoint" and "Azure Language Service (Aahill, 2023)". While a specific checkpoint/version for OpenAI API is given and a commercial tool name, it does not list multiple key software components with their versions (e.g., Python, PyTorch, CUDA, etc.) in a reproducible manner. |
| Experiment Setup | Yes | Using the prompt template presented Appendix I, we then jointly predicted all attributes (per profile). For each attribute, we ask the models for their top 3 guesses in order (presenting all options for categoricals, see Appendix A). ... We set the sampling temperature for all models to 0.1 whenever applicable with a maximum generation of 600 tokens. ... We provide a full overview of our experimental setup in Appendix C. |