Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
Authors: Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, Danica J. Sutherland
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper outlines key characteristics of agents behavior in the Bayesian-IL framework, including predictions that are supported by experimental verification with various LLMs. |
| Researcher Affiliation | Academia | Yi Ren UBC renyi.joshua@gmail.com Shangmin Guo University of Edinburgh s.guo@ed.ac.uk Linlu Qiu MIT linluqiu@mit.edu Bailin Wang MIT bailin.wang28@gmail.com Danica J. Sutherland UBC & Amii dsuth@cs.ubc.ca |
| Pseudocode | No | The paper describes algorithms and procedures conceptually but does not include a formal pseudocode block or algorithm listing. |
| Open Source Code | Yes | The code for experiments is available at https://github.com/Joshua-Ren/i ICL. |
| Open Datasets | Yes | We finetune a pretrained llama-2-7B model (Touvron et al. 2023) using Antropic-HH dataset (Bai et al. 2022) |
| Dataset Splits | No | The paper describes initial data (`d0`) and data pools (`Dpool`) used in experiments, but it does not specify explicit training, validation, or test dataset splits (e.g., percentages, sample counts, or predefined split citations) needed for reproducibility. |
| Hardware Specification | No | The paper mentions the use of LLMs (GPT3.5, GPT4, Claude3-haiku, Mixtral-8x7b) for experiments, but it does not specify the underlying hardware (e.g., GPU/CPU models, memory, or specific cloud instances) used for running these experiments. |
| Software Dependencies | No | The paper mentions using specific LLMs (GPT3.5, GPT4, Claude3-haiku, Mixtral-8x7b) but does not provide details on specific software dependencies, libraries, or frameworks with version numbers (e.g., Python, PyTorch, TensorFlow) for reproducibility. |
| Experiment Setup | Yes | The temperature is 0.1 and the probability feedback is enabled. |