Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Ease-of-Teaching and Language Structure from Emergent Communication
Authors: Fushan Li, Michael Bowling
NeurIPS 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We explore the connection between ease-of-teaching and the structure of the language through empirical experiments. |
| Researcher Affiliation | Academia | Fushan Li Department of Computing Science University of Alberta Edmonton, Canada EMAIL Michael Bowling Department of Computing Science University of Alberta Edmonton, Canada EMAIL |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that the source code for their methodology is publicly available. |
| Open Datasets | No | The paper describes the attributes used to create objects for the game (colors and shapes), which are generated within their experimental setup, but it does not refer to a pre-existing, publicly available dataset with concrete access information (link, DOI, or specific citation to the dataset itself). |
| Dataset Splits | No | The paper describes a game setup where objects are generated based on attributes, and agents are trained over a certain number of iterations. It does not define explicit train/validation/test dataset splits from a fixed dataset, but rather describes a continuous training and evaluation process involving dynamically generated game instances. |
| Hardware Specification | No | The paper does not specify the hardware used for running experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions methods like Adam and REINFORCE but does not provide specific version numbers for any software dependencies, libraries, or frameworks used in the implementation. |
| Experiment Setup | Yes | For training, we use the Adam (Kingma and Ba, 2014) optimizer with learning rate 0.001 for both S and L. We use a batch size of 100 to compute policy gradients. We use λS = 0.1 and λL = 0.05 in all our experiments. The dimensionalities of the hidden states in both gS and gL are 100. In the reset regime, the listener is trained with the speaker for 6k (i.e., 6000) iterations and then we reinitialize the listener. |