Emergent Communication in a Multi-Modal, Multi-Step Referential Game
Authors: Katrina Evtimova, Andrew Drozdov, Douwe Kiela, Kyunghyun Cho
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments indicate that a robust and efficient communication protocol emerges, where gradual information exchange informs better predictions and higher communication bandwidth improves generalization." and "We evaluate and analyze the proposed multi-modal, multi-step referential game by creating a new dataset consisting of images of mammals and their textual descriptions. ... Our experiments indicate that a robust and efficient communication protocol emerges..." and sections like "5 EXPERIMENTAL SETTINGS", "6 RESULTS AND ANALYSIS". |
| Researcher Affiliation | Collaboration | Katrina Evtimova1, Andrew Drozdov2, Douwe Kiela3, and Kyunghyun Cho1,2,3,4 1Center for Data Science. New York University 2Department of Computer Science. New York University 3Facebook AI Research 4CIFAR Azrieli Global Scholar |
| Pseudocode | No | The paper describes the model architecture and training process using mathematical equations and textual descriptions, but no explicit pseudocode or algorithm blocks are provided. |
| Open Source Code | Yes | Code We used Py Torch [http://pytorch.org]. Our implementation of the agents and instructions on how to build the dataset are available on Github [https://github.com/nyu-dl/Multimodal Game]. |
| Open Datasets | No | We collect a new dataset consisting of images and textual descriptions of mammals. ... Our implementation of the agents and instructions on how to build the dataset are available on Github [https://github.com/nyu-dl/Multimodal Game]. |
| Dataset Splits | Yes | First, we keep a subset of sixty mammals for training (550 images per mammal) and set aside data for validation (50 images per mammal) and test (20 images per mammal). |
| Hardware Specification | Yes | We train on a single GPU (Nvidia Titan X Pascal), and a single experiment takes roughly 8 hours for 500 epochs. |
| Software Dependencies | No | The paper mentions 'Py Torch', 'NLTK', and 'Glo Ve word embeddings' but does not specify version numbers for these software components. |
| Experiment Setup | Yes | We train both the sender and receiver as well as associated baseline networks using RMSProp (Tieleman & Hinton, 2012) with learning rate set to 10 4 and minibatches of size 64 each. The coefficients for the entropy regularization, λs and λm, are set to 0.08 and 0.01 respectively, based on the development set performance from the preliminary experiments. Each training run is early-stopped based on the development set accuracy for a maximum of 500 epochs. We set the maximum length of a conversation to be 10, i.e., Tmax = 10. |