Emergent Communication with Conversational Repair
Authors: Mitja Nikolaus
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here we explore the effects of conversational repair on languages emerging in signaling games. We extend the basic Lewis signaling game setup with a feedback channel that allows for the transmission of messages backwards from the receiver to the sender. Further, we add noise to the communication channel so that repair mechanisms become necessary for optimal performance. We find that languages emerging in setups with feedback channel are less compositional. However, the models still achieve a substantially higher generalization performance in conditions with noise, putting to question the role of compositionality for generalization. These findings generalize also to a more realistic case involving a guessing game with naturalistic images. |
| Researcher Affiliation | Academia | Mitja Nikolaus Cer Co, CNRS mitja.nikolaus@cnrs.fr. Work performed at Aix-Marseille University. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source code of the models and all experiments is publicly available at https://github.com/mitjanikolaus/ emergent_communication. |
| Open Datasets | Yes | Finally, we develop a more realistic guessing game setup with naturalistic scenes based on the Guess What?! dataset (De Vries et al., 2017), in which the receiver needs to discriminate a target object from a set of distractor objects within the same visual scene. [...] We keep the original train and validation splits as defined in Co Co (Lin et al., 2014). |
| Dataset Splits | Yes | We split the set of all possible objects into a training set (90%) and a test set (10%). [...] We keep the original train and validation splits as defined in Co Co (Lin et al., 2014). In total, there are 70,702 images (385,961 objects; 5.5 per image on average) in the training split and 8,460 (45,541 objects; 5.4 per image on average) in the validation split (which we use as test set). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running its experiments. It mentions 'Vision Transformer (vit-b-16; Dosovitskiy et al., 2020)' which is a model architecture, not hardware. |
| Software Dependencies | No | The paper mentions software components like 'gated Recurrent Neural Networks (RNNs) using single-layer GRUs with layer normalization (Ba et al., 2016)' and 'Vision Transformer (vit-b-16; Dosovitskiy et al., 2020)', but does not provide specific version numbers for general software dependencies or libraries (e.g., PyTorch version, Python version, specific libraries used for training). |
| Experiment Setup | Yes | Hyperparameters were configured as indicated in Table 1, unless stated otherwise. Table 1 includes: optimizer Adam, initial learning rate 0.001, batch size 1000, gradient clipping 1, message length 10, vocab size 2, sender embedding size 16, sender hidden dim 128, sender entropy coefficient 0.01, receiver embedding size 16, receiver hidden dim 128, receiver entropy coefficient 0.01. |