Emergent Quantized Communication
Authors: Boaz Carmeli, Ron Meir, Yonatan Belinkov
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally validate our approach in multiple scenarios. We consider three different games that fall into the well-known design of referential games, where a sender transmits information about a target object, which a receiver needs to identify (Lewis 2008; Lazaridou, Peysakhovich, and Baroni 2016; Choi, Lazaridou, and De Freitas 2018; Guo et al. 2019). Our objects include synthetic discrete objects, images, and texts. We also experiment with a variant, which we call the classification game, where the receiver needs to identify the class to which the object belongs. In all cases, we find our quantized communication to outperform the standard approach using Gumbel-softmax by a large margin, often even approaching the performance with fully continuous communication (Figure 1, bottom). |
| Researcher Affiliation | Academia | Technion Israel Institute of Technology boaz.carmeli@campus.technion.ac.il, rmeir@ee.technion.ac.il, belinkov@technion.ac.il |
| Pseudocode | Yes | Algorithm 1: Quantizing continuous communication. |
| Open Source Code | No | The paper does not provide an unambiguous statement of open-sourcing its code or a direct link to a code repository for the methodology described. The arXiv link is for the paper itself. |
| Open Datasets | Yes | Images. We use the Egg implementation of the image game from Lazaridou, Peysakhovich, and Baroni (2016).4 The Dataset contains images from Image Net (Deng et al. 2009). ... 4https://dl.fbaipublicfiles.com/signaling game data |
| Dataset Splits | Yes | In all experiments we split the data 80/10/10 into training, validation, and test sets, respectively. |
| Hardware Specification | Yes | Each experiment took under 24 hours on a single v100 GPU and a CPU with 128GB RAM. |
| Software Dependencies | No | The paper mentions using a "distilbert-base-uncased model (Sanh et al. 2019) from Huggingface (Wolf et al. 2020)", but does not provide specific version numbers for general software components or libraries required for reproducibility. |
| Experiment Setup | Yes | Word length of 100 for continuous (CN) and quantized (QT) modes, except for QT-RNN in Sent-Cls, where word length is 10. Alphabet size of QT is 10 in all configurations. For Gumbel-softmax (GS), Alphabet size is 10, 50, 100, and 10 for the RNN channel, and 100, 50, 100, and 100 for the Instant channel, for the Object, Image, Sent-Ref, and Sent-Cls games, respectively. |