Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication
Authors: Olaf Lipinski, Adam Sobey, Federico Cerutti, Timothy Norman
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indicate that agents can develop a language capable of expressing the relationships between parts of their observation, achieving over 90% accuracy when trained in a referential game which requires such communication. |
| Researcher Affiliation | Academia | Olaf Lipinski1 Adam J. Sobey2,1 Federico Cerutti3 Timothy J. Norman1 1University of Southampton 2The Alan Turing Institute 3University of Brescia |
| Pseudocode | Yes | Pseudocode We provide a condensed pseudocode for both algorithms in Algorithm 1. In the case of the PMInc, the n-grams in the pseudocode would be whole messages, i.e., trigrams. This base pseudocode would then be duplicated, interpreting the context as either an observation that may provide positional information (e.g., begin+1) or an integer. For the PMIc algorithm, only the unigrams and bigrams would be evaluated. |
| Open Source Code | Yes | Our code is available on Git Hub at https://github.com/olipinski/TPG |
| Open Datasets | No | To train and evaluate the agents, we use datasets consisting of 200,000 samples for training, 200,000 for validation, and 20,000 for testing. Each dataset is generated independently, with sequences created randomly. Given the sequence length of 60 and the fact that no integers are repeated, the number of possible permutations is 60! 8 1081, which vastly exceeds the number of samples we generate. |
| Dataset Splits | Yes | To train and evaluate the agents, we use datasets consisting of 200,000 samples for training, 200,000 for validation, and 20,000 for testing. |
| Hardware Specification | Yes | The processors used were a mixture of Intel Xeon Silver 4216s and AMD EPYC 7502s. The GPUs used were a mixture of NVIDIA Quadro RTX 8000s, NVIDIA Tesla V100s, and NVIDIA A100s. |
| Software Dependencies | No | The paper mentions specific algorithms and optimizers like 'Adam optimiser' and 'Gumbel-Softmax reparametrization trick' but does not specify software versions for programming languages, libraries, or frameworks used (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | Table 5: Hyperparameters Parameter Value Epochs 1000 Optimizer Adam Learning Rate α 0.001 Gumbel-Softmax Temperature [1.0] Training Dataset Size 200k Test Dataset Size 20k No. Distractors 4 No. Points [20,40,60,100] Message Length 3 Vocabulary Size [13,26,52] Sender Hidden Size [64,128] Receiver Hidden Size [64,128] |