Geometric Multimodal Contrastive Representation Learning
Authors: Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva, Danica Kragic
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally demonstrate that GMC representations are semantically rich and achieve state-of-the-art performance with missing modality information on three different learning problems including prediction and reinforcement learning tasks. 5. Experiments We evaluate the quality of the representations learned by GMC on three different scenarios: |
| Researcher Affiliation | Academia | 1KTH Royal Institute of Technology, Stockholm, Sweden 2INESC-ID & Instituto Superior T ecnico, University of Lisbon, Portugal. |
| Pseudocode | No | The paper includes architectural diagrams (Figures 4, 5, 6) which illustrate the model structure, but these are not pseudocode or algorithm blocks describing procedural steps. |
| Open Source Code | Yes | Our code is available on Git Hub2. 2https://github.com/miguelsvasco/gmc |
| Open Datasets | Yes | Multimodal Handwritten Digits (MHD) dataset (Vasco et al., 2022b). CMU-MOSI (Zadeh et al., 2016) and CMU-MOSEI (Bagher Zadeh et al., 2018) |
| Dataset Splits | No | The dataset is split into 50, 000 training and 10, 000 testing samples. (MHD) and CMU-MOSEI consists of 18134 and 4643 training and testing samples, respectively. A separate, explicit validation split (e.g., specific percentages or counts for validation data) is not provided. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | We report all model architectures and training hyperparameters in Appendix D and E. E. Training Hyperparameters In Table 19 we present the hyperparameters employed in this work. For training the controller in the RL task, we employ the same training hyperparameters as in Silva et al. (2020). Table 19 lists 'Learning rate', 'Batch size', 'Model training epochs', 'Temperature'. |