Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning
Authors: Mingcheng Li, Dingkang Yang, Yang Liu, Shunli Wang, Jiawei Chen, Shuaibing Wang, Jinjie Wei, Yue Jiang, Qingyao Xu, Xiaolu Hou, Mingyang Sun, Ziyun Qian, Dongliang Kou, Lihua Zhang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments on three datasets demonstrate that HRLF significantly improves MSA performance under uncertain modality missing cases. |
| Researcher Affiliation | Academia | 1Academy for Engineering and Technology, Fudan University, Shanghai, China 2 Institute of Metaverse & Intelligent Medicine, Fudan University, Shanghai, China 3Cognition and Intelligent Technology Laboratory, Shanghai, China 4Jilin Provincial Key Laboratory of Intelligence Science and Engineering, Changchun, China 5Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The paper does not provide open access to the data and code. |
| Open Datasets | Yes | We conduct our experiments on three MSA benchmarks, including MOSI [71], MOSEI [72], and IEMOCAP [4]. |
| Dataset Splits | Yes | MOSI... There are 1,284, 229, and 686 video clips in train, valid, and test data, respectively. MOSEI... has 16,326, 1,871, and 4,659 samples in train, valid, and test data. IEMOCAP... Its predetermined data partition has 2,717, 798, and 938 samples in train, valid, and test data. |
| Hardware Specification | Yes | All models are built on the Pytorch [34] toolbox with four NVIDIA Tesla V100 GPUs. |
| Software Dependencies | No | The paper mentions "Pytorch [34] toolbox" and "Adam optimizer [20]" but does not specify their version numbers. |
| Experiment Setup | Yes | For MOSI, MOSEI, and IEMOCAP, the detailed hyper-parameter settings are as follows: the learning rates are {1e-3, 2e-3, 4e-3}, the batch sizes are {128, 16, 32}, the epoch numbers are {50, 20, 30}, and the attention heads are {10, 8, 10}. The embedding dimension is 40 on all three datasets. |