MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment
Authors: Zicheng Zhang, Wei Sun, Xiongkuo Min, Qiyuan Wang, Jun He, Quan Zhou, Guangtao Zhai
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our approach outperforms all compared state-of-the-art methods and is far ahead of previous no-reference PCQA methods, which highlights the effectiveness of the proposed method. Extensive experiments show that MM-PCQA achieves the best performance among the compared state-of-the-art methods (even including the FR-PCQA methods). |
| Researcher Affiliation | Collaboration | Zicheng Zhang1 , Wei Sun1 , Xiongkuo Min1 , Qiyuan Wang3 , Jun He3 , Quan Zhou3 and Guangtao Zhai1,2 1Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University 2Mo E Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University 3Bilibili Inc, Shanghai, China |
| Pseudocode | No | The paper describes the method using prose and mathematical equations but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/zzc-1998/MM-PCQA. |
| Open Datasets | Yes | To test the performance of the proposed method, we employ the subjective point cloud assessment database (SJTUPCQA) [Yang et al., 2020a], the Waterloo point cloud assessment database (WPC) proposed by [Liu et al., 2022a], and the WPC2.0 database [Liu et al., 2021a] for validation. |
| Dataset Splits | Yes | The k-fold cross validation strategy is employed for the experiment to accurately estimate the performance of the proposed method. Since the SJTU-PCQA, WPC, and WPC2.0 contain 9, 20, 16 groups of point clouds respectively, 9-fold, 5-fold, and 4-fold cross validation is selected for the three database to keep the train-test ratio around 8:2. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for experiments (e.g., GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper mentions specific optimizers (Adam) and neural network architectures (Point Net++, ResNet50) and databases (ImageNet), but does not specify software versions for libraries, frameworks, or programming languages (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | The Adam optimizer [Kingma and Ba, 2015] is utilized with weight decay 1e-4, the initial learning rate is set as 5e-5, and the batch size is set as 8. The model is trained for 50 epochs by default. Specifically, We set the point cloud sub-model size Ns as 2048, set the number of sub-models Nδ = 6, and set the number of image projections NI = 4. [...] The multi-head attention module employs 8 heads and the feed-forward dimension is set as 2048. The weights λ1 and λ2 for LMSE and Lrank are both set as 1. |