Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding
Authors: Jingping Liu, Mingchuan Zhang, Weichen Li, Chao Wang, Shuang Li, Haiyun Jiang, Sihang Jiang, Yanghua Xiao, Yunwen Chen
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In experiments, the manual and automatic evaluations prove the reliable quality of our Img Fact. We further use the obtained images to enhance model performance on two tasks. |
| Researcher Affiliation | Collaboration | 1School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China 2Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China 3School of Future Technology, Shanghai University, Shanghai, China 4Tencent AI Lab, Shenzhen, China 5Data Grand Inc., Shanghai, China jingpingliu@ecust.edu.cn, {mczhang18, wcli18, lishuang18, shawyh}@fudan.edu.cn cwang@shu.edu.cn, haiyunjiang@tencent.com, tedsihangjiang@gmail.com, chenyunwen@datagrand.com |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. Figure 2 presents a pipeline diagram, not pseudocode. |
| Open Source Code | Yes | We release Img Fact and its instructions at https://github.com/kleinercubs/Img Fact. |
| Open Datasets | Yes | We release Img Fact and its instructions at https://github.com/kleinercubs/Img Fact. ... we construct a new dataset from our Img Fact for automatic evaluation. The construction principle is that entities and relations in both validation and test sets need to appear in the training set. Based on this principle, a dataset (denoted as DL) is built, containing 3,340, 717, and 716 positive triplets for training, validation, and testing, respectively. |
| Dataset Splits | Yes | The dataset is constructed, containing 1,566 and 1,434 positive and negative entities, respectively, and then randomly split into training, validation, and test sets with 8:1:1, where the Fleiss kappa (Fleiss 1971) is 0.782, showing substantial agreement among these annotators. ... In this way, the built dataset contains 727 positive and 1,073 negative triplet-images pairs and is split into training, validation, and test sets according to 8:1:1, where the Fleiss kappa among the annotators on this task is 0.796. ... the dataset is split into training, validation, and test sets in a ratio of 8:1:1, where the Fleiss kappa on this task is 0.832. ... Based on this principle, a dataset (denoted as DL) is built, containing 3,340, 717, and 716 positive triplets for training, validation, and testing, respectively. |
| Hardware Specification | No | No specific hardware details (like GPU models or CPU processors) were provided. The paper only mentions the use of models like ResNet50 and ViLT, and classifiers without hardware context. |
| Software Dependencies | No | No specific version numbers for software dependencies (e.g., Python, PyTorch, specific library versions) were provided. Models like BERT, ResNet, CLIP, VGG, and DBSCAN are mentioned, but without versions. |
| Experiment Setup | No | No specific experimental setup details such as hyperparameter values (learning rate, batch size, epochs, optimizer settings) were found in the main text. |