计算机技术与发展2024，Vol.34Issue(7)：123-130,8.DOI:10.20165/j.cnki.ISSN1673-629X.2024.0089
基于双融合图注意力网络多模态知识图谱链路预测
Multi-modal Knowledge Graph Link Prediction Based on Dual Fusion and Graph Attention Networks
摘要
Abstract
Knowledge graph link prediction aims to predict missing facts within the knowledge graph,addressing the issue of knowledge graph incompleteness.However,existing knowledge graph link prediction methods exhibit certain limitations.Traditional methods are re-stricted to utilizing a single data modality,thereby missing the opportunity to fully leverage the wealth of information provided by diverse data modalities.Moreover,in the context of graph neural networks,entities and relationships are frequently treated as independent elements,often overlooking the varying significance of entity relationships within distinct neighborhoods.To address these problems,a multi-modal knowledge graph link prediction model based on dual-fusion graph attention network is proposed.Firstly,three modalities of image,text and attribute were incorporated.To ensure consistency and synergy among data modal features,a dual-fusion mechanism that combines early and late fusion strategies was devised for multi-modal data amalgamation.To strengthen the fusion of entity relationships and neighborhood relationships in the knowledge graph,consideration is given to the diversity of entities and relationships.The fusion of entity and relationship representations is followed by aggregation through the graph attention network,thereby enhancing the feature representation of the entities.By conducting simulation experiments on four public datasets,specifically FB15K-237,WN18RR,DB15K,and YAGO15K,the results demonstrate the strong performance of the proposed multi-modal knowledge graph link prediction method.关键词
多模态/知识图谱/链路预测/模态融合/图注意力网络Key words
multi-modal/knowledge graph/link prediction/model fusion/graph attention network分类
信息技术与安全科学引用本文复制引用
张冬,梁平,顾进广..基于双融合图注意力网络多模态知识图谱链路预测[J].计算机技术与发展,2024,34(7):123-130,8.基金项目
国家社会科学基金重大项目(11&ZD189) （11&ZD189）
科技创新2030-"新一代人工智能"重大项目(2020AAA0108500) （2020AAA0108500）
国家重点研发计划(2022YFC3300800) （2022YFC3300800）
