郑州大学学报（工学版）2024，Vol.45Issue(2)：60-71,12.DOI:10.13705/j.issn.1671-6833.2024.02.001
多模态命名实体识别方法研究进展
Research Progress of Multimodal Named Entity Recognition
摘要
Abstract
In order to solve the problems in studies of multimodal named entity recognition,such as the lack of text feature semantics,the lack of visual feature semantics,and the difficulty of graphic feature fusion,a series of mul-timodal named entity recognition methods were proposed.Firstly,the overall framework of multi modal named entity recognition methods and common technologies in each part were examined,and classified into BilSTM-based MNER method and Transformer based MNER method.Furthermore,according to the model structure,it was further divid-ed into four model structures,including pre-fusion model,post-fusion model,Transformer single-task model and Transformer multi-task model.Then,experiments were carried out on two data sets of Twitter-2015 and Twitter-2017 for these two types of methods respectively.The experimental results showed that multi-feature cooperative representation could enhance the semantics of each modal feature.In addition,multi-task learning could promote modal feature fusion or result fusion,so as to improve the accuracy of MNER.Finally,in the future research of MNER,it was suggested to focus on enhancing modal semantics through multi-feature cooperative representation,and promoting model feature fusion or result fusion by multi-task learning.关键词
多模态命名实体识别/Transformer/BiLSTM/多模态融合/多任务学习Key words
multimodal named entity recognition/Transformer/BiLSTM/multimode fusion/multitasking learning分类
信息技术与安全科学引用本文复制引用
王海荣,徐玺,王彤,荆博祥..多模态命名实体识别方法研究进展[J].郑州大学学报（工学版）,2024,45(2):60-71,12.基金项目
宁夏回族自治区自然科学基金资助项目(2023AAC03316) （2023AAC03316）
宁夏回族自治区教育厅高等学校科学研究重点项目(NYG2022051) （NYG2022051）
北方民族大学中央高校基本科研业务费专项资金资助项目(2022PT_S04) （2022PT_S04）
北方民族大学校级科研项目(2021XYZJK06) （2021XYZJK06）
