多模态命名实体识别方法研究进展OACSTPCD

In order to solve the problems in studies of multimodal named entity recognition,such as the lack of text feature semantics,the lack of visual feature semantics,and the difficulty of graphic feature fusion,a series of mul-timodal named entity recognition methods were proposed.Firstly,the overall framework of multi modal named entity recognition methods and common technologies in each part were examined,and classified into BilSTM-based MNER method and Transformer based MNER method.Furthermore,according to the model structure,it was further divid-ed into four model structures,including pre-fusion model,post-fusion model,Transformer single-task model and Transformer multi-task model.Then,experiments were carried out on two data sets of Twitter-2015 and Twitter-2017 for these two types of methods respectively.The experimental results showed that multi-feature cooperative representation could enhance the semantics of each modal feature.In addition,multi-task learning could promote modal feature fusion or result fusion,so as to improve the accuracy of MNER.Finally,in the future research of MNER,it was suggested to focus on enhancing modal semantics through multi-feature cooperative representation,and promoting model feature fusion or result fusion by multi-task learning.