计算机工程与科学2024，Vol.46Issue(7)：1321-1330,10.DOI:10.3969/j.issn.1007-130X.2024.07.020
基于BERT字句向量与差异注意力的短文本语义匹配策略
A short text semantic matching strategy based on BERT sentence vector and differential attention
摘要
Abstract
Short text semantic matching is a core issue in the field of natural language processing,which can be widely used in automatic question answering,search engines,and other fields.In the past,most of the work only considered the similar parts between texts,while ignoring the different parts be-tween texts,making the model unable to fully utilize the key information to determine whether texts match.In response to the above issues,this paper proposes a short text semantic matching strategy based on BERT sentence vectors and differential attention.BERT is used to vectorize sentence pairs,BiLSTM is used,and a multi-header differential attention mechanism is introduced to obtain attention weights that represent intention differences between the current word vector and the global semantic in-formation of the text.A one-dimensional convolutional neural network is used to reduce the dimension of the semantic feature vectors of the sentence pairs,Finally,the word sentence vector is spliced and sent to the full connection layer to calculate the semantic matching degree between the two sentences.Experiments on LCQMC and BQ datasets show that this strategy can effectively extract text semantic difference information,thereby enabling the model to display better results.关键词
短文本语义匹配/字句向量/表征意图/差异注意Key words
short text semantic matching/word sentence vector/represent intention/difference notice分类
信息技术与安全科学引用本文复制引用
王钦晨,段利国,王君山,张昊妍,郜浩..基于BERT字句向量与差异注意力的短文本语义匹配策略[J].计算机工程与科学,2024,46(7):1321-1330,10.基金项目
山西省自然科学基金(202203021221234) （202203021221234）
