计算机与数字工程2024，Vol.52Issue(10)：3059-3064,6.DOI:10.3969/j.issn.1672-9722.2024.10.036
基于XLNet与双向注意力的机器阅读理解研究
Research on Machine Reading Comprehension Based on XLNet and Bidirectional Attention
摘要
Abstract
The purpose of machine reading comprehension is to enable the machine to read and accurately understand a natu-ral language text,and answer a given question,which has high research and application value.Aiming at the problem of low accura-cy due to the lack of effective interaction information between articles and questions in the existing universal domain machine read-ing comprehension models,this paper proposes a reading comprehension model based on XLNet and bidirectional attention.In this model,XLNet pretraining language model is used to generate context-dependent word vectors for sequential representation of con-tent and problem at the embedding layer,and two layers of LSTM are used to extract semantic features at the coding layer,and two bidirectional attention mechanisms(Bi-Attention and Co-Attention)are used to extract sequence features at the interaction layer,and then the self-attention mechanism is used to further enhance the representation of text features,and vector fusion is carried out.Finally,the start and end positions of the answers are obtained at the input and output layer after bidirectional LSTM modeling.Ex-perimental results in DuReader Chinese dataset show that EM and F1 values are improved.关键词
机器阅读理解/XLNet/双向注意力/LSTMKey words
machine reading comprehension/XLNet/bidirectional Attention/LSTM分类
信息技术与安全科学引用本文复制引用
解红涛,牛甲奎..基于XLNet与双向注意力的机器阅读理解研究[J].计算机与数字工程,2024,52(10):3059-3064,6.基金项目
黑龙江省自然科学基金项目(编号:LH2019F004) （编号:LH2019F004）
东北石油大学引导性创新基金项目(编号:2019YDL-20)资助. （编号:2019YDL-20）
