首都师范大学学报（自然科学版）2025，Vol.46Issue(2)：1-11,11.DOI:10.19789/j.1004-9398.2025.02.001
基于预训练模型与双向注意力流的抽取式阅读理解模型
An extractive reading comprehension model based on pre-trained model with bi-directional attention flow
文勇军 1吴金铭 1梅硕1
作者信息
- 1. 长沙理工大学物理与电子科学学院,湖南 长沙 410114
- 折叠
摘要
Abstract
An extractive reading comprehension model based on a pre-training model with bidirectional attention flow is constructed for the problem of low accuracy in answer prediction that occurs in extractive reading comprehension tasks of service robots.The model first uses a pre-training model to extract the shallow joint semantic representations of the question and the document context,then uses a bidirectional attention network to enhance feature interaction and information fusion to obtain the deep joint semantic features of the question and the document context.Finally,combines the shallow and deeps joint semantic representations to complete the extraction of answers through ranking,error filtering and localization operations.Experiments were conducted on the Stanford English machine reading comprehension dataset SQuAD 1.1 and the"iFlytek Cup"Chinese machine reading comprehension dataset CMRC 2018 for the extractive question and answer task.The results show that compared with the English pre-trained language model BERT,the performance metrics EM and F1 values of this model are improved by 1.172%and 1.194%,respectively;compared with the Chinese pre-trained language model RoBERTa-wwm-ext,the EM and F1 values are improved by 1.336%and 0.921%,respectively.关键词
自然语言处理/机器阅读理解/预训练模型/双向注意力流(BERT)/RoBERTa-wwm-ext/答案抽取Key words
natural language processing/machine reading comprehension/pre-training model/bidirectional encoder representations from transformers(BERT)/RoBERTa-wwm-ext/answer extraction分类
计算机与自动化引用本文复制引用
文勇军,吴金铭,梅硕..基于预训练模型与双向注意力流的抽取式阅读理解模型[J].首都师范大学学报（自然科学版）,2025,46(2):1-11,11.
