计算机工程2024，Vol.50Issue(5)：41-50,10.DOI:10.19678/j.issn.1000-3428.0068121
基于MacBERT与对抗训练的机器阅读理解模型
Machine Reading Comprehension Model Based on MacBERT and Adversarial Training
摘要
Abstract
Machine reading comprehension is designed to allow machines to understand natural language texts,resembling humans,and perform question-answering tasks accordingly.In recent years,owing to the development of deep learning and large-scale datasets,machine reading comprehension has received widespread attention.However,input problems in practical applications typically involve various noises and interferences,which affect the prediction results of a model.To improve the generalizability and robustness of a model,a machine reading comprehension model based on Masked language modeling as correction Bidirectional Encoder Representations from Transformers(MacBERT)and Adversarial Training(AT)is proposed.First,MacBERT is used to convert input questions and texts into word embeddings and vector representations.Subsequently,a small perturbation is added to the original word vector based on the gradient change of the original sample backpropagation to generate an adversarial sample.Finally,the original and adversarial samples are input into a Bidirectional Long Short-Term Memory(BiLSTM)network to further extract the contextual features of the text and output the predicted answer.Experimental results show that the F1 and Exact Matching(EM)values of this model on the simplified Chinese dataset CMRC2018 improve by 1.39 and 3.85 percentage points,respectively,compared with those of the baseline model.Meanwhile,the F1 and EM values on the traditional Chinese dataset DRCD improve by 1.22 and 1.71 percentage points,respectively,compared with those of the baseline model.Moreover,the F1 and EM values on the English dataset SQuADv1.1 improve by 2.86 and 1.85 percentage points,respectively,compared with those of the baseline model.The experimental results are better than those of most existing machine reading comprehension models.Based on actual question-answering results,the proposed model outperforms the baseline model in terms of robustness and generalizability;additionally,it performs better when the input problems contain noise.关键词
机器阅读理解/对抗训练/预训练模型/掩码校正的来自Transformer的双向编码器表示/双向长短期记忆网络Key words
machine reading comprehension/Adversarial Training(AT)/pre-trained model/Masked language分类
信息技术与安全科学引用本文复制引用
周昭辰,方清茂,吴晓红,胡平,何小海..基于MacBERT与对抗训练的机器阅读理解模型[J].计算机工程,2024,50(5):41-50,10.基金项目
成都市重大科技应用示范项目(2019-YF09-00120-SN). （2019-YF09-00120-SN）
