南京大学学报（自然科学版）2019，Vol.55Issue(1)：125-132,8.DOI:10.13232/j.cnkij.nju.2019.01.013
基于双向注意力流和自注意力结合的机器阅读理解
Research on machine reading comprehension task based on BiDAF with self-attention
摘要
Abstract
Machine Reading Comprehension (MRC)is always the research hotspot and core problem in Natural Language Processing(NLP).How to make the machine get close to human understanding will be the continuous research goal before the arrival of the intelligent era.Recently,Baidu released a large open-source Chinese reading comprehension data set DuReader,which aims to handle real-life RC (Reading Comprehension )issues.This large-scaleQA(questionandanswer)datasetismorepracticaland moredifficultthanever.Notlongago,attention mechanismhasbeensuccessfullyextendedto NLP.Typically,these methodsuseattentiontofocusonasmall portionofthecontextandsummarizeitwithafixed-sizevector,coupleattentionstemporally,andoftenform a uni-directionalattention.InviewoftheexcellenteffectofattentionmechanismappliedinthefieldofNLP,westudy andusetheBi-DirectionalAttentionFlow(BiDAF)withself-attentionnetworktodealwiththe MRCtaskinthis paper.Byusingthemodel,thequery-awarecontextrepresentationcanbeobtainedandthegranularitycanalsobe classified.Wealso useself-attention mechanism tocapture word dependenciesand syntaxinformationinthe sentencesoftextandquestions.Thisstepcanreducesemanticlossofsentencesduringinformationaggregation. Thenweaggregatesemanticinformationbybi-LSTM(LongShort-Term Memory)togettheinformation matrix whichisusedtopredictthefinalanswer.Aftertraining,weobtaintheresultthatpercentageofidenticalwords (BLEU-4)is44.7% andpercentageofoverlappingunits(Rouge-L)is49.1%,wherehumanaveragelevelare55.1% and54.4% respectively.Thereisstillacertaingapbetweentheexperimentalresultsandthehumanlevelbutitis notverylarge,indicatingthatthemethodiseffectiveandscalable.关键词
中文机器阅读理解/DuReader数据集/BiDAF模型/自注意力机制Key words
MRC/DuReader/BiDAF/self-attention分类
信息技术与安全科学引用本文复制引用
顾健伟,曾诚,邹恩岑,陈扬,沈艺,陆悠,奚雪峰..基于双向注意力流和自注意力结合的机器阅读理解[J].南京大学学报（自然科学版）,2019,55(1):125-132,8.基金项目
国家自然科学基金(61673290,61728205,61750110534),江苏省研究生实践创新计划(SJCX17_0681),苏州市科技发展计划产业前瞻性项目(SYG201707,SYG201817) （61673290,61728205,61750110534）
