重庆理工大学学报2024，Vol.38Issue(3)：170-180,11.DOI:10.3969/j.issn.1674-8425(z).2024.02.019
以对比学习与时序递推提升摘要泛化性的方法
Improving generalization of summarization with contrastive learning and temporal recursion
摘要
Abstract
To address the problems of the traditional text summarization models trained based on cross-entropy loss functions,such as degraded performance during inference,low generalization,serious exposure bias phenomenon during generation,and low similarity between the generated summary and the reference summary text,a novel training approach is proposed in this paper.On the one hand,the model itself generates a candidate set using beam search and selects positive and negative samples based on the evaluation scores of the candidate summaries.Two sets of contrastive loss functions are built using"argmax-greedy search probability values"and"label probability values"within the output candidate set.On the other hand,a time-series recursive function designed to operate on the candidate set's sentences guides the model to ensure temporal accuracy when outputting each individual candidate summary and mitigates exposure bias.Our experiments show the method significantly improves the generalization performance on the CNN/DailyMail and Xsum public datasets.The Rouge and BertScore reach 47.54 and 88.51 respectively on CNN/DailyMail while they reach 48.75 and 92.61 on Xsum.关键词
自然语言处理/文本摘要/对比学习/模型微调Key words
natural language processing/text summarization/contrastive learning/model fine-tuning分类
信息技术与安全科学引用本文复制引用
汤文亮,陈帝佑,桂玉杰,刘杰明,徐军亮..以对比学习与时序递推提升摘要泛化性的方法[J].重庆理工大学学报,2024,38(3):170-180,11.基金项目
国家自然科学基金项目(52062016) （52062016）
江西省重点研发计划(20203BBE53034) （20203BBE53034）
江西省03专项及5G项目(20224ABC03A16) （20224ABC03A16）
