计算机科学与探索2024，Vol.18Issue(3)：780-794,15.DOI:10.3778/j.issn.1673-9418.2211052
k-best维特比解耦合知识蒸馏的命名实体识别模型
Named Entity Recognition Model Based on k-best Viterbi Decoupling Knowledge Distillation
摘要
Abstract
Knowledge distillation is a general approach to improve the performance of the named entity recognition(NER)models.However,the classical knowledge distillation loss functions are coupled,which leads to poor logit distillation.In order to decouple and effectively improve the performance of logit distillation,this paper proposes an approach,k-best Viterbi decoupling knowledge distillation(kvDKD),which combines k-best Viterbi decoding to im-prove the computational efficiency,effectively improving the model performance.Additionally,the NER based on deep learning is easy to introduce noise in data augmentation.Therefore,a data augmentation method combining data filtering and entity rebalancing algorithm is proposed,aiming to reduce noise introduced by the original dataset and to enhance the problem of mislabeled data,which can improve the quality of data and reduce overfitting.Based on the above method,a novel named entity recognition model NER-kvDKD(named entity recognition model based on k-best Viterbi decoupling knowledge distillation)is proposed.The comparative experimental results on the datasets of MSRA,Resume,Weibo,CLUENER and CoNLL-2003 show that the proposed method can improve the general-ization ability of the model and also effectively improves the student model performance.关键词
命名实体识别(NER)/知识蒸馏/k-best维特比解码/数据增强Key words
named entity recognition(NER)/knowledge distillation/k-best Viterbi decoding/data augmentation分类
信息技术与安全科学引用本文复制引用
赵红磊,唐焕玲,张玉,孙雪源,鲁明羽..k-best维特比解耦合知识蒸馏的命名实体识别模型[J].计算机科学与探索,2024,18(3):780-794,15.基金项目
国家自然科学基金(61976124,61976125,62176140).This work was supported by the National Natural Science Foundation of China(61976124,61976125,62176140). （61976124,61976125,62176140）
