陕西师范大学学报（自然科学版）2024，Vol.52Issue(3)：96-104,9.DOI:10.15983/j.cnki.jsnu.2024013
基于多任务蒸馏的意图识别和槽位填充
Research on sentence intention recognition and slot filling based on multi-task distillation
摘要
Abstract
At present,pre-trained models such as BERT have achieved good results in many NLP tasks,but the pre-trained models are difficult to deploy in small configuration environments because of their large parameter scale,large computation and high requirements on hardware resources.Model compression is the key to solve this problem,and knowledge distillation is currently a better model compression method.A joint model of sentence intent recognition and slot filling based on multi-task distillation is proposed.The model applies ALBERT to task-based dialogue system,and uses the knowledge distillation strategy to migrate the ALBERT model knowledge to the BiLSTM model.Experimental results show that the sentence accuracy rate of the ALBERT based joint model in the SMP 2019 evaluation data set is 77.74%,the sentence accuracy rate of the BiLSTM model trained separately is 58.33%,and the sentence accuracy rate of the distillation model is 67.22%,which is 8.89%higher than the BiLSTM model while offering an inference speed approximately 18.9 times faster than ALBERT.关键词
意图识别与槽位填充/神经网络/知识蒸馏Key words
intention recognition and slot filling/neural network/knowledge distillation分类
数理科学引用本文复制引用
高子雄,蒋盛益,欧炎镁,禤镇宇..基于多任务蒸馏的意图识别和槽位填充[J].陕西师范大学学报（自然科学版）,2024,52(3):96-104,9.基金项目
国家自然科学基金(61572145) （61572145）
