基于多任务蒸馏的意图识别和槽位填充OA北大核心CSTPCD

At present,pre-trained models such as BERT have achieved good results in many NLP tasks,but the pre-trained models are difficult to deploy in small configuration environments because of their large parameter scale,large computation and high requirements on hardware resources.Model compression is the key to solve this problem,and knowledge distillation is currently a better model compression method.A joint model of sentence intent recognition and slot filling based on multi-task distillation is proposed.The model applies ALBERT to task-based dialogue system,and uses the knowledge distillation strategy to migrate the ALBERT model knowledge to the BiLSTM model.Experimental results show that the sentence accuracy rate of the ALBERT based joint model in the SMP 2019 evaluation data set is 77.74%,the sentence accuracy rate of the BiLSTM model trained separately is 58.33%,and the sentence accuracy rate of the distillation model is 67.22%,which is 8.89%higher than the BiLSTM model while offering an inference speed approximately 18.9 times faster than ALBERT.