计算机与现代化Issue(9)：101-106,120,7.DOI:10.3969/j.issn.1006-2475.2024.09.017
结合注意力机制和Mengzi模型的短文本分类
Short Text Classification Combining Attention Mechanism and Mengzi Model
摘要
Abstract
How to use short text classification technology to mine useful text information is one of the current hot research direc-tions.To solve the problem of sparse feature information and difficult extraction of short text,a short text classification model named Mengzi-ADCBU is proposed.This model uses Mengzi pre-training model to convert input text information into correspond-ing text representation.Then,the obtained text vectors are input to the improved deep pyramid convolutional neural network and the bidirectional gated unit integrated with multi-head attention mechanism to extract text feature information,and the extracted feature information is fused and sent to the full connection layer and Softmax function to complete short text classification.Multiple models comparison experiments are carried out on the publicly available THUCNews short text data set and SougouCS short text data set respectively.The experimental results show that the proposed Mengzi-ADCBU model is better than the current mainstream models in the accuracy,precision,recall rate and F1 value of short text classification and has better short text classification ability.关键词
短文本/多头注意力/深度金字塔卷积神经网络/双向门控单元Key words
short text/multi-head attention/deep pyramid convolutional neural netwrks/bidirectional gated unit分类
信息技术与安全科学引用本文复制引用
陈雪松,李衡,王浩畅..结合注意力机制和Mengzi模型的短文本分类[J].计算机与现代化,2024,(9):101-106,120,7.基金项目
国家自然科学基金资助项目(61402099,61702093) （61402099,61702093）
