加强融合表情和语音的抑郁症检测模型OA北大核心CSTPCD

The expressions and speech of patients with depression have the advantages of being intuitive and easy to be obtained,so they have been widely used in the depression detection.However,the existing research has overlooked the role of the information contained in the process of expression change in the depression detection,and has failed to effectively combine the information contained in dynamic expressions with static expressions and speech,which results in low recognition accuracy.In view of the above,a depression detection model enhancing the fusion of static expression features with dynamic expressions and speech is proposed.The Bi-LSTM network is added to the speech feature extraction module to mine the temporal information of speech,and perform transfer learning with emotional speech,and then implement training with depression speech.The expression feature extraction module is structured with a dual channel,and a mixed attention mechanism is utilized to extract dynamic and static expression features,so the features are more discriminative.In the feature enhancement fusion module,the speech and dynamic expressions are used to enhance the fusion of static expressions,which enhances the complementation of feature information.The experiment results show that the proposed method can reduce the RMSE(root mean square error)and MAE(mean absolute error)detected on the dataset AVEC2014 to 8.21 and 6.03,respectively,so it is superior to the current methods that detect depression with speech and facial expressions.