北京生物医学工程2024，Vol.43Issue(6)：575-583,9.DOI:10.3969/j.issn.1002-3208.2024.06.004
一种基于知识蒸馏和注意力损失的时间增量学习系统
A session-incremental learning system based on knowledge distillation and attention loss
摘要
Abstract
Objective The brain-computer interface technology based on deep learning for motor imagery has a good development prospect in the field of intelligent rehabilitation.However,the motor imagery electroencephalogram(MI-EEG)signal is a non-stationary signal,its data distribution and feature space will change with the advancement of the rehabilitation process,which will cause the recognition ability of the convolutional neural network(CNN)model to decline.To enhance the temporal adaptability of the motor imagery(MI)decoding model,this paper proposes a session-incremental learning system(SILS)based on knowledge distillation and attention loss.Methods First,we performed band-pass filtering and down-sampling on the motor imagery EEG signals to enhance the information related to motor imagery.Next,a multi-branch,dual-attention,multi-module convolutional neural network was developed for extracting and integrating multi-scale temporal and spatial features from multi-lead MI-EEG data,utilizing an attention mechanism to amplify crucial channel and spatial information.Then,the ability of the incremental stage decoding model to continuously learn new knowledge and retain old knowledge was improved by using knowledge distillation technology and attention loss.Further,a small number of high-quality old samples were selected for data replay based on the nearest neighbor method to enhance the anti-forgetting performance of the incremental model.Finally,extensive experimental research was conducted by using the publicly available BCI Competition Ⅳ Dataset 2b,and the performance of SILS was verified through two indicators,plasticity and stability.Results SILS achieved average accuracies of 79.21％,79.05％,89.06％,88.38％,and 88.47％for stages 1 to 5,respectively,and the average forgetting rates of SILS for sessions 1 to 4 data in stage 5 were 9.72％,9.10％,9.88％,and 6.04％,respectively.Conclusions SILS has the ability to automatically adjust model parameters,maintain continuous learning and self-update,showing good temporal adaptability and performance stability.关键词
脑电信号/运动想像/增量学习/知识蒸馏/注意力机制Key words
electroencephalography/motor imagery/incremental learning/knowledge distillation/attention mechanism分类
医药卫生引用本文复制引用
李明爱,徐裕超..一种基于知识蒸馏和注意力损失的时间增量学习系统[J].北京生物医学工程,2024,43(6):575-583,9.基金项目
国家自然科学基金(62173010)资助 （62173010）
