智能系统学报2026，Vol.21Issue(1)：83-94,12.DOI:10.11992/tis.202507024
面向智能座舱的多源混合模态数据集及层次化融合分类方法
Multi-source hybrid-modality dataset and hierarchical fusion classification method for intelligent cockpits
摘要
Abstract
The scarcity of open-source data for intelligent cockpits in the driving domain is characterized by limited modality dimensions,insufficient annotations,and restricted scene diversity.To address these challenges,a multi-source hybrid-modality dataset has been constructed.This dataset incorporates RGB,depth,and infrared visual data,along with structured textual data detailing vehicle information and driving scenarios.A dual-layer annotation scheme is applied to capture ten behavior categories.Leveraging this dataset,a hierarchical multi-modal fusion framework is proposed to en-hance feature extraction via cross-modal information exchange and semantically guided fusion mechanisms.Experi-ments on video classification tasks reveal significant improvements in environmental understanding when combining RGB data with additional modalities.Using the full range of modalities leads to a 15.75％increase in accuracy com-pared to using only RGB data.These results validate the effectiveness of the multi-source hybrid-modality dataset in ad-vancing intelligent cockpit systems.关键词
智能座舱/数据集/多模态融合/视觉多模态/行为分类/危险行为/行为识别/多源数据Key words
intelligent cockpit/dataset/multimodal fusion/visual multimodality/behavior classification/dangerous be-havior/behavior recognition/multi-source data分类
信息技术与安全科学引用本文复制引用
赵荣峰,卢宝莉,唐小江,胡敏,李卫军,宁欣..面向智能座舱的多源混合模态数据集及层次化融合分类方法[J].智能系统学报,2026,21(1):83-94,12.基金项目
北京市自然科学基金-小米创新联合基金(L233036). （L233036）
