数据与计算发展前沿2025，Vol.7Issue(3)：81-93,13.DOI:10.11871/jfdc.issn.2096-742X.2025.03.007
多模态交互:从人机协同迈向人智协同
Multimodal Interaction:From Human-Computer Collaboration to Human-Intelligence Collaboration
摘要
Abstract
[Objective]This paper explores the paradigm shift of multimodal interaction technology from"human-computer collaboration"to"human-intelligence collaboration".[Coverage]The refer-ences include 70 relevant works from recent domestic and international journals or conferences on multimodal human-intelligent interaction.[Methods]First,it introduces the development of multimodal interaction technology from traditional methods(speech,gestures,eye movements)to the human-intelligence interaction paradigm integrated with large models(LLMs,VLMs).Secondly,it focuses on the cutting-edge methods of interaction context awareness and user intention understanding,and presents cas-es of multimodal human-intelligence interaction technology in healthcare,education,creation,and daily life.[Re-sults]Multimodal human-intelligence interaction has been explored in various vertical and general fields,but still faces core technical challenges such as lack of compensation mechanisms and appropriate handling of erroneous or ambiguous intents.[Limitations]Due to the scope of available literature,only typical types of interaction mo-dalities are listed,and the coverage of application scenarios is limited.[Conclusions]Multimodal interaction is progressing towards greater intelligence.Future research should focus on optimizing compensation mechanisms,improving intent understanding accuracy,enhancing dynamic balancing mechanisms,and offering embodied de-sign to better support diverse intent tasks and facilitate scalable applications.关键词
多模态交互/智能体/智能交互/扩展现实Key words
multimodal interaction/agent/intelligent interaction/extended reality引用本文复制引用
王镇远,田东,董禹,乔娜,单桂华..多模态交互:从人机协同迈向人智协同[J].数据与计算发展前沿,2025,7(3):81-93,13.基金项目
中华人民共和国人与生物圈国家委员会项目"候鸟迁徙可视分析关键技术研究"(MAB-CN-2023-HNQX) （MAB-CN-2023-HNQX）
中国科学院计算机网络信息中心青年基金"面向科学数据分析的多模态空间智能交互"(25YF08) （25YF08）
