智能系统学报2024，Vol.19Issue(2)：429-437,9.DOI:10.11992/tis.202207030
特征融合的装修案例跨模态检索方法
A cross-modal retrieval algorithm of decoration cases on feature fusion
摘要
Abstract
Currently,home decoration customer service systems chiefly depend on manual decoration case retrieval,which leads to the system not meeting user demand for fast and timely consulting service and high labor costs.Thus,we propose a feature fusion-based cross-modal retrieval algorithm for decoration cases.Aiming at the problems of insuffi-cient semantic information mining of multimodal data and low accuracy of model retrieval,the existing style aggrega-tion module is improved.The channel attention mechanism is introduced into the original module to add suitable weights to the feature vectors of different pictures in each group of decoration cases,thereby improving the important features that include more helpful information and weakening other unimportant features.Conversely,a multimodal fea-ture fusion module is developed for retrieval scenarios to make full use of multimodal information.The module can ad-aptively control a series of fusion operations of feature vectors from two different modalities to achieve knowledge flow and sharing between cross-modal data,thereby producing feature vectors with richer semantics and stronger expressive ability to improve the retrieval performance of the model further.Our developed algorithm is compared with other meth-ods on a self-built multimodal dataset of decoration cases,and results show that the algorithm performs better in decora-tion case retrieval.关键词
家装客服系统/装修案例检索/跨模态检索/风格聚合/多模态/特征融合/通道注意力机制/语义信息Key words
home decoration customer service system/decoration case retrieval/cross-modal retrieval/style aggrega-tion/multimodal/feature fusion/channel attention mechanism/semantic information分类
信息技术与安全科学引用本文复制引用
亢洁,刘威..特征融合的装修案例跨模态检索方法[J].智能系统学报,2024,19(2):429-437,9.基金项目
陕西省重点研发计划项目(2021GY-022). （2021GY-022）
