华南理工大学学报（自然科学版）2026，Vol.54Issue(8)：36-48,13.DOI:10.12141/j.issn.1000-565X.250375
多模态商品摘要生成的要素评估与偏好优化
Component Evaluation and Preference Optimization for Multimodal Product Summary Generation
摘要
Abstract
The task of multimodal product summary generation aims to produce concise and accurate summary texts that highlight the core selling points of products by integrating both textual and visual information.However,existing methods still face two major challenges.First,traditional evaluation metrics such as ROUGE,which are based on lexical overlap,fail to precisely measure the completeness of a summary in expressing key product information.Second,mainstream supervised fine-tuning paradigms struggle to capture users'implicit preferences regarding the relative importance of different product information components,resulting in a misalignment between generated summaries and actual usage requirements.To address these issues,this paper proposes a component-based summarization evaluation metric,termed CSE(Component-based Summary Evaluation),which comprehensively evaluates the expression of key information from two perspectives:Component Hit Accuracy(CHA)and Component Quantity Ratio(CQR).Furthermore,a preference optimization-based multimodal product summary generation model,PAMPS,is designed.The model aligns its outputs with users'preferences for product component expression through four training stages:supervised fine-tuning,summary resampling,component evaluation-guided preference pair construction,and direct preference optimization.Experimental results on the large-scale Chinese e-commerce dataset CEPSUM show that PAMPS achieves significant improvements on ROUGE metrics.In particular,compared with Qwen2-VL-SFT,Qwen2-VL-DPO-ROUGE improves ROUGE-1,ROUGE-2,and ROUGE-L by 0.25,0.44,and 1.21 percentage points on average,respectively,demonstrating superior overall summary generation quality.Under the proposed CSE evaluation framework,the Qwen2-VL-DPO-CSE model demonstrates particularly prominent improvements in the CHA metric,with an average relative improvement of about 4%,indicating that component-oriented preference optimization can effectively enhance the model's ability to capture and express core product features.The comprehensive experimental results fully validate the effectiveness and practical applicability of the proposed method in improving the quality of multimodal product summary generation.关键词
多模态大模型/摘要评估/偏好优化Key words
multimodal large model/summarization evaluation/preference optimization分类
信息技术与安全科学引用本文复制引用
宋雪萌,李芷墨,侯博涵,尹建华..多模态商品摘要生成的要素评估与偏好优化[J].华南理工大学学报（自然科学版）,2026,54(8):36-48,13.基金项目
国家自然科学基金项目(62376137,62172261) （62376137,62172261）
山东省自然科学基金项目(ZR2022YQ59) Supported by the National Natural Science Foundation of China(62376137,62172261)and the Natural Science Foundation of Shandong Province(ZR2022YQ59) （ZR2022YQ59）
