南京信息工程大学学报2017，Vol.9Issue(6)：661-668,8.DOI:10.13878/j.cnki.jnuist.2017.06.011
多模态融合的家庭音乐相册自动生成
Automatic generation of family music album based on multi-modal fusion
摘要
Abstract
With the development of the big data and social network,electronic albums and online services have become basic uses of computers and the Internet.Especially in recent years,the number of electronic albums has exploded with the popularity of social network.So how to improve the user experience of music album becomes particularly important.A photo album with certain topic usually has some emotion information.This paper studies the problem of automatic generation of family music album based on multimodal fusion,so that users can enjoy music when browsing album photos with matched emotion. According to the emotions in music and images, the representative sentence-level features both for music and images are selected,and the LPP ( Locality Preserving Projection) is employed to study the relevance between the music and the images in the same emotion.The image feature and the music feature are mapped into the latent space with more emotional classification ability to realize the automatic generation of music album.In the experiments,the objective evaluation result shows that the LPP method is higher than pure CCA (Canonical Correlation Analysis) method in precision;and in the subjective evaluation,the proposed LPP method achieves 72. 06% at satisfaction level,which is close to the results of manually recommended approach (78. 09%) and is higher than the results of randomly recommended approach and pure CCA approach.关键词
音乐相册/情感模型/句子级别/多模态融合/隐式空间Key words
music album/emotion model/sentence-level/multi-modal fusion/latent space分类
信息技术与安全科学引用本文复制引用
刘君芳,邵曦..多模态融合的家庭音乐相册自动生成[J].南京信息工程大学学报,2017,9(6):661-668,8.基金项目
国家自然科学基金(61401227) （61401227）
北京市自然科学基金(4152053) （4152053）
