现代电影技术Issue(12)：4-10,56,8.
视听同步的细粒度脚步音效合成方法
Fine-grained footsteps sound synthesis for audiovisual synchronisation
摘要
Abstract
Film post-production sound effects are currently mainly produced by manual operation,which is costly and time-consuming.Existing intelligent Foley technologies cannot meet the realistic demands of film post-production sound effects due to the lack of fine-grained content and realism in synthesized sound.To address these challenges,this paper pro-poses a fine-grained footsteps sound synthesis method for audiovisual synchronization,which leverages the visual image information to achieve synchronized and content-matched footsteps sound effects.Specifically,this paper adopts a data-driven approach to audiovisual cross-modal generation to learn the audio-visual temporal correlations and achieve audio-vi-sual synchronization.Furthermore,to enhance the content granularity of the synthesized footsteps sounds,the study deeply analyzes the ground material and character motion information in the visual images,and mapped them to the corre-sponding sounds with particular rules.Experiments show that the proposed method can synthesize time-synchronized and content-reasonable footsteps sound effects that match the visual information,as well as realize the automated generation of footstep sound effects,to improve the audiovisual realism.关键词
电影音效制作/智能化拟音/脚步音效合成/跨模态视听生成Key words
Film Sound Production/Intelligent Foley/Footsteps Sound Synthesis/Cross-modal Audiovisual Generation分类
计算机与自动化引用本文复制引用
刘子航,齐秋棠,程皓楠,崔健,叶龙..视听同步的细粒度脚步音效合成方法[J].现代电影技术,2023,(12):4-10,56,8.基金项目
国家自然科学基金青年项目《基于数据与机理融合的交互感环境声合成理论与方法研究》(62201524). （62201524）
