华中科技大学学报（自然科学版）2025，Vol.53Issue(3)：8-13,6.DOI:10.13245/j.hust.250503
文本语义引导的自动动态场景新视角渲染方法
Automatic novel view synthesis for dynamic scenes guided by text semantics
摘要
Abstract
A novel view synthesis method for dynamic scenes guided by text priors was proposed.The text information of dynamic foreground content was used as a semantic prior to guide the segmentation model to automatically generate high-quality foreground and background masks.Consequently,novel view synthesis for dynamic scenes can be achieved without manual annotation.Specifically,the Grounding DINO was first employed to convert text prompts into bounding box prompts,and the segment anything model(SAM)was used,based on the original image and bounding box prompts,to automatically generate dynamic foreground masks.Finally,a dynamic neural radiance field was constructed based on these dynamic foreground masks to achieve automatic novel view synthesis for dynamic scenes.The effectiveness of this method was validated on the NVIDIA dynamic scene dataset.In subjective comparative experiments,compared to other methods,our method successfully renders clearer dynamic foreground and static background using semantic guided prior knowledge from a new perspective.In objective comparison experiments,this method outperforms other state-of-the-art methods in terms of peak signal to noise ratio(PSNR),structural similarity(SSIM),and learned perceptual image patch similarity(LPIPS),which are metrics for evaluating image generation quality.关键词
新视角渲染/动态场景/文本引导/分割一切模型/掩码自动生成Key words
novel view synthesis/dynamic scene/text prior guidance/segment anything model(SAM)/automatic mask generation分类
信息技术与安全科学引用本文复制引用
林玉萍,李胜鹏,田丰瑞..文本语义引导的自动动态场景新视角渲染方法[J].华中科技大学学报（自然科学版）,2025,53(3):8-13,6.基金项目
国家重点研发计划资助项目(2020AAA0108102) （2020AAA0108102）
陕西省社会科学基金资助项目(2021K014). （2021K014）
