基于蓝图可分离的视频超分辨率重建技术研究OACSTPCD
Blueprint separable convolution for video super-resolution
目前视频超分辨率重建技术面临着以下 2 个问题:相邻帧对齐过程中,难以做到内容对齐,这种情况尤其会发生于大运动视频帧超分辨率重建过程中;其次,在视频超分辨率重建过程中出现的运动模糊和多运动难以良好的进行融合.我们针对以上2 个问题提出了基于蓝图可分离的视频超分辨率重建技术(BSCVSR),设计出一种性能优越、针对性强的视频超分辨率重建技术算法.BSCVSR引入了金字塔级联蓝图可分离卷积对齐网络和时空注意力融合SR网络,分别在不同层级基于不同信息对视频帧进行对齐,接着通过加入时空注意力机制解决多运动和运动模糊的融合问题.同时,对齐网络中,我们引入蓝图可分离卷积来减少冗余.通过研究发现,内核内部的相关性相较于跨内核的相关性对于分离操作具有更明显更直接的效果,最终的实验数据也可以很好的证明这一点.通过最终的实验数据我们可以看到,BSCVSR为视频超分辨率重建技术在效果上、在模型的轻量化上都提供了可参考的方案.
Video super-resolution reconstruction technology has always faced the following two issues:it is diffi-cult to achieve content alignment during the alignment of adjacent frames,which is especially likely to occur during the super-resolution reconstruction of video frames with large motion.Secondly,in the process of video super-resolution reconstruction,motion blur and the fusion of multiple motions are challenging to handle effectively.In re-sponse to these two issues,we propose BSCVSR,a high-performance and targeted video super-resolution recon-struction technology algorithm.BSCVSR introduces a pyramid cascade blueprint separable convolution alignment network and a spatiotemporal attention fusion SR network,which aligns video frames at different levels based on dif-ferent information and then addresses the fusion problem of multiple motions and motion blur by incorporating spati-otemporal attention mechanisms.In the alignment network,we introduce blueprint separable convolutions to reduce redundancy.Through research,it is found that the internal correlations within a kernel have a more pronounced and direct effect on the separable operation compared to cross-kernel correlations,which is well supported by the final experimental data.From the final experimental data,we can see that BSCVSR provides a reference solution for video super-resolution reconstruction technology in terms of both performance and model lightweighting.
邬凌霄;杨欣;王翔辰
南京航空航天大学 自动化学院,江苏 南京 211100
计算机与自动化
视频超分辨率重建神经网络蓝图可分离卷积时空注意力机制
video super-resolution reconstructionneural networkblueprint separable convolutionspatiotempo-ral attention mechanism
《云南民族大学学报(自然科学版)》 2024 (005)
616-623 / 8
国家自然科学基金(62073164),中央高校基本科研业务费(NS2022041).
评论