|国家科技期刊平台
首页|期刊导航|光学精密工程|基于混合时空卷积的轻量级视频超分辨率重建

基于混合时空卷积的轻量级视频超分辨率重建OA北大核心CSTPCD

Lightweight video super-resolution based on hybrid spatio-temporal convolution

中文摘要英文摘要

针对三维卷积神经网络在视频超分辨率任务上具有较高的计算复杂度以及提取时空特征有限的问题,本文设计了一种基于混合时空卷积的轻量级视频超分辨率重建网络.首先,提出了一个基于混合时空卷积的模块,实现了网络时空特征提取能力的提升以及计算复杂度的降低;其次,提出了一个基于相似性的选择性特征融合模块,进一步增强了相关特征的提取能力;最后,设计了一种基于注意力机制的运动补偿模块,在一定程度上减轻了错误的特征融合的影响.实验结果表明:所提网络可以在视频超分辨率性能和网络复杂度之间取得很好的平衡,而且在基准数据集SPMCS-11上4倍超分辨率达到8 frame/s.所提网络满足了边缘设备推理运行中快速、准确等要求.

Addressing the issue of high computational complexity and limited extraction of spatio-temporal features in 3D convolutional neural networks for video super-resolution tasks,this paper introduced a novel lightweight video super-resolution reconstruction network based on hybrid spatio-temporal convolution.Firstly,a hybrid spatio-temporal convolution-based module was proposed to realize the enhancement of the spatio-temporal feature extraction capability of the network as well as reduction of the computational complexity.Then,a similarity-based selective feature fusion module was proposed to further enhance the extraction capability of relevant features.Lastly,a motion compensation module based on the attention mechanism was designed to mitigate the effects of erroneous feature fusion to a certain extent.The experi-mental results demonstrate that the proposed network can achieve a favorable balance between video super-resolution performance and network complexity,and the 4-fold super-resolution reaches 8 frames per sec-ond on the benchmark dataset SPMCS-11.The proposed network meets the requirements for fast and ac-curate reasoning operations on edge devices.

夏振平;陈豪;张宇宁;程成;胡伏原

苏州科技大学 电子与信息工程学院,江苏 苏州 215009||江苏省工业智能低碳技术工程中心,江苏 苏州 215009苏州科技大学 电子与信息工程学院,江苏 苏州 215009东南大学 电子科学与工程学院 显示技术研究中心,江苏 南京 210096||新型显示与视觉感知石城实验室,江苏 南京 210013

计算机与自动化

视频超分辨率深度学习三维卷积神经网络特征融合

video super-resolutiondeep learning3D Convolutional Neural Networkfeature fusion

《光学精密工程》 2024 (016)

2564-2576 / 13

国家自然科学基金资助项目(No.62002254);江苏省自然科学基金资助项目(No.BK20200988);苏州市科技计划项目(No.SNG-2023002)

10.37188/OPE.20243216.2564

评论