南通大学学报（自然科学版）2024，Vol.23Issue(3)：10-22,13.DOI:10.12194/j.ntu.20230919001
基于时间特征细化网络的时空视频超分辨率研究
Space-time video super-resolution based on temporal feature refinement network
摘要
Abstract
Space-time video super-resolution(STVSR)enhances video quality across both temporal and spatial dimensions,enabling real-time presentation of high-resolution and high-frame-rate videos despite limitations in video capture devices,transmission,or storage,thus meeting the demand for ultra-high-definition image quality.Compared to two-stage methods,one-stage approaches achieve frame interpolation at the feature level rather than the pixel level,significantly outperforming in terms of inference speed and computational complexity.Some existing one-stage STVSR methods employ pixel hallucination-based feature interpolation,which struggles to predict fast-moving objects between frames.To address this,a pyramid encoder-decoder network based on optical flow for temporal feature inter-polation is proposed,to achieve rapid bidirectional optical flow estimation and more realistic texture synthesis.This network structure,termed temporal feature refinement network(TFRnet),enhances efficiency while mitigating the insta-bility of optical flow estimation for large motions.Additionally,the spatial module incorporates sliding window-based local propagation and bidirectional propagation based on recurrent networks to strengthen frame alignment.To further exploit TFRnet's potential,spatial super-resolution is prioritized over temporal super-resolution(space-first approach).Experiments on several widely used data benchmarks and evaluation metrics demonstrate the excellent performance of our proposed method,TFRnet-sf.While improving overall peak signal-to-noise ratio(PSNR)and structural similarity index(SSIM),it also enhances PSNR and SSIM for inserted intermediate frames,alleviating to some extent the issue of significant disparities in PSNR and SSIM between inserted intermediate frames and original frames.关键词
时空视频超分辨率/金字塔编码器-解码器网络/时间特征插值/空-时超分策略/深度学习Key words
space-time video super-resolution/pyramid encoder-decoder network/temporal feature interpolation/space-first strategy/deep learning分类
信息技术与安全科学引用本文复制引用
姚晓娟,穆柯,潘沛,杨紫伊,赵雨飞,朱永贵..基于时间特征细化网络的时空视频超分辨率研究[J].南通大学学报（自然科学版）,2024,23(3):10-22,13.基金项目
中央高校基本科研业务费专项资金项目(CUC2019A002,CUC2019B021) （CUC2019A002,CUC2019B021）
