基于弹性权重巩固的视频单曝光压缩成像算法研究OA
Video Snapshot Compressive Imaging Based on Elastic Weight Consolidation
[目的]为视频单曝光压缩成像(Snapshot Compressive Imaging,SCI)设计一种对原始压缩比例、调制掩模和测量分辨率等超参数具有较高鲁棒性的统一模型.[方法]本文基于弹性权重巩固(EWC)对所提出的模型进行训练,该模型具有结合了Transformer和卷积神经网络两种网络结构的特殊设计,在此基础上本文在初始化阶段引入广义交替投影进一步增加了模型对于不同掩码的鲁棒性.[结果]广泛的实验结果表明,本文提出的统一模型可以很好地适应不同的压缩比、调制掩膜和测量分辨率,同时实现了最先进的结果.实验结果在PSNR、SSIM方面表现优于之前的SOTA算法,其中平均PSNR涨幅超过5 dB.[局限]尽管本文提出的模型在适应性和平均PSNR指标上优于之前的SOTA算法,但引入了EWC的模型在特定单一任务上的结果可能不会优于针对该特定任务训练的模型.[结论]通过引入广义交替投影和EWC以及对于网络结构的特殊设计,本文提出的具有高度适应性的模型为解决其他复杂场景下的压缩感知重建任务提供了新的思路和方法.
[Objective]This work aims to design a unified model with high robust hyperparameters,in-cluding compression ratio,modulation mask and measurement resolution,for Snapshot Com-pressive Imaging(SCI).[Methods]We train the proposed model based on Elastic Weight Con-solidation(EWC).The model is uniquely designed by combining Transformer and Convolution-al neural network architectures.Additionally,during the initialization phase,we incorporate Generalized Alternating Projection to enhance the model's robustness to different masks.[Results]Extensive ex-perimental results demonstrate that our proposed unified model can well adapt to different compression ratios,modula-tion masks,and measurement resolutions while achieving state-of-the-art results.Our model outperforms previous SO-TA algorithms in terms of PSNR and SSIM,with an average PSNR improvement of over 5 dB.[Limitations]Al-though our model outperforms previous SOTA algorithms in terms of adaptability and average PSNR,the model with EWC may not perform better than a model specifically trained for a particular single task.[Conclusions]By introduc-ing Generalized Alternating Projection and EWC,as well as the special design of the network structure,our proposed highly adaptive model provides new ideas and methods for solving compressive sensing reconstruction tasks in other complex scenarios.
郑巳明;朱明宇;袁鑫;杨小渝
西湖大学工学院,浙江杭州 310024||中国科学院计算机网络信息中心,北京 100083中国科学院大学,北京 100049
单曝光压缩成像高光谱连续学习Transformer3D卷积
snapshot compressive imaginghyperspectralcontinual learningtransformer3D convolution
《数据与计算发展前沿》 2024 (005)
111-125 / 15
评论