南京信息工程大学学报2025，Vol.17Issue(2)：203-214,12.DOI:10.13878/j.cnki.jnuist.20230927001
基于梯度权值追踪的域自适应分类研究
Domain adaptive classification based on gradient weight pursuit
摘要
Abstract
Here,we propose a pruning and optimization approach based on Gradient Weight Pursuit(GWP)to ad-dress the overfitting in unsupervised domain,which manifests as significantly lower accuracy on downstream tasks compared to that on training sets.To tackle the overfitting challenge in unsupervised domain,we employ the dense-sparse-dense strategy,focusing on both difference-based and adversarial adaptive methods.First,the network is pre-trained intensively to identify crucial connections.Second,during the pruning stage,the optimization algorithm in this paper distinguishes itself from original dense-sparse-dense strategy by jointly considering both weight and gradient information.Specifically,it leverages both weight(i.e.zero-order information)and gradient(i.e.first-order informa-tion)to influence pruning process.In the final dense phase,the pruned connections are restored and the dense net-work is retrained with a reduced learning rate.Finally,the obtained network achieves desirable outcomes in down-stream tasks.The experimental results show that the proposed GWP approach can effectively improve the accuracy of downstream tasks,offering a plug-and-play capability compared with original difference-based and adversarial domain adaptation methods.关键词
梯度权值追踪/无监督领域自适应/稠密-稀疏-稠密/过拟合/零阶信息/一阶信息Key words
gradient weight pursuit(GWP)/unsupervised domain adaptation(UDA)/dense-sparse-dense(DSD)/overfitting/zero-order information/first-order information分类
计算机与自动化引用本文复制引用
崔绍君,季繁繁,王婷,袁晓彤..基于梯度权值追踪的域自适应分类研究[J].南京信息工程大学学报,2025,17(2):203-214,12.基金项目
科技创新2030—"新一代人工智能"重大项目(2018AAA0100400) （2018AAA0100400）
国家自然科学基金(U21B2049,61936005) （U21B2049,61936005）
