深圳大学学报（理工版）2024，Vol.41Issue(3)：264-273,10.DOI:10.3724/SP.J.1249.2024.03264
基于动态步长交替方向乘子法正则化极限学习机
Regularized extreme learning machine based on variable step alternating direction method of multipliers
摘要
Abstract
To address the deficiency of slow convergence rate and stagnation of error decay during later iteration of alternating direction method of multipliers(ADMM)for regularized extreme learning machine(RELM),we propose a dynamic step size ADMM-based RELM algorithm denoted as VAR-ADMM-RELM.This method iterates with dynamically decaying step sizes based on the ADMM algorithm and simultaneously constrains the model complexity using both L1 and L2 regularization,such that the calculated output weight of ELM exhibited greater sparsity and robustness.We conduct fitting,classification,and regression comparative experiments with ELM,RELM,and ADMM-based L1 regularized ELM(ADMM-RELM)on UCI and MedMNIST datasets.The results show that VAR-ADMM-RELM improves the average classification accuracy and average regression prediction by 1.94%and 2.49%,respectively,compared to ELM.It achieves a speedup of 3 to 5 times compared to the standard ADMM algorithm and exhibits better robustness and generalization capabilities against outliers.Furthermore,it approaches the modeling efficiency of standard ELM in high-dimensional multi-sample scenarios.The proposed algorithm effectively enhances the convergence rate of the ADMM algorithm and achieves superior performance compared to mainstream ELM algorithms.关键词
人工智能/机器学习/极限学习机/交替方向乘子法/正则化/动态衰减Key words
machine learning/extreme learning machine/alternating direction method of multipliers/regularization/dynamic decay分类
信息技术与安全科学引用本文复制引用
卢辉煌,邹伟东,李钰祥..基于动态步长交替方向乘子法正则化极限学习机[J].深圳大学学报（理工版）,2024,41(3):264-273,10.基金项目
National Natural Science Foundation of China(61906015) （61906015）
Natural Science Foundation of Beijing(L201004) 国家自然科学基金资助项目(61906015) （L201004）
北京市自然科学基金资助项目(L201004) （L201004）
