计算机技术与发展2024，Vol.34Issue(4)：168-173,6.DOI:10.20165/j.cnki.ISSN1673-629X.2024.0025
基于m×2正则化交叉验证的神经网络超参数调优方法
A Method for Hyper-parameter Tuning of Neural Network Based on m×2 Regularized Cross-validation
摘要
Abstract
Hyper-parameter tuning is a key issue in neural network modeling.From the viewpoint of the problems of traditional hyper-parameter tuning methods,we propose a hyper-parameter tuning method based on m×2 regularized cross-validation.The goal is to present a robust hyper-parameter tuning method with low computational cost suitable for complex models and large datasets.The idea of the proposed method is to select a small number of data from the complete dataset for tuning,so as to avoid the time-consuming problem of hyper-parameter tuning when the dataset is large.Then,on the basis of m×2 cross-validation,regularization is adopted to balance the distribution difference between the training set and the validation set to reduce the performance fluctuation caused by the distribution in-consistency.The signal-to-noise ratio is used as the metric of hyper-parameter tuning,so that the mean and variance of the model per-formance can be comprehensively considered.The orthogonal design is used to select a combination of hyper-parameters with low correlation to improve the tuning efficiency.The experimental results on the CoNLL 2003 dataset show that the proposed method can obtain a combination of hyper-parameters that is not significantly different from the grid search,and the tuning time can be significantly reduced by about 66％.关键词
m×2交叉验证/正则化/神经网络/超参数调优/信噪比Key words
m×2 cross-validation/regularization/neural network/hyper-parameter tuning/signal-to-noise分类
信息技术与安全科学引用本文复制引用
曹学飞,杨帆,李济洪,王瑞波,牛倩..基于m×2正则化交叉验证的神经网络超参数调优方法[J].计算机技术与发展,2024,34(4):168-173,6.基金项目
国家自然科学基金(61806115,62076156) （61806115,62076156）
