南京大学学报（自然科学版）2009，Vol.45Issue(5)：585-592,8.
支持向量回归多参数的同时调节
Simultaneous tuning of multiple parameters for support vector regression
摘要
Abstract
Parameter tuning is fundamental for support vector regression (SVR). There are three types of parameters we focus on. The first is the insensitive factor ε. SVR uses the e-insensitive loss function which does not penalize errors below some ε. The second is the penalty factor C, which is a compromise between the model complexity and the empirical risk. The third is the kernel function parameter, usually, the radius basis function is considered, so the parameter is σ. Previous tuning methods mainly adopted a nested two-layer optimization framework. In this framework, the inner layer optimizes the Lagrange multipliers α, and the outer layer makes use of these Lagrange multipliers to optimize penalty factors C, insensitive factors ε and kernel parameters σ. The parameters and hyperparameters were trained alternately, which directly led to high computational complexity. To solve this problem, we propose a simultaneous tuning model for multiple parameters of SVR. First, we combine Lagrange multipliers, penalty factors, insensitive factors and kernel parameters into one vector, and derive a new optimization formula for SVR, which converts the two separate tuning processes into one optimization process.Then, we transform the optimization formula into one unconstraint multivariate optimization problem through sequential unconstrained minimization technique (SUMT). Based on these theoretical results, we design, analyze and implement an algorithm for the simultaneous tuning model with variable metric method (VMM). Finally, by experiments on benchmark databases, we verify the convergence of the simultaneous tuning algorithm, and compare the accuracy and efficiency of the algorithm with that of common tuning algorithms. Theoretical and experimental results show that the simultaneous tuning model is valid and efficient.关键词
支持向量回归/模型选择/参数调节/同时调节Key words
support vector regression/ model selection/ parameter tuning/ simultaneous tuning分类
信息技术与安全科学引用本文复制引用
廖士中,丁立中,贾磊..支持向量回归多参数的同时调节[J].南京大学学报（自然科学版）,2009,45(5):585-592,8.基金项目
国家自然科学基金(60678049),天津市自然科学基金(07JCYBJC14600) （60678049）
