计算机技术与发展2017，Vol.27Issue(8)：1-6,6.DOI:10.3969/j.issn.1673-629X.2017.08.001
基于Spark的K-means安全区间更新优化算法
Optimization of K-means Updating Security Interval Based on Spark
摘要
Abstract
At each time when the K-means algorithm updates the cluster center,it needs to calculate iteratively the distance between all the points in the dataset with the latest clustering center to get the latest clustering of each point.This feature of global iterative computation leads to low efficiency of traditional K-means algorithm.As the data set increases,its time efficiency and clustering performance decrease too fast,so that the traditional K-means algorithm is not suitable for clustering in big data.Therefore,a new K-means secure interval updating algorithm based on Spark is proposed for time efficiency and performance optimization in big data.After updated the cluster center every time,it updates security interval label.According to whether the label is greater than 0 instead of calculation of the distance between all the points and the new center and cluster identification of all the data in the interval every time,which reduces the overhead of time and computation.The performance of the algorithm model based on the point vector model of Spark MLlib component has been optimized.It is made a comparison with the traditional K-means algorithm on average error criterion and operation time.The experimental results show that it is superior to the traditional K-means clustering algorithm in the above two indexes and is suitable for data clustering scenario in big data.关键词
K-means/安全区间/Spark/大数据/时间效率Key words
K-means/security interval/Spark/big data/time efficiency分类
信息技术与安全科学引用本文复制引用
李玉波,杨余旺,唐浩,陈光炜..基于Spark的K-means安全区间更新优化算法[J].计算机技术与发展,2017,27(8):1-6,6.基金项目
江苏省农业科技自主创新资金项目(CX(16)1006) （CX(16）
