电测与仪表2017，Vol.54Issue(14)：33-38,6.
基于Spark框架的电力大数据清洗模型
A data cleaning model for electric power big data based on Spark framework
摘要
Abstract
Aiming at the difficulties of the extracting of the unified anomaly detection pattern and the low accuracy and continuity of the anomaly data correction in the process of the electrical power big data cleaning, the data cleaning model of the electrical power big data based on Spark framework is proposed. Firstly, the normal clusters and the corresponding boundary samples are obtained by the improved CURE clustering algorithm. Then, the anomaly data identification algorithm based on boundary samples is designed. Finally, the anomaly data modification is realized by using exponential weighting moving mean value. The high efficiency and accuracy are proved by the experiment of the data cleaning of the wind power generation monitoring data from the wind power station.关键词
电力大数据/数据清洗/异常识别/异常修正/Spark框架Key words
big data of electric power/data cleaning/anomaly identification/anomaly modification/Spark framework分类
信息技术与安全科学引用本文复制引用
王冲,邹潇..基于Spark框架的电力大数据清洗模型[J].电测与仪表,2017,54(14):33-38,6.基金项目
国家自然科学基金资助项目(51277023) （51277023）
