电测与仪表2018，Vol.55Issue(2)：39-44,6.
基于Spark框架的能源互联网电力能源大数据清洗模型
Big energy data cleaning model for energy internet based on Spark framework
摘要
Abstract
Big energy data cleaning can improve the accuracy,completeness,consistency,and reliability of the quality of energy large data.Big data for energy extraction cleaning process is difficultly unified the anomaly detection mode,and its continuous and abnormal data correction accuracy is low and other issues,we proposed a framework based Spark energy clean energy large data model.Firstly,we obtain the normal cluster based on the improved CURE clustering algorithm;Secondly,boundary sample acquisition method of a normal cluster is achieved,and the anomaly recognition algorithm based boundary samples is designed;Finally,the abnormal data corrected is realized by the index weighted moving average.The efficiency and accuracy of the cleaning model are verified by analyzing the experimental data cleansing of wind power generation monitoring data in a wind farm.关键词
能源大数据/数据清洗/异常识别/异常修正/Spark框架Key words
big energy data/data cleaning/abnormal recognition/abnormal correction/Spark framework分类
能源科技引用本文复制引用
曲朝阳,张艺竞,王永文,赵莹..基于Spark框架的能源互联网电力能源大数据清洗模型[J].电测与仪表,2018,55(2):39-44,6.基金项目
国家自然科学基金资助项目(51277023) （51277023）
吉林省科技计划重点转化项目(20140307008GX) （20140307008GX）
