电测与仪表2018，Vol.55Issue(11)：1-6,6.
基于Hadoop的风力发电监测大数据存储优化及并行查询方法
Storage optimization and parallel query method for big data of wind power monitoring based on Hadoop
摘要
Abstract
With the extensive development of wind power generation and the generalized application of intelligent monitoring technology,wind power monitoring data shows the big data characteristics of large volume,multi types and fast growth.In order to solve the two major problems of big data with efficient storage and quick query,in this paper,the optimization method of big data storage is studied based on Hadoop platform.A Hash bucket algorithm considering wind power monitoring data correlation is proposed,which realizes the centralized storage of related data,so as to enhance the efficiency of data query and processing.On the basis of data storage optimization,the parallel association query for multi-source big data of wind power monitoring based on MapReduce is realized.Tests on a Hadoop platform show that the time of the multi-source data parallel association query is significantly shortened than traditional Hadoop method after optimization of hash bucket storage.关键词
大数据/风力发电监测/Hadoop/哈希分桶算法Key words
big data/wind power monitoring/Hadoop/Hash bucket algorithm分类
信息技术与安全科学引用本文复制引用
王林童,赵腾,张焰,苏运..基于Hadoop的风力发电监测大数据存储优化及并行查询方法[J].电测与仪表,2018,55(11):1-6,6.基金项目
国家高技术研究发展计划项目(863计划)(2015AA050203) （863计划）
国家电网公司科技项目(520900150037) （520900150037）
