计算机应用与软件2017，Vol.34Issue(2)：35-41,7.DOI:10.3969/j.issn.1000-386x.2017.02.006
适应冷热数据存储的多编码架构的设计与实证
DESIGN AND DEMONSTRATION OF MULTI-CODES FRAMEWORK FOR COLD/HOT DATA STORAGE
摘要
Abstract
With the rapid development of the Internet and the explosive growth of data,large-scale distributed storage systems are widely used in Internet application.Recent Internet applications usually involve different types of data,and data can be considered as hot data or cold data based on their access frequency.However,a storage system with erasure codes is generally implemented with a fixed coding mechanism,which cannot adapt well to the diverse types of data coexisting in the same system.As a result,the system performance may greatly degrade.Thus,a new storage system framework is suggested to improve the system performance based on multiple codes,considering the difference between hot and cold data.For cold data,it can adopt a low-redundancy coding mechanism to improve space efficiency.For hot data,in contrast,it can reduce the data access time by taking a code that can be rapidly decoded.Then,real-world implementations of such a framework based on HDFS-RAID are designed,which is deployed in a Hadoop tested cluster.Besides,based on a real-world data access trace,the effectiveness of our system in improving the system performance is verified.The results show that the system can adapt well to the diverse types of data.关键词
分布式数据存储/HDFS/编码存储Key words
Distributed data storage/HDFS/Encoding storage分类
信息技术与安全科学引用本文复制引用
魏学才,宫庆媛,沈佳杰,周扬帆,王新..适应冷热数据存储的多编码架构的设计与实证[J].计算机应用与软件,2017,34(2):35-41,7.基金项目
国家自然科学基金项目(61571136) （61571136）
上海市“科技创新行动计划”项目(14511101000) （14511101000）
综合业务网理论及关键技术国家重点实验室开放研究课题(ISN15-08). （ISN15-08）
