计算机应用与软件2018，Vol.35Issue(5)：15-20,94,7.DOI:10.3969/j.issn.1000-386x.2018.05.003
一种基于LRFU缓存替换策略的HDFS客户端本地缓存设计与实现
DESIGN AND IMPLEMENTATION OF AN HDFS CLIENT LOCAL CACHE BASED ON LRFU CACHE REPLACEMENT POLICY
摘要
Abstract
Hadoop distributed file system(HDFS)is a distributed file system based on Master /Slave master-slave architecture.NameNode manages the metadata of all files and handles client-side access to files.As application data continues to increase, as DataNodes continue to be added, the workload of NameNode maintaining file metadata and handling file operations such as read and delete of Hadoop applications increases.The performance of NameNode becomes the bottleneck of the entire HDFS.In this paper,we proposed an idea that some of the NameNode tasks should be submitted to the HDFS client so that each HDFS client became a NameNode on some functions.Based on the least recently used and least frequently used(LRFU)cache replacement strategy,a solution to establish a local cache on the HDFS client was proposed to reduce the NameNode load.Experiments show that Hadoop applications read file block information from the cache without having to request NameNode, which improves data access efficiency and reduces NameNode load.关键词
HDFS/主从架构/LRFU/客户端/缓存Key words
HDFS/Master/Slave architecture/LRFU/Client Cache分类
信息技术与安全科学引用本文复制引用
谢磊,吴建华,廖卓凡,罗可..一种基于LRFU缓存替换策略的HDFS客户端本地缓存设计与实现[J].计算机应用与软件,2018,35(5):15-20,94,7.基金项目
国家自然科学基金项目(61402056). （61402056）
