计算机工程2017，Vol.43Issue(7)：22-28,37,8.DOI:10.3969/j.issn.1000-3428.2017.07.004
基于Spark的时态查询扩展与时态索引优化研究
Research on Temporal Query Expansion and Temporal Index Optimization Based on Spark
摘要
Abstract
There exists some temporal databases and temporal analysis tools based on cluster-based computing systems.However,most of them are disk-oriented and performance degenerate rapidly when processing big data.This paper proposes a system which is based on Spark,and provides accessible and scalable temporal query scheme with large temporal data for users.Specifically,it extends Spark SQL parser to support SQL-like temporal operations.Besides,it uses the index manager based on Spark SQL which is proposed by SIMBA,and embeds optimization strategies in two aspects:global filtering and local temporal index.Depending on these optimization rules,the system achieves high throughput and low latency in temporal operations.Evaluation experiment results on temporal query efficiency and effectiveness show this system has improved temporal query performance over original Spark SQL in different factors.关键词
时态大数据/Spark系统/SparkSQL组件/时态查询/时态索引/高吞吐量/低延迟Key words
temporal big data/Spark system/Spark SQL component/temporal query/temporal index/high throughput/low latency分类
信息技术与安全科学引用本文复制引用
周亮,李格非,邰伟鹏,郑啸..基于Spark的时态查询扩展与时态索引优化研究[J].计算机工程,2017,43(7):22-28,37,8.基金项目
安徽省高校自然科学研究重点项目"基于关键字的大规模地理数据查询方法研究"(KJ2015A310). （KJ2015A310）
