同济大学学报（自然科学版）2024，Vol.52Issue(3)：462-471,10.DOI:10.11908/j.issn.0253-374x.22219
面向周期性工业时序数据的流式清洗系统
Streaming Cleaning System for Periodic Industrial Time Series Data
摘要
Abstract
To efficiently clean industrial time series with the characteristics of periodicity,a streaming data cleaning system was first designed using distributed components.The system employs Mosquitto for data gathering,Flume for connection,and Kafka for the buffer,which provides benefits of high throughput and a large buffer.The data cleaning component serves as the core of the system.Then,a periodic time series cleaning algorithm was proposed based on a constraint model.Integrating the characteristics of temporality,periodicity,and physical meaning,the methods of periodic detection and data slicing were added to the original speed constraint algorithm,so as to solve the distortion problem of the original algorithm and improve the availability to deal with periodic data.Finally,the effectiveness of the system and the improved algorithm was verified using a tunnel boring machine data set as a case study.关键词
数据清洗/工业大数据/时序数据/速度约束/周期性Key words
data cleaning/industrial big data/time series data/speed constraint/periodic分类
信息技术与安全科学引用本文复制引用
王耀,赵炯,周奇才,熊肖磊,陈传林,张恒..面向周期性工业时序数据的流式清洗系统[J].同济大学学报（自然科学版）,2024,52(3):462-471,10.基金项目
上海申通地铁集团有限公司科研计划(JS-KY21R003-3) （JS-KY21R003-3）
