微型电脑应用2023，Vol.39Issue(11)：1-4,4.
地平线扫描数据消重去噪系统设计
Design of Horizon Scan Data Deduplication Denoising System
摘要
Abstract
Combined with the multi-source heterogeneous characteristics of horizon scanning data,aimed at the problems of data duplication and data noise,the duplicate data detection method is selected to detect and delete duplicate data and noise data,based on variable length data block and the noise data detection method and TF-IDF.Using the idea of SO A archi tecture,the horizon scan data deduplication denoising system is designed using Java.By the system to preprocess the horizon scan data,it can effectively improve the proportion of high-quality data and provide data support for subsequent industrial analysis and tech-nical identification.关键词
地平线扫描数据/重复数据/噪声数据/多源异构数据Key words
horizon scan data/duplicate data/noise data/multi-source heterogeneous data分类
信息技术与安全科学引用本文复制引用
鄢天安,张文强,吴思,张英杰..地平线扫描数据消重去噪系统设计[J].微型电脑应用,2023,39(11):1-4,4.基金项目
国家重点研发计划项目课题(2019YFA0707202) （2019YFA0707202）
