计算机技术与发展2023，Vol.33Issue(12)：49-57,9.DOI:10.3969/j.issn.1673-629X.2023.12.007
基于词嵌入的元组级数据溯源方法
A Tuple-level Data Lineage Approach Based on Word Embedding
摘要
Abstract
In the era of information explosion,the volume of data is increasing day by day,and data mining technology can be used to explore its inner connection,but only if the data used is correct,otherwise all the subsequent work will be meaningless.Data lineage tech-nology can help data analysts quickly locate the source and processing process of erroneous data,reduce the time and difficulty of analyzing erroneous data,and has important value for data quality control and trustworthy management.The existing tuple-level data lineage methods have the problems of high storage overhead and low lineage efficiency,and we use word embedding technology to improve the tuple-level data lineage methods.Firstly,the tuple vectorization encoding mechanism is investigated and tuple lineage rela-tionships based on the similarity of tuple vectors is identified.Secondly,an optimization algorithm based on attribute importance is proposed to improve the precision of lineage.Thirdly,the approximate nearest neighbor search and tuple filtering optimization mechanism is used to reduce the lineage time complexity.Finally,a directed acyclic graph is used to show the lineage relationships of tuple data.The experiment shows that the proposed method has higher lineage precision,lower time complexity and less storage consumption,and can ef-fectively improve the tuple-level data lineage method.关键词
结构化数据/数据溯源/元组向量/相似度比较/词嵌入Key words
structured data/data lineage/tuple vectors/similarity comparison/word embedding分类
信息技术与安全科学引用本文复制引用
杨彬,高俊涛,王志宝,李菲,马强,江树涛..基于词嵌入的元组级数据溯源方法[J].计算机技术与发展,2023,33(12):49-57,9.基金项目
国家自然科学基金资助项目(61902222) （61902222）
东北石油大学优秀中青年科研创新团队培育基金(KYCXTDQ202101) （KYCXTDQ202101）
