基于图划分的分布式推荐系统OA
A Distributed Recommender System Based on Graph Partition
[目的]设计一个数据处理效率高的推荐系统具有重要的意义.[方法]使用图结构来模拟推荐系统中的用户偏好关系,将其通过图划分算法处理,可以更深层次地挖掘推荐系统中数据的信息价值,并将得到的负载均衡的子图数据作为分布式系统的输入,最终经过一个自适应聚合模块的融合实现了一个分布式推荐系统.[结果]该系统可以提高推荐算法对于大规模数据的处理效率,在预测精度不下降的前提下,算法在一个由16个CPU构成的集群训练相比于单个CPU训练可提高6.4倍的效率.[结论]实验结果证明了该系统于推荐效率方面的有效性.
[Objective]It is of great significance to design a recommender system with high data process-ing efficiency.[Methods]The graph structure is used to simulate the user preference relation-ship in the recommender system.Through the graph partition algorithm processing,the infor-mation value of the data in the recommender system can be further mined,and the obtained sub-graph data with load balancing can be used as the input of the distributed system.Finally,a dis-tributed recommender system is implemented through the fusion of an adaptive aggregation module.[Results]The system can improve the processing efficiency of the recommender algo-rithm for large-scale data.On the premise that the prediction accuracy does not decline,the al-gorithm can improve the efficiency 6.4 times in a cluster training consisting of 16 CPUs com-pared with a single CPU training.[Conclusions]The experimental results show that the system is effective in rec-ommendation efficiency.
杨锦光;熊菲;顾峻瑜;席炜亭
北京交通大学,北京 100044中国科学院计算机网络信息中心,北京 100083||中国科学院大学,北京 100049华北电力大学,北京 100096
推荐系统图划分负载均衡分布式系统
recommender systemgraph partitionload balancingdistributed system
《数据与计算发展前沿》 2024 (005)
102-110 / 9
国家自然科学基金(61872033);国家自然科学基金(72004009);国家重点研发计划(2018YFC0832304);北京市科技新星计划(Z201100006820015)
评论