基于Spark和NRSCA策略的并行深度森林算法OACSTPCD

Aiming to address several issues encountered by parallel deep forest algorithms in big data environments,such as excessive redundancy and irrelevant features,low utilization rate of features at both ends,slow model convergence speed,and low parallel efficiency of cascading forests,this paper proposed a parallel deep forest algorithm based on Spark and NRSCA strategy(PDF-SNRSCA).Firstly,the algorithm proposed a feature selection strategy(FS-NRS)based on neighborhood rough sets and Fisher score,which measured the correlation and redundancy of features to effectively reduce the number of redundant and irrelevant features.Secondly,it proposed a scanning strategy based on random selection and equidistant extraction(S-RSEE)to ensure that all features were utilized with the same probability and solved the problem of low utilization rate of two ends in multi-granularing scanning.Finally,combining with the Spark framework,the algorithm realized the parallel trai-ning of cascading forests,and it proposed a feature filtering mechanism based on the importance index(FFM-Ⅱ)to balance the dimensions of enhanced class vectors and original class vectors,thereby accelerating the model convergence speed.Mean-while,the algorithm designed a task scheduling mechanism based on SCA(TSM-SCA)to redistribute tasks and ensure load balancing in the cluster,which solved the problem of low parallel efficiency of cascading forests.Experiments show that the PDF-SNRSCA algorithm can effectively improve the classification performance of deep forests and greatly enhance the efficien-cy of parallel training of deep forests.