NM-SpMM:面向国产异构向量处理器的半结构化稀疏矩阵乘算法OA北大核心CSTPCD

Deep neural networks have achieved excellent results in natural language processing,com-puter vision and other fields.Due to the growth of the scale of data processed by intelligent applications and the rapid development of large models,the inference performance of deep neural networks is in-creasingly demanding.N∶M semi-structured sparse scheme has become one of the hot technologies to balance the computing power demand and application effect.The domestic heterogeneous vector proces-sor FT-M7032 provides more space for data parallelism and instruction parallelism development in intel-ligent model processing.In order to address the challenges of N∶M semi-structured sparse model com-putation with various sparse patterns,a flexible configurable sparse matrix multiplication algorithm NM-SpMM is proposed for FT-M7032.NM-SpMM designs an efficient compressed offset address sparse encoding format COA,which avoids the impact of semi-structured parameter configuration on sparse da-ta access.Based on the COA,NM-SpMM performs fine-grained optimization of sparse matrix multipli-cation in different dimensions.The experimental results on FT-M7032 single core show that NM-SpMM can obtain 1.73～21.00 times speedup compared to dense matrix multiplication,and 0.04～1.04 times speedup compared to NVIDIA V100 GPU with CuSPARSE.