管理工程学报2024，Vol.38Issue(3)：213-226,14.DOI:10.13587/j.cnki.jieem.2024.03.016
基于LSTM和多头注意力机制的企业违约预测模型
Enterprise default prediction model based on LSTM and multi-head attention
摘要
Abstract
Default prediction refers to the use of the data and default state of the company in the past to predict the future probability of default of the company.Default prediction is extremely important for stock investment,bond investment and bank loans.This research involves two scientific issues:one is how to use continuous years of corporate data to predict the default probability,and the other is to study the impact of each time window of the input default prediction model on the default prediction state. In this paper,the default prediction model based on the LSTM network uses continuous years of corporate data to predict the probability of default,which has changed the current situation that only one year of data is used for default prediction modeling.In order to explore the impact of each time window on the default prediction value,this paper first applies the multi head attention mechanism to the default prediction model.This study selects the data of listed companies from 2000 to 2019 as an empirical sample.Each sample of listed companies has 542 indicators,including financial indicators,non-financial indicators and macroeconomic indicators. In order to obtain the most suitable default prediction model for Chinese listed companies based on LSTM and multi-head attention mechanism,this paper has carried out multiple verifications on the key hyper parameters involved in the modeling.Further,in order to better analyze the impact of each structure in the model on the accuracy of default prediction,this paper conducts ablation analysis on the default prediction model built,that is,starting from the structure corresponding to the best performance of the model,and gradually removing the neural network where these structures are located Layer,observe the changes in the accuracy of the algorithm.Finally,in order to study the degree of influence of each time window on the default prediction value,this paper visualizes the output results of the LSTM layer,the attention matrix and the weights of the fully connected layer. The results are as follows:1)It is more reasonable to consider the time series of enterprise data when modeling default prediction,and the use of time series data modeling can help improve the accuracy of the default prediction.2)Through the study of the multi-head attention matrix,it is found that the data of different time windows have different effects on the default prediction results,the same time window has different effects on the default prediction results of different samples,and the sample information captured by different attention heads is different.3)The optimal number of time windows for default prediction can be a number between 5 and 10.Generally speaking,the more time windows,the higher the accuracy of default prediction.4)Model ablation experiments show that the default prediction model built in this paper effectively reduces the second type of error in the default prediction results and reduces the risk of bad customers being predicted as good customers. This paper introduces the LSTM model and the multi-head attention mechanism into the system of the research theory of default prediction of listed companies,to achieve the purpose of predicting the default probability of companies with data for many years,and to a certain extent improve the theory of the research of default prediction of listed companies.The default prediction model established in this paper can provide risk warning information for the market and enterprises,promote the healthy development of the capital market,and prompt enterprises to solve existing problems in a timely manner.The prediction results of the establishment of the default prediction model in the article can also provide a basis for decision-making in the investment activities of institutions or individuals.关键词
长短期记忆神经网络/多头注意力机制/违约预测Key words
LSTM neural network/Multi-head attention/Default prediction分类
管理科学引用本文复制引用
柏凤山,迟国泰,温武军..基于LSTM和多头注意力机制的企业违约预测模型[J].管理工程学报,2024,38(3):213-226,14.基金项目
国家自然科学基金重点项目(71731003) （71731003）
国家自然科学基金项目(72071026、72173096) The Key Programs of the National Natural Science Foundation of China(71731003) （72071026、72173096）
The National Natural Science Foundation of China(72071026,72173096) （72071026,72173096）
