网络与信息安全学报2024，Vol.10Issue(5)：119-133,15.DOI:10.11959/j.issn.2096-109x.2024071
面向金融领域大语言模型的提示注入攻击防御机制研究及应用
Research and application of defense mechanism for prompt injection attack of large language model in financial industry
摘要
Abstract
The large language models had a broad application prospect in the financial sector,and they were expected to play an important role in both asset management and wealth management.With the rapid development and wide application of large language models such as ChatGPT and GPT-4,attention to the security of large language models increased.The financial industry,characterized by strict regulations and supervision,demanded heightened security measures.Consequently,a comprehensive study on prompt injection attacks and a security defense framework was delved into in large language models within the financial sector.A risk taxonomy encompassing eight forms of input prompt injection attacks and five categories of safety scenarios on the output side was developed,and a financial do-main large model prompt injection attack dataset,FIN-CSAPrompts,was collected using a human-machine adver-sarial approach.An end-to-end security defense framework against prompt injection attacks in large language models was proposed and tested,and comparative evaluations were performed using prevalent open-source large language models.The research indicated that in the financial industry,the application of the proposed security defense frame-work significantly enhanced the defensive capabilities of Chinese large language models,effectively reducing the generation of inappropriate content and improving their resilience against prompt injection attacks.This research pro-vided a reference and foundation for further research on the security issues of Chinese large language models in the fi-nancial domain,offering datasets,evaluation metrics,and solutions for consideration and adaptation.关键词
金融大语言模型安全/提示注入/风险分类体系/大模型数据集/法律风险检测Key words
financial large language model security/prompt injection/risk taxonomy/large model dataset/illegal risk detection分类
信息技术与安全科学引用本文复制引用
牟大恩,卫志华,孙铭隆,宋娜,倪琳..面向金融领域大语言模型的提示注入攻击防御机制研究及应用[J].网络与信息安全学报,2024,10(5):119-133,15.基金项目
国家自然科学基金(62376199) The National Natural Science Foundation of China(62376199) （62376199）
