现代信息科技2026，Vol.10Issue(18)：65-71,79,8.DOI:10.19850/j.cnki.2096-4706.2026.18.012
基于REALM与LoRA融合的审计专业问答大模型研究
Research on Audit Professional Question-Answering Large Model Based on the Integration of REALM and LoRA
摘要
Abstract
To address the issues of general large language models in audit applications—such as deviations in understanding professional terminology,insufficient dynamic adaptation to regulatory standards,and high fine-tuning costs—this paper proposes a construction scheme for an audit professional question-answering large model integrating Retrieval-Augmented Language Model Pre-training(REALM)and Low-Rank Adaptation(LoRA)fine-tuning.By integrating industry data including audit reports,laws and regulations,and professional standards,a dedicated dataset containing 10,000 high-quality audit question-answering dialogues is constructed.The REALM retrieval-augmented pre-training mechanism enables modular storage and accurate retrieval of audit knowledge during the pre-training phase.Qwen-7B-Chat serves as the base model,with LoRA technology employed for parameter fine-tuning;performance differences among full LoRA,QLoRA-8bit,and QLoRA-4bit quantization schemes are also evaluated.Experimental results show that after REALM pre-training and LoRA fine-tuning,Qwen-7B-Chat achieves 26.34,48.79,29.87,and 42.51 on the BLEU-4,ROUGE-1,ROUGE-2,and ROUGE-L metrics,respectively,improving by 9.5％,6.9％,8.9％,and 8.3％over the single LoRA fine-tuned model.The audit qualitative accuracy reaches 94.2％,the precision of regulatory citation increases to 93.7％,and the training computing power demand is reduced by more than 78％.This model demonstrates good accuracy and professionalism in audit professional knowledge question-answering,providing a lightweight and cost-effective technical approach for assisting auditors in decision-making,professional training,intelligent compliance review,and other scenarios.关键词
审计智能化/大语言模型/REALM预训练/LoRA微调/数据集构建Key words
intelligent auditing/Large Language Model/REALM pre-training/LoRA fine-tuning/dataset construction分类
信息技术与安全科学引用本文复制引用
苏星语,陈稳,胡汪昊,刘敏..基于REALM与LoRA融合的审计专业问答大模型研究[J].现代信息科技,2026,10(18):65-71,79,8.基金项目
江苏省高等学校大学生创新创业计划国家级大学生创新计划项目(X2025112870021) （X2025112870021）
