数据采集与处理2025，Vol.40Issue(3)：637-646,10.DOI:10.16337/j.1004-9037.2025.03.006
武信:一种垂直领域大语言模型系统架构设计与实证
Wuxin:Architecture Design and Empirical Study for Vertical-Domain Large Language Model System
摘要
Abstract
In customized scenarios,it is urgent to enhance the understanding and generation capabilities of large language models(LLMs)in specific vertical domains.We propose a paradigm for developing vertical-domain LLM system named"Wuxin",which covers a series of development methods for LLM systems,including architecture,data,model,and training.Wuxin utilizes human-in-the-loop data augmentation to improve the quality of military training injury question and answer datasets,and employs the GaLore strategy to perform efficient full-parameter fine-tuning on small LLMs.Experimental results show that the adopted full-parameter fine-tuning method outperforms LoRA fine-tuning in terms of convergence and accuracy.Furthermore,Wuxin demonstrates significant advantages in understanding professional military training injury knowledge,as well as overcoming hallucinations.Our achievements can provide references for the design and application of question-answering LLM systems in vertical domains.关键词
数据增强/大语言模型系统/全参微调/垂直领域大模型Key words
data augmentation/large language model system/full-parameter fine-tuning/vertical-domain large model分类
计算机与自动化引用本文复制引用
朱新立,高志强,姬纬通,李少华,李松杰..武信:一种垂直领域大语言模型系统架构设计与实证[J].数据采集与处理,2025,40(3):637-646,10.基金项目
国家社会科学基金(2022-SKJJ-C-093) （2022-SKJJ-C-093）
武警部队科技创新团队创新研究项目(ZZKY20222103). （ZZKY20222103）
