计算机科学与探索2024，Vol.18Issue(9)：2361-2369,9.DOI:10.3778/j.issn.1673-9418.2406067
面向工艺规范文本的大语言模型知识注入方法研究
Research on Knowledge Injection Method for Large Language Model Oriented to Process Specification Texts
摘要
Abstract
The application of large language models in process specifications is an effective approach to addressing the issue of inaccurate process knowledge queries.At present,the domain model construction methods through do-main knowledge graph embedding or fine-tuning with instruction data are not effective.The difficulty lies in the fact that the process knowledge in the process specifications involves relationships between multiple process elements,which is highly complex.The data are sparse because the standards are only used through citation.The high com-plexity of process knowledge and sparse data limit the model's ability to learn process domain concepts,the rela-tionships between concepts and attributes,the relationships between concepts,the relationships between multiple concepts,and reference-based knowledge.To address this difficulty,this paper proposes a large language model knowledge injection method for process specification texts.According to the characteristics of process specification data,this paper designs knowledge injection data including auxiliary sentence identification task,concept-chapter generation task,chapter continuation task and chapter-summary generation task.The model is fine-tuned through su-pervised learning by combining question-answer pair data to inject domain concepts,attributes,relationships be-tween multiple concepts,and reference knowledge into the model.Experimental results show that the model trained with knowledge injection data and question-answer pair data improves ACC(accuracy)by 7.3 percentage points,ROUGE-L by 7.4 percentage points,and BLEU-4 by 6.2 percentage points compared with the model trained only with question-answer pair data,indicating the effectiveness of the proposed knowledge injection method.关键词
工艺规范/大语言模型/知识注入/有监督微调Key words
process specification/large language model/knowledge injection/supervised fine-tuning分类
信息技术与安全科学引用本文复制引用
纪贵阳,王裴岩,余卓..面向工艺规范文本的大语言模型知识注入方法研究[J].计算机科学与探索,2024,18(9):2361-2369,9.基金项目
辽宁省应用基础研究计划(2022JH2/101300248) （2022JH2/101300248）
国家自然科学基金(U1908216). This work was supported by the Applied Basic Research Program of Liaoning Province(2022JH2/101300248),and the National Natu-ral Science Foundation of China(U1908216). （U1908216）
