基于大语言模型增强表征对齐的小样本持续关系抽取方法OA北大核心CSTPCD
Large Language Model Augmentation and Feature Alignment Method for Few-Shot Continual Relation Extraction
关系抽取作为自然语言处理的关键任务,对于深化语言理解、构建知识图谱以及优化信息检索系统具有重要作用.然而,由于新关系不断涌现且缺乏大量标注示例,传统的监督学习方法并不适合实际场景.尽管大语言模型的出现显著提升了许多自然语言处理任务的性能,但仍然无法直接有效地解决小样本持续关系抽取任务的挑战.为了充分利用大语言模型的语义知识来缓解灾难性遗忘与过拟合问题,提出了一种基于大语言模型增强表征对齐的小样本持续关系抽取方法LAFA,通过关系实例改写、语义扩充和关系增强表征等策略,在保持数据量和计算成本较低的同时,有效提升了模型对新关系的适应性和对旧知识的保持能力.在两个关系抽取数据集FewRel、TACRED上进行实验验证,与现有方法相比,LAFA在小样本持续关系抽取任务中展现出较好的效果,尤其在增量阶段取得了最佳的实验结果.通过消融实验进一步揭示了方法中各个模块对整体性能的显著贡献.LAFA的推理效率与开销远远低于现有的基于大语言模型的方法,并且具有很强的扩展性,能够适配多种语言模型.
Relation extraction,as a key task in natural language processing,plays a significant role in deepening lan-guage understanding,constructing knowledge graphs,and optimizing information retrieval systems.However,tradi-tional supervised learning methods are not well-suited for real-world scenarios due to the continuous emergence of new relations and the lack of large annotated datasets.Although the advent of large language models has significantly improved the performance of many natural language processing tasks,they still cannot effectively address the chal-lenges of few-shot continual relation extraction.To fully leverage the semantic knowledge of large language models to mitigate catastrophic forgetting and overfitting issues,a novel few-shot continual relation extraction method,LAFA(large language model augmentation and feature alignment),is proposed.This method enhances representation align-ment through various strategies such as relation instance rewriting,semantic expansion,and enhanced relation repre-sentation.It effectively improves the model adaptability to new relations and the retention of old knowledge while maintaining low data and computational costs.Experimental validation on two relation extraction datasets,FewRel and TACRED,demonstrates that LAFA outperforms existing methods in few-shot continual relation extraction tasks,particularly achieving the best results in incremental stages.Ablation experiments further reveal the signifi-cant contributions of each module to overall performance.Moreover,the inference efficiency and cost of LAFA are substantially lower than those of existing large language model-based methods,and it boasts strong scalability,being able to adapt to various language models.
李逸飞;张玲玲;董宇轩;王佳欣;仲宇杰;魏笔凡
传播内容认知全国重点实验室,北京 100733||西安交通大学 计算机科学与技术学院,西安 710049西安交通大学 计算机科学与技术学院,西安 710049||陕西省大数据知识工程重点实验室,西安 710049
计算机与自动化
大语言模型(LLM)关系抽取持续学习小样本学习
large language model(LLM)relation extractioncontinual learningfew-shot learning
《计算机科学与探索》 2024 (009)
2326-2336 / 11
国家重点研发计划(2022YFC3303600);国家自然科学基金(62137002,62293553,62293554,62176207,62106190,62192781);传播内容认知全国重点实验室科研课题资助项目(A202402);陕西省自然科学基金(2023-JC-YB-593);陕西高校青年创新团队项目. This work was supported by the National Key Research and Development Program of China(2022YFC3303600),the National Natural Science Foundation of China(62137002,62293553,62293554,62176207,62106190,62192781),the Research Project of State Key Laboratory of Communication Content Cognition(A202402),the Natural Science Foundation of Shaanxi Province(2023-JC-YB-593),and the Youth Innovation Team Project of Shaanxi Universities.
评论