曲阜师范大学学报（自然科学版）2024，Vol.50Issue(3)：89-95,7.DOI:10.3969/j.issn.1001-5337.2024.3.089
大型预训练语言模型基础逻辑能力测评研究
Research on the evaluation of basic logic ability of large-scale pre-trained language models
摘要
Abstract
For four basic logical reasoning abilities of quantity problem,set relationship,quantifier prob-lem and common sense reasoning,we construct few-shot learning sample templates for few-sort learning,which contain 11 logical reasoning subtasks.Two few-shot learning methods of in-context learning and prompt tuning are used to test the logical reasoning ability of GPT-Neo-1.3B and other models from the three dimensions of model,test method and task.The experimental results show that GPT-3 is relatively excellent in quantity problem,quantifier problem and common sense reasoning problem,GPT-Neo and GPT-J have more advantages in set-relation problem.Compared with in-context learning,the pre-trained models can significantly improve the prediction ability by prompt tuning.关键词
自然语言处理/预训练语言模型/语境学习/提示微调/少样本学习Key words
natural language processing/pre-trained language models/in-context learning/prompt-tun-ing/few-shot learning分类
信息技术与安全科学引用本文复制引用
倪睿康,肖达,高鹏..大型预训练语言模型基础逻辑能力测评研究[J].曲阜师范大学学报（自然科学版）,2024,50(3):89-95,7.基金项目
中国博士后科学基金(2023M732022) （2023M732022）
山东省自然科学基金(ZR2021QF061) （ZR2021QF061）
曲阜师范大学科研基金(167/602801). （167/602801）
