现代教育技术2026，Vol.36Issue(3)：62-71,10.DOI:10.3969/j.issn.1009-8097.2026.03.007
大语言模型作文评价反馈质量的实证分析
An Empirical Analysis on the Feedback Quality in Essay Evaluation by Large Language Models
摘要
Abstract
At present,large language model(LLM)povide an optimized opportunity for intelligent essay evaluation with their with advanced text processing capabilities.However,their feedback quality and semantic analysis ability remain insufficient.Issues such as superficial theme and over-polishing often reveal that LLM feedback cannot yet meet the core evaluation requirements of Chinese essay focusing on theme and expression.Therefore,it is necessary to evaluate their feedback quality.Therefore,this paper first developed a quality evaluation standard for LLM essay evaluation feedback through literature review and expert consultation.Ten LLM were used to evaluate 50 narrative and 50 argumentative essays respectively according to the given essay scoring criteria.Then,the paper scored the LLM feedback based on the standard,calculated inter-rater reliability,and conducted descriptive analysis and qualitative summary of the results.The study found out that LLM's feedback had room for improvement in stylistic stability,richness and in-depth understanding;LLM's feedback lacked teacher-like abilities in guiding thinking,critical evaluation and judging linguistic literary grace;LLM was good at providing feedback on theme,material,structure,emotion and ideology;LLM's feedback showed a certain degree of disciplinary professionalism.Finally,suggestions were put forward based on the findings,aiming to provide a reference for promoting the in-depth integration of LLM and Chinese language essay evaluation.关键词
大语言模型/作文评价/反馈质量/智能作文评价Key words
large language model/essay evaluation/feedback quality/automated writing evaluation分类
社会科学引用本文复制引用
瞿锦雯,叶丽新,孙潘懿,徐悦,吴蓓蕾..大语言模型作文评价反馈质量的实证分析[J].现代教育技术,2026,36(3):62-71,10.基金项目
本文为中央高校基本科研业务费项目华东师范大学哲学社会科学"数智化语文教育创新团队"项目(项目编号:2024QKT004)的阶段性研究成果. （项目编号:2024QKT004）
