효율적 AI 피드백 설계를 위한 LLM 성능 비교 연구: 대학생의 설득적 글쓰기 과제를 중심으로

초록

This study compares the feedback profiles produced by ChatGPT and Gemini on the same undergraduate persuasive-writing task under an identical prompt. Using an RTPR-based LLM-as-a-Judge framework with counter-balancing, we tested seven rubric criteria and the overall mean via independent two-sample t-tests. Significant differences emerged for most criteria and for the overall mean (p<.001); Lexical Accuracy favored ChatGPT over Gemini (p=.017), whereas Sentence Clarity was not significant (p=.084). These findings indicate systematic, model-specific strengths and weaknesses even under identical inputs, suggesting instructionally targeted model choice (e.g., argument/evidence/organization vs. lexical refinement). The study was limited to two models and a single genre; no multiple-comparison correction was applied.

키워드