초록 열기/닫기 버튼
There is a lack of research on the use of the AI voice chatbot in speaking tests. This study compared ESWT (English Speaking & Writing Test) and CEST (Chatbot-based English Speaking Test) scores, analyzed the impact of interactional competence on the wholistic scores of CEST, and examined test-takers’ perceptions of item difficulties. Twenty-seven Korean university students, primarily at novice English language proficiency levels, participated in the study. They took both ESWT and CEST followed by surveys asking their self-perceived item difficulty for each test. The results of paired ttest showed that ESWT yielded higher mean scores across all analytical constructs and holistic scores with significant differences in pronunciation and coherence. A Ridge regression revealed that the interactivity and fluency constructs of CEST strongly influenced its wholistic score, highlighting the potential of AI chatbots to enhance EFL learners' speaking abilities. Participants, however, perceived CEST as more challenging than ESWT, pointing out the need for more preparation and response time due to their low proficiency levels. Regarding item difficulty, read-aloud and selfintroduction tasks were considered the easiest, while opinion-giving and long consecutive questions were the most challenging in ESWT. Participants also found all items challenging overall, reflecting their limited language competence and underscoring the necessity of targeted speaking skill development. Based on these findings, the study presents discussions and practical implications for integrating AI chatbots into speaking assessment.