Measuring Non-native Speakers’ Proficiency of English by Using a Testwith Automatically-Generated Fill-in-the-Blank Questions

摘要

This paper proposes the automatic generationof Fill-in-the-Blank Questions (FBQs) togetherwith testing based on Item Response Theory(IRT) to measure English proficiency. First, theproposal generates an FBQ from a given sentencein English. The position of a blank in thesentence is determined, and the word at thatposition is considered as the correct choice.The candidates for incorrect choices for theblank are hypothesized through a thesaurus.Then, each of the candidates is verified by usingthe Web. Finally, the blanked sentence, thecorrect choice and the incorrect choices survivingthe verification are together laid out toform the FBQ. Second, the proficiency of nonnativespeakers who took the test consisting ofsuch FBQs is estimated through IRT.Our experimental results suggest that:(1) the generated questions plus IRT estimatethe non-native speakers’ English proficiency;(2) while on the other hand, the test can becompleted almost perfectly by English nativespeakers; and (3) the number of questions canbe reduced by using item information in IRT.The proposed method provides teachersand testers with a tool that reduces timeand expenditure for testing English proficiency.