tailieunhanh - Báo cáo khoa học: "Toward Evaluation of Writing Style: Finding Overly Repetitive Word Use in Student Essays"

Automated essay scoring is now an established capability used from elementary school through graduate school for purposes of instruction and assessment. Newer applications provide automated diagnostic feedback about student writing. Feedback includes errors in grammar, usage, and mechanics, comments about writing style, and evaluation of discourse structure. This paper reports on a system that evaluates a characteristic of lower quality essay writing style: repetitious word use. | Toward Evaluation of Writing Style Finding Overly Repetitive Word Use in Student Essays Jill Burstein Educational Testing Service Princeton New Jersey 08541 USA jburstein@ Magdalena Wolska Universităt des Saarlandes Saarbiicken Germany magda @ coli .uni- sb. de Abstract Automated essay scoring is now an established capability used from elementary school through graduate school for purposes of instruction and assessment. Newer applications provide automated diagnostic feedback about student writing. Feedback includes errors in grammar usage and mechanics comments about writing style and evaluation of discourse structure. This paper reports on a system that evaluates a characteristic of lower quality essay writing style repetitious word use. This capability is embedded in a commercial writing assessment application Criterion8 . The system uses a machine-learning approach with word-based features to model repetitious word use in an essay. System performance well exceeds several baseline algorithms. Agreement between the system and a single human judge exceeds agreement between two human judges. 1 Introduction Automated evaluation of student essay writing is a rapidly growing field. Over the past few years at least four commercially automated essay scoring systems have been made available PEG Page 1966 e-raier Burstein et al. 1998 Intelligent Essay Assessor Foltz Kintsch and Landauer 1998 and Intellimetric Elliot 2003 . In addition based on the demands of users of the automated scoring technology tools have been developed that perform more detailed evaluations of student writing. One such application is Critique Writing Analysis Tools. Critique and erater are embedded in a broader writing instruction application Criterion8 Online Essay Evaluation see http . Critique performs a number of evaluations on a student essay related to errors in grammar Chodorow and Leacock 2000 usage and mechanics comments on style and analysis of essay-based .