In this study, general and prompt-specific models were again compared, but this time, general models performed as well as or better than prompt-specific models.

They reported empirical evidence that general scoring models performed nearly as well in agreeing with human readers as did prompt-specific models, the 'status quo' for most AES systems.

They reasoned that such models may enhance score credibility by signifying that an AES system measures the same writing characteristics across all essays. Powers, Burstein, Chodorow, Fowles, and Kukich (2002) suggested that automated essay scoring (AES) may benefit from the use of 'general' scoring models designed to score essays irrespective of the prompt for which an essay was written. Common criticisms of AES are defined and explicated-fundamental objections to the construct measured, methods used to measure the construct, and technical inadequacies-and a direction for future research is identified through a socio-cognitive approach to AES. The papers begins by defining writing as a construct and then turns to the e-rater scoring engine as an example of AES state-of-the-art construct measurement. However, since there is a relationship between ease of text production and ability to mobilize cognitive resources to address rhetorical and conceptual problems, AES systems have strong correlations with overall performance and can effectively distinguish students in a position to apply a broader writing construct from those for whom text production constitutes a significant barrier to achievement. In the current state-of-the-art, AES provide little direct evidence about such matters as strength of argumentation or rhetorical effectiveness.

AES systems measure features of the text structure, linguistic structure, and conventional print form of essays as such, the systems primarily measure text production skills. This paper examines the construct measured by automated essay scoring (AES) systems.

Spanish essay grader