RSS

Legitimate concerns about writing rubrics

16 Dec

There is no doubt that some criterion-referenced assessment procedures and related documents such as the Australian ESL Scales can be extremely useful; one example of the latter in use comes from The New England (NSW) Girls School. A “cut-down” form I use myself is here, and it has proven very accurate.

However, there are legitimate concerns about the use of writing rubrics. See for example The Trouble with Rubrics by Alfie Kohn (ENGLISH JOURNAL March 2006 — vol. 95, no. 4).

Once upon a time I vaguely thought of assessment in dichotomous terms: The old approach, which consisted mostly of letter grades, was crude and uninformative, while the new approach, which included things like portfolios and rubrics, was detailed and authentic. Only much later did I look more carefully at the individual floats rolling by in the alternative assessment parade — and stop cheering…

Consistent and uniform standards are admirable, and maybe even workable, when we’re talking about, say, the manufacture of DVD players. The process of trying to gauge children’s understanding of ideas is a very different matter, however. It necessarily entails the exercise of human judgment, which is an imprecise, subjective affair. Rubrics are, above all, a tool to promote standardization, to turn teachers into grading machines or at least allow them to pretend that what they’re doing is exact and objective. Frankly, I’m amazed by the number of educators whose opposition to standardized tests and standardized curricula mysteriously fails to extend to standardized in-class assessments.

The appeal of rubrics is supposed to be their high interrater reliability, finally delivered to language arts. A list of criteria for what should be awarded the highest possible score when evaluating an essay is supposed to reflect near-unanimity on the part of the people who designed the rubric and is supposed to assist all those who use it to figure out (that is, to discover rather than to decide) which essays meet those criteria…

I worry more about the success of rubrics than their failure. Just as it’s possible to raise standardized test scores as long as you’re willing to gut the curriculum and turn the school into a test-preparation factory, so it’s possible to get a bunch of people to agree on what rating to give an assignment as long as they’re willing to accept and apply someone else’s narrow criteria for what merits that rating. Once we check our judgment at the door, we can all learn to give a 4 to exactly the same things.

This attempt to deny the subjectivity of human judgment is objectionable in its own right. But it’s also harmful in a very practical sense. In an important article published in 1999, Linda Mabry, now at Washington State University, pointed out that rubrics “are designed to function as scoring guidelines, but they also serve as arbiters of quality and agents of control” over what is taught and valued. Because “agreement among scorers is more easily achieved with regard to such matters as spelling and organization,” these are the characteristics that will likely find favor in a rubricized classroom. Mabry cites research showing that “compliance with the rubric tended to yield higher scores but produced ‘vacuous’ writing.”…

Maja Wilson has written a book on the subject: Rethinking Rubrics in Writing Assessment . The fact it won a NCTE James Britton Research Award attracts me, and the chapter I downloaded looks very promising indeed. The gist of her argument may be seen here:

So if you really do want to think about what rubrics, for all their uses, might miss, and whether what they miss could after all be very important, I suggest you follow these ideas and keep thinking…

See Maja Wilson (PDF) Chapter One

RELATED

Right-wing education critique is historically inaccurate and perpetuates myths on my personal site quarrels about some relevant issues. It was written around the same time as this post. Teachers may be interested. As the title indicates, it is a bit of a rant…

Advertisements
 

Tags: , ,

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: