Legitimate concerns about writing rubrics

There is no doubt that some criterion-referenced assessment procedures and related documents such as the Australian ESL Scales can be extremely useful; one example of the latter in use comes from The New England (NSW) Girls School. A “cut-down” form I use myself is here, and it has proven very accurate.

However, there are legitimate concerns about the use of writing rubrics. See for example The Trouble with Rubrics by Alfie Kohn (ENGLISH JOURNAL March 2006 — vol. 95, no. 4).

Once upon a time I vaguely thought of assessment in dichotomous terms: The old approach, which consisted mostly of letter grades, was crude and uninformative, while the new approach, which included things like portfolios and rubrics, was detailed and authentic. Only much later did I look more carefully at the individual floats rolling by in the alternative assessment parade — and stop cheering…

Consistent and uniform standards are admirable, and maybe even workable, when we’re talking about, say, the manufacture of DVD players. The process of trying to gauge children’s understanding of ideas is a very different matter, however. It necessarily entails the exercise of human judgment, which is an imprecise, subjective affair. Rubrics are, above all, a tool to promote standardization, to turn teachers into grading machines or at least allow them to pretend that what they’re doing is exact and objective. Frankly, I’m amazed by the number of educators whose opposition to standardized tests and standardized curricula mysteriously fails to extend to standardized in-class assessments.

The appeal of rubrics is supposed to be their high interrater reliability, finally delivered to language arts. A list of criteria for what should be awarded the highest possible score when evaluating an essay is supposed to reflect near-unanimity on the part of the people who designed the rubric and is supposed to assist all those who use it to figure out (that is, to discover rather than to decide) which essays meet those criteria…

I worry more about the success of rubrics than their failure. Just as it’s possible to raise standardized test scores as long as you’re willing to gut the curriculum and turn the school into a test-preparation factory, so it’s possible to get a bunch of people to agree on what rating to give an assignment as long as they’re willing to accept and apply someone else’s narrow criteria for what merits that rating. Once we check our judgment at the door, we can all learn to give a 4 to exactly the same things.

This attempt to deny the subjectivity of human judgment is objectionable in its own right. But it’s also harmful in a very practical sense. In an important article published in 1999, Linda Mabry, now at Washington State University, pointed out that rubrics “are designed to function as scoring guidelines, but they also serve as arbiters of quality and agents of control” over what is taught and valued. Because “agreement among scorers is more easily achieved with regard to such matters as spelling and organization,” these are the characteristics that will likely find favor in a rubricized classroom. Mabry cites research showing that “compliance with the rubric tended to yield higher scores but produced ‘vacuous’ writing.”…

Maja Wilson has written a book on the subject: Rethinking Rubrics in Writing Assessment . The fact it won a NCTE James Britton Research Award attracts me, and the chapter I downloaded looks very promising indeed. The gist of her argument may be seen here:

So if you really do want to think about what rubrics, for all their uses, might miss, and whether what they miss could after all be very important, I suggest you follow these ideas and keep thinking…

See Maja Wilson (PDF) Chapter One

RELATED

Right-wing education critique is historically inaccurate and perpetuates myths on my personal site quarrels about some relevant issues. It was written around the same time as this post. Teachers may be interested. As the title indicates, it is a bit of a rant…

 

The literacy we need but many don’t want…

I wrote a careful essay on the nature of literacy in 1998; you may read an updated version here. At one level literacy involves just learning to read and write, using whatever teaching methods work — and that is always a combination of methods. (The whole-language VERSUS phonics myth is just that, a myth; it is rather whole-language AND phonics.) Conservative critics always focus on one end of this, and berate schools if 100% of students have not mastered basic literacy by, say, the end of primary school — a great aim, but an unrealistic one.

There are ALWAYS, whatever you do, going to be those who do not master reading and writing as well as we would like them to, just as there are those who achieve literacy even before going to school. Of course we all want an outcome that allows all those who can be literate to be literate; no quarrel there, but let’s stop nonsense such as bleating about 25% of students being "below average" and let’s stop imposing standardised tests, or at least let’s stop tying too much to them, or regarding them as anything other than potentially useful diagnostic tools.

A bit less time spent on testing and bean-counting and a bit more time, funding, and effort dedicated to actual teaching and teaching environments might do a lot more good.

But there is a type of literacy conservatives not only do not talk about but positively discourage: critical literacy. My belief is that this is so important that a democracy cannot function without it.

Here is someone who knows why.

Vanessa Andreotti is a Brazilian teacher/trainer who is currently a research fellow and education coordinator at the Centre for the Study of Social and Global Justice at Nottingham University.

 

My English Teachers 5: Bill Maidment (revisited)

See my earlier entry where I mention a problem with the Quadrant site. This is now fixed, and “Men Without Borders” by Neil McDonald is back online.

It was Maidment’s ability to analyse every nuance of an individual passage of literature, elucidating the rhythm, symbolism and allusions, then to place it in the context of the work as a whole—all the while keeping us aware of the period when it was written—that was of special value to us all as film critics and teachers. In addition, there was his deep understanding of imagery, traditional emblems, heraldry and associations with the paintings of the period of the work being examined. Unlike many contemporary critics, Maidment was particularly good at defining a genre, exploring precisely how it related to other literary forms…

I learnt from Bill to ask these questions about any film or literary work I was examining: What does it mean? What did it mean when it was first released or published? What is it about? What is it really about? What does it assume? What does it assert? And what does it imply? …

Bill Maidment’s influence on my work was, I believe, more extensive. I first encountered him when I enrolled in his Eighteenth-Century Literature option in 1963. It was a small group, and we soon became friends. Even after I began to write Shakespearean criticism, Bill continued as mentor and friend. When my first article on Macbeth was rejected in 1964, he was there with encouragement and advice: “Certainly it has some rough edges, but it deserves to be published.” And it was, by Frank Moorhouse no less.

OFTEN OVERLOOKED by even his warmest admirers was how good a Shakespearean critic Maidment was. Only very recently he pointed out to me how the breaking of even the most trivial of oaths had a religious significance in Elizabethan England—very difficult to convey to a modern mainstream audience. Consequently when Kenneth Branagh adapted Love’s Labour’s Lost to the screen, he needed the wartime setting to create an appropriately serious modern equivalent to explain the lovers’ partings. When I was teaching Shakespeare using the Elizabethan theatre models made by my father, it was Maidment who pointed out that the playhouse itself was part of the play’s imagery. This coupling of imagery and form became vital when I worked on Shakespearean film. It was Bill who alerted me to the way Orson Welles played cinematic variations on Shakespeare’s imagery as well as enhancing the word pictures with visual equivalents…

The intellectual rigour and sceptical tolerance Bill instilled in his students gave us the confidence to see through, refute and ultimately systematically ignore the jargon-infested discourse theorists, open and covert Marxists and dogmatic gender-studies experts—who have come close to destroying film studies in recent years. There was really no debate: their want of elementary film scholarship made them easy game whenever they wrote or spoke to anyone but each other. There would be lots of eye-rolling, heavy sighs and throat clearing, but rarely any argument. Quadrant readers have, in a way, experienced this phenomenon for themselves. I have only to raise a political issue for the letters column to be filled with missives of dissent. I have even taken issue with our editor! And this is as it should be: a journal of ideas like ours is no place for unquestioned opinion. But when I accused the New South Wales Board of Studies of compelling students to misrepresent their set films for ideological reasons in my article “How Not to Teach Film”, and attacked jargon-ridden film criticism in “Screen Studies and Lantana”, the silence was deafening…

AS A MAN Bill Maidment was gentlemanly and unassuming to a fault. In his prime he was very handsome, but dressed down as if he feared any sartorial display would distract from his teaching. All Bill’s students know of his battle with his stammer, which, in the early 1960s, threatened to destroy his career as a teacher. By the way—who, today, would hire a lecturer with a stammer? Professor Wesley Milgate did, and gave the English department at the University of Sydney one of its greatest scholars and teachers of the last century. So how did Bill survive as a lecturer? He had Milgate’s support, and the students didn’t want to forgo what this unassuming, brilliant man had to offer. I remember vividly how we would simply sit there, willing him to keep going so we could make our notes and read or re-read the text he was discussing, knowing we were getting insights that few other lecturers could provide.

The stammer too was the basis of some of the best Maidment stories. No one dared so much as move during his lectures for fear it would put him off! So when, during a lecture on D.H. Lawrence, Bill mentioned that the writer’s sexual problems were rooted in his relationship with his mother, there was dead silence. The class remained quiet when he added that Lawrence’s sexuality was also rooted in the English Puritan tradition, and continued to be silent as Bill used the same word to describe a whole range of other influences in which D.H. Lawrence was rooted. The joke that went around campus the next day was that Maidment had managed to root Lawrence fifteen times! When I mentioned the story to Bill, he couldn’t remember the incident, but added, “I’m sure there was some deep-seated Freudian significance.”

Interwoven with Bill’s battle with his stammer were the triumphs. These were the occasions when it was heard around campus that Bill Maidment was about to lecture on one of his many specialties, and it would be standing room only. These lectures would be received in hushed silence, followed by a rousing ovation at the conclusion. Ultimately Bill overcame his stammer by deciding that it simply didn’t matter…

Confirming and extending my own memories of this remarkable teacher.

ca-pub-7987726976142433

 

Recycle 5: from January 20, 2004 — Values in NSW schools

This one is deep in my archives! In its original context it is very political too, being an angry response to Prime Minister John Howard’s unjustified attack on public education. You may see a leading non-government educator’s reply here.

I posted the “Values We Teach” document, since superseded but essentially the same. See Core Values on the NSW Department of Education site — unless they move it again! Terrible, that way, the NSW government. Always fiddling… However, here is the version I posted back in 2004.

Love of learning

NSW public schools aim to create young Australians who value learning and knowledge and who relish the effort and possess the confidence needed to resolve problems, or to master a skill, topic or subject; who can compose clear and precise prose and construct well-founded arguments; who have mastered the art of talking with others as a route to better understanding; who are deeply interested in finding common ground with other people, other ways of life and ways of thinking and believing; and who are interested in imaginative and new ideas, and in seeking out truth.

NSW public schools teach the value of:

scholarship, accurate and extensive knowledge, wide reading and understanding of traditional and new fields of study, including information technology

rational inquiry and logical, well-founded argument

clarity, confidence and coherence in thinking, writing and speaking

curiosity and imagination as the basis for pleasure in learning

communicating with others as a way of establishing agreement and arriving at truth.

Aiming for high standards

NSW public school students are encouraged to achieve their personal best and to aim for excellence in everything they do.

They are encouraged to participate in sport and creative performances and to learn ways of winning and losing graciously.

NSW public schools teach the value of:

aiming for the best in academic, creative and sporting achievement and in all public performances.

Care and respect for ourselves and others

In partnership with parents and carers, NSW public school students are taught how to respect and care for themselves and others, in order to achieve self-discipline and physical and mental well being. They learn respect and care for others through the codes and practice of good manners, the give and take of friendship, the routines of companionship and the management of friendly rivalry. They learn respect for expertise, legitimate authorities, and leadership through acceptance of responsibility. They are taught ways of recognising right from wrong.

NSW public schools teach the value of:

recognising right over wrong

honesty and courtesy

health, fitness and well being

discipline, punctuality, reliability

experience, expertise and authority

friendship, companionship and friendly rivalry

self-discipline, independence and responsibility

Care and respect for families and communities

NSW public school students are encouraged to feel and demonstrate empathy and respect for those who are vulnerable and dependent. They learn to demonstrate the values of generosity and compassion and the principles of fairness. In turn they earn the right to expect to be treated by others with respect and fairness. As members of families and communities they learn how to treat others with consideration.

NSW public schools teach the value of:

kindness and helpfulness towards those who are vulnerable, or who are less able than others

the rights of individuals and groups to a fair go

sharing and equity as principles of personal and social relationships

different histories, customs, cultures and outlooks within home and school communities and in the Australian community

Respect for work

NSW public school students learn the need to grasp opportunities, the rewards of effort, and the value of work. They learn to see how work is changing and how new forms of work encourage experiment and resilience. They learn with new and evolving technologies and are taught to welcome innovation. Public school students learn to work well together with different kinds of people.

NSW public schools teach the value of:

paid, unpaid and voluntary work

opportunity, aspiration and enterprise

creativity, experiment and resilience

working together and in competition

skilled workmanship

productive habits and methods.

Proud Australians and citizens of the world

As young Australians, NSW public school students learn to understand and appreciate the beauty and uniqueness of their land.

They learn about Australia’s creative arts, literature, and history, and the insights to be gained for the future good of Australia. They learn to appreciate the significance of Australia’s Indigenous people and of immigration to Australian identity.

NSW public school students are taught to respect the rule of law and Australia’s democratic institutions and procedures. They are taught their own rights and responsibilities, and those of groups and governments under the code of law and systems of justice.

NSW public schools teach the value of:

Australia’s democratic institutions and procedures

the rights and obligations of governments, individuals and groups under the rule of law

the contributions of Indigenous people to Australia, and their history and struggles as our country’s first custodians

the beauty and uniqueness of Australia’s landscapes and environments

the histories and cultures of all Australians

the role of migration in building Australia’s place in the world

the interdependence of human beings with each other and with the natural world

Values for Australia’s future

These values help each NSW public school student to take full advantage of new ideas and knowledge which characterise the social and economic environment emerging in Australia, and in the world community.

In conjunction with an excellent general and vocational education, this code of values enables young Australians educated in NSW public schools to freely choose and enjoy their paths through adult life, to master the complexity and variety of the contemporary world, and to contribute as citizens to making Australia a better, more prosperous and happier place.

Perhaps the PM regards some of these as “excessive political correctness”? There are probably some values there the PM would have a problem with — but that is his problem, and ours in having a neanderthal for a Prime Minister. I can understand someone who hasn’t had an original or really broad-minded thought in the past forty years thinking that way, just as I can find it quite remarkable that a man whose prime value is how to hang onto power, stifle debate, and lie to the Australian people whenever it seems necessary to achieve his goals is suddenly the mouthpiece for “Australian values.” Am I being disrespectful? Bloody oath I am.

With a taste there of the original context, you will note. But this present blog is of course a rant-free zone. 😉

 

Open Education

Tom Hanson, the editor of this good US site, left a comment on “Changing lives without aiming to”, a post on my new Blogspot blog. First I am glad he did as I like both that post and the new blog — a rant-free zone of personal reflections on teaching — and then I find I like his site.