The mysteries and injustices (?) of the NSW UAI ranking

A very thoughtful Sydney blogger, Jim Belshaw, who has a strong background in public administration, has posted three times recently on HSC and UAI (University Admissions Index) issues, beginning with Gripes about Australian Education – 1, then Gripes About Australian Education – 2, and finally HSC, UAI and Education – a further note. I commend these long posts to you for careful reading. Note also that x/100 in the UAI is a RANK not a MARK.

In the first post, among other things, Jim writes:

Note to readers: This is the first of two posts setting out some of my gripes with the current Australian education system. While I was writing the first part, one of youngest’s friends and her mum called in. Like Clare, the friend will do the New South Wales Higher School Certificate this year. The friend’s mum was not aware of some of the things I had found out re student choice. So it may be that this post will actually be helpful to some, not just a gripe…

To ensure a good UAI, students must maintain the required level in the on-going assessment tasks. Get a few of the early tasks wrong and your target UAI may go out the window well in advance of the final exams themselves. There is a further problem. The assessment tasks themselves take time for both students and teachers, reducing the time available for teaching and learning.

In all these circumstances it is perhaps not surprising that there should be so many students, my own daughters included, who feel the need for special tutoring outside school to build their strength in weaker subjects.

The HSC and UAI results themselves create new pressures.

I think that it is still generally accepted that education is more than narrowly defined academic results. I think, too, that there is a degree of acceptance that the HSC and following UAI outcomes are a far from perfect predictor of university results, even less so of longer term success. Yet the reality is that none of this matters.

At school level, the HSC results provide the base for the creation of league tables comparing school performance. Schools talk about broader education, but they boast about or put the best spin they can on HSC outcomes.

Parents are just as bad. We read the published HSC results avidly. When, as happened in my eldest’s year, an apparently good class fails to get a single band six (the top band) with this outcome adversely affecting subsequent UAIs, the outcome becomes a matter of agitated debate.

At university level, the publication of UAI cut-offs – a simple measure of demand for individual courses at individual universities – creates another league table. Students and parents use these to guide their choices for the next year, with a bias towards higher UAI universities…

The second post takes up ” the uncertainties associated with the process.”

…In the words, of my eldest, how was I to know that English would be my best subject?

This was a subject that Helen struggled in to some degree, a subject where we arranged some external tutoring, a subject where her place in class was lower than in most other subjects. Then come the HSC her exam mark was far better than expected, miles in front of the raw school assessment, with the assessment itself being scaled up by a substantial margin.

Conversely, drama was one of Helen’s favourite subjects and one she was good at. Unexpectedly, in drama the exam mark was lower than expected, with the no scaling up of school assessment.

I had attended all of the class and individual performances and did not understand this result. I still do not, even after hours spent analysing the official data…

The third post teases out further problems, consequent on careful reading of the Universities Admission Centre site.

I have done a little more thinking and investigation since then talking to people and also working my way yet again through the UAC web site.

As all parents know, the UAI index is a device for competitive matching of demand by students for courses to available places in courses within NSW and ACT universities. At the end of each entry round there are always some vacancies in some subjects in some institutions. In this sense, the UAI is most important as a rationing device for the more popular courses and institutions.

Given all this, the final UAI required for entry to a particular course in a particular institution becomes a price proxy for demand and supply for that course. The result is a quite dramatic 40 spread between the lowest entry UAI that I could find (60 in 2006) and the 99 plus required for some courses in some institutions.

The UAI is not a perfect market clearing mechanism simply because students are not themselves perfectly mobile, while universities will drop UAIs only so far to attract students. This means that at the end of the process there are always some vacancies in some institutions one one side, some unplaced eligible students on the other…

All this sounds fair and equitable. However, there are some very real problems.

To begin with, no one seems able to tell us – perhaps a reader can – just how the university study component is turned into an index number for ranking purposes across grades, courses and institutions. So we have another black box uncertainty.

The second problem is the nature of the equivalence between the HSC/UAI ranking and the University results…

That is the merest sampling of Jim’s posts. I should add that Jim is far more knowledgable than I am when it comes to data analysis and statistics. In this area, I really am an English teacher, I’m afraid.

There is of course a shiny brochure explaining how it all works, which you can download as a PDF here. I have read it, but in all honesty I am none the wiser. The cynic in me sees some of the processes involved as pure mumbo-jumbo, and deeply contradictory as well. The UAI seems to make unjustified assumptions about the relative merits (or “difficulty”) of courses that are logically not comparable. This may work against students doing the courses that from an educational viewpoint are the best to do, in the sometimes vain pursuit of the best UAI possible.

OK then. I have to admit I can’t help Jim with his concerns. This is galling, but as I said I am merely an English teacher.

Note too, as Jim does, that this is quite a separate issue from the desirability or otherwise of particular courses, so don’t get sidetracked by the “dumbing down” argument. It is not relevant to this particular set of issues.

Can you help?

Update

A fourth somewhat related post appeared on Jim’s blog today: Drought, Higher Education and Mental Traps.

… One of my aims here has been to sell the value of the regional experience for education and work.

My most recent writing on the NSW HSC, the UAI and the linkages between this and university education has taken me in a new direction because I wrote this in part from the perspective of a parent now living in Sydney looking at education for his own daughters.

This writing caused an interesting paradigm shift because it caused me to look at university education in Sydney as compared to regional NSW from a different perspective, leading me to the unexpected conclusion that in some ways regional kids are significantly advantaged as compared to Sydney kids. This is the opposite of the conventional wisdom, a wisdom that I had accepted to some degree…

Students with very high UAIs can essentially pick and choose. Beyond that point, New England year 12 students would appear to be advantaged.