RSS

The mysteries and injustices (?) of the NSW UAI ranking

07 Jan

A very thoughtful Sydney blogger, Jim Belshaw, who has a strong background in public administration, has posted three times recently on HSC and UAI (University Admissions Index) issues, beginning with Gripes about Australian Education – 1, then Gripes About Australian Education – 2, and finally HSC, UAI and Education – a further note. I commend these long posts to you for careful reading. Note also that x/100 in the UAI is a RANK not a MARK.

In the first post, among other things, Jim writes:

Note to readers: This is the first of two posts setting out some of my gripes with the current Australian education system. While I was writing the first part, one of youngest’s friends and her mum called in. Like Clare, the friend will do the New South Wales Higher School Certificate this year. The friend’s mum was not aware of some of the things I had found out re student choice. So it may be that this post will actually be helpful to some, not just a gripe…

To ensure a good UAI, students must maintain the required level in the on-going assessment tasks. Get a few of the early tasks wrong and your target UAI may go out the window well in advance of the final exams themselves. There is a further problem. The assessment tasks themselves take time for both students and teachers, reducing the time available for teaching and learning.

In all these circumstances it is perhaps not surprising that there should be so many students, my own daughters included, who feel the need for special tutoring outside school to build their strength in weaker subjects.

The HSC and UAI results themselves create new pressures.

I think that it is still generally accepted that education is more than narrowly defined academic results. I think, too, that there is a degree of acceptance that the HSC and following UAI outcomes are a far from perfect predictor of university results, even less so of longer term success. Yet the reality is that none of this matters.

At school level, the HSC results provide the base for the creation of league tables comparing school performance. Schools talk about broader education, but they boast about or put the best spin they can on HSC outcomes.

Parents are just as bad. We read the published HSC results avidly. When, as happened in my eldest’s year, an apparently good class fails to get a single band six (the top band) with this outcome adversely affecting subsequent UAIs, the outcome becomes a matter of agitated debate.

At university level, the publication of UAI cut-offs – a simple measure of demand for individual courses at individual universities – creates another league table. Students and parents use these to guide their choices for the next year, with a bias towards higher UAI universities…

The second post takes up ” the uncertainties associated with the process.”

…In the words, of my eldest, how was I to know that English would be my best subject?

This was a subject that Helen struggled in to some degree, a subject where we arranged some external tutoring, a subject where her place in class was lower than in most other subjects. Then come the HSC her exam mark was far better than expected, miles in front of the raw school assessment, with the assessment itself being scaled up by a substantial margin.

Conversely, drama was one of Helen’s favourite subjects and one she was good at. Unexpectedly, in drama the exam mark was lower than expected, with the no scaling up of school assessment.

I had attended all of the class and individual performances and did not understand this result. I still do not, even after hours spent analysing the official data…

The third post teases out further problems, consequent on careful reading of the Universities Admission Centre site.

I have done a little more thinking and investigation since then talking to people and also working my way yet again through the UAC web site.

As all parents know, the UAI index is a device for competitive matching of demand by students for courses to available places in courses within NSW and ACT universities. At the end of each entry round there are always some vacancies in some subjects in some institutions. In this sense, the UAI is most important as a rationing device for the more popular courses and institutions.

Given all this, the final UAI required for entry to a particular course in a particular institution becomes a price proxy for demand and supply for that course. The result is a quite dramatic 40 spread between the lowest entry UAI that I could find (60 in 2006) and the 99 plus required for some courses in some institutions.

The UAI is not a perfect market clearing mechanism simply because students are not themselves perfectly mobile, while universities will drop UAIs only so far to attract students. This means that at the end of the process there are always some vacancies in some institutions one one side, some unplaced eligible students on the other…

All this sounds fair and equitable. However, there are some very real problems.

To begin with, no one seems able to tell us – perhaps a reader can – just how the university study component is turned into an index number for ranking purposes across grades, courses and institutions. So we have another black box uncertainty.

The second problem is the nature of the equivalence between the HSC/UAI ranking and the University results…

That is the merest sampling of Jim’s posts. I should add that Jim is far more knowledgable than I am when it comes to data analysis and statistics. In this area, I really am an English teacher, I’m afraid.

There is of course a shiny brochure explaining how it all works, which you can download as a PDF here. I have read it, but in all honesty I am none the wiser. The cynic in me sees some of the processes involved as pure mumbo-jumbo, and deeply contradictory as well. The UAI seems to make unjustified assumptions about the relative merits (or “difficulty”) of courses that are logically not comparable. This may work against students doing the courses that from an educational viewpoint are the best to do, in the sometimes vain pursuit of the best UAI possible.

OK then. I have to admit I can’t help Jim with his concerns. This is galling, but as I said I am merely an English teacher.

Note too, as Jim does, that this is quite a separate issue from the desirability or otherwise of particular courses, so don’t get sidetracked by the “dumbing down” argument. It is not relevant to this particular set of issues.

Can you help?

Update

A fourth somewhat related post appeared on Jim’s blog today: Drought, Higher Education and Mental Traps.

… One of my aims here has been to sell the value of the regional experience for education and work.

My most recent writing on the NSW HSC, the UAI and the linkages between this and university education has taken me in a new direction because I wrote this in part from the perspective of a parent now living in Sydney looking at education for his own daughters.

This writing caused an interesting paradigm shift because it caused me to look at university education in Sydney as compared to regional NSW from a different perspective, leading me to the unexpected conclusion that in some ways regional kids are significantly advantaged as compared to Sydney kids. This is the opposite of the conventional wisdom, a wisdom that I had accepted to some degree…

Students with very high UAIs can essentially pick and choose. Beyond that point, New England year 12 students would appear to be advantaged.

Advertisements
 
3 Comments

Posted by on January 7, 2007 in equity/welfare, gifted education, HSC, questions asked

 

Tags:

3 responses to “The mysteries and injustices (?) of the NSW UAI ranking

  1. Delenio

    January 7, 2007 at 9:24 pm

    For all the problems noted about the ranking/entry system used by the UAC, I have always had trouble imagining a better one.

    First, the method of ranking the ‘merit’ of each subject for scaling the undisclosed UAI ‘score’ (used to be called the TES) has nothing to do with its perceived merit (you’re always going to get people who say that drama is great and others who say that drama is a bludge). Instead, it is revised year to year based on the HSC marks of all the students doing that subject in all their other subjects. So extension Classical Greek, with around 10-15 candidates each year, scales the best of all the subjects because almost every candidate scores very well in all his/her other subjects. If they all score very badly in their other subjects, then Classical Greek will scale very badly for the UAI. The faulty assumption is probably that a student who works hard for a given subject would also work hard in every other subject, so a subject like drama may have very many students brilliant at drama but not so good at other chosen subjects (I don’t see why this would be so). As you say, logically there is no way to rank so many different subjects, but unless we split university entrance rankings into 100 individual categories (the number of different HSC subjects), or types of subjects (which are fairly arbitrary anyway), then you have to have a method of comparing marks across subjects. I can’t think of a fairer way to scale them.

    Secondly, the need for equivalence between HSC marks and UAI rankings is frequently overstated. HSC marks are based on HSC achievement. UAI rankings are based on the universities’ need to distribute a limited resource in the fairest way possible. The UAC happens to use the HSC marks to avoid making everyone do a set of matriculation exams. Before 2001, this difference was masked because HSC marks were themselves scaled depending on the subject. Besides, the UAC releases the statistics for scaling the HSC marks in each given subject into TESs (and subsequently into UAIs) at the end of each year, and these do not tend to vary much. Any student can have a look to work out the likely equivalence.

    In my year most of us at the Mine averaged out the scaling patterns for the 5 previous years and predicted our UAIs (once we had our HSC marks) to about 0.1+- accuracy.

     
  2. ninglun

    January 7, 2007 at 9:30 pm

    Thanks, Delenio. That is very clear, and I almost understood it 😉 . You also provide me with a good example of the old cliche “when you become a teacher by your pupils you’ll be taught”. 🙂

    Delenio was a member of the very memorable Class of 2000.

     
  3. Random

    November 12, 2008 at 5:43 pm

    I hope someone like Kevin Rudd or our Minister of Education reads this.

     

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s