Sunday 16 August 2009

Teach Them HARD!

An interesting article on the Conservatives' plans for school league tables. My thoughts on this one are kind of scattered, to be honest, but there are a number of issues in this that need to be considered.
  • It is most certainly true that universities regard certain A levels as more important than others, and this is a fact made clear to students while they are choosing their A levels. That means that whether or not Ofqual are correct and there is no such thing as "hard" and "soft" A levels is completely irrelevant. If the universities believe that there are, then it's game over.
  • Whilst on the subject of "hard" and "soft" subjects at A-level, though (leaving aside the alternative qualifications, and all things being equal I would rather keep those if possible, and don't find it at all surprising that the Tories don't), it seems very hard to believe that it can be said with certainty, by Ofqual or anyone else, that they are standardised. This is mainly because I don't see how they could be standardised. The skills required and marking methods applied vary so much it is difficult to believe they can all be considered level with each other. Obviously this may be ignorance talking, Ofqual might have an incredibly impressive method of comparison that I simply can't conceptualise, but I tend to doubt it. Certainly in maths, classes in our school who took maths and statistics consistently scored higher than those who take maths and mechanics, a pattern which had existed for some time before I arrived at the school I taught at, and continued despite having the least experienced teacher (i.e. me) handling statistics, and a ferociously competent former HoD on the mechanics side. There's also the fact that we were told in no uncertain terms to warn every applicant for maths that more than any other A-level subject their grades would drop compared to how well they did at GCSE. It is by no means unheard of for students to complete their A-levels with maths as their highest grade, but statistically speaking, that really isn't the way to bet. [1]
  • Whilst wondering about how well Ofqual can possibly judge similarity of difficulty across the board, I'm going to embrace totally the fact that they've said (along with Ofsted) that there is no evidence of "dumbing down". This has been a bugbear of mine for years. Partially this is because I've seen a number of syllabuses from various years in my time, and what is immediately apparent is that they have grown more coherent, with topics feeding into each other and building upon each other far more than they did in the past. That has made understanding easier, but that isn't dumbing down, it's imparting knowledge more sensibly (and more to the point, artificially raising the difficulty of the exams to compensate would be ridiculous). More importantly, though, the reason this stuff pisses me off is that there are three variables to consider here: exam difficulty, pupil intelligence, and teaching competence. The third is always ignored in this discussion. "Either the exams get easier each year, or the kids get smarter" is an easy formulation to make (of course, from the perspective of A-levels the latter could be objectively true if seondary school teaching is improving, which I would argue it has, at least to some extent), but the truth is that compared with secondary school teaching, the syllabuses that A-level teachers work from are subject to remarkably little fluctuation, at least in subjects that are well established. Whilst GCSE teachers are constantly being knocked around by the latest new initiatives (some of which, to be fair, are good ideas, a few of which even work in practice), an A-level teacher has the opportunity to deliver almost the exact same material in the exact same way to students who have been admitted to the course using almost the exact same criteria. I have no evidence to prove that this is what is happening, but the argument that this situation means it is to be expected that marks will improve each year is at least worthy of some consideration. A more major overhaul to the syllabus would almost certainly see a drop in results, as teachers get used to the new material and how best to impart it. As I say, the fact that this theory is possible doesn't make it true, but hopefully it makes the point that the third variable should at least be considered.
  • I think some of the proposed plan might well be a solution looking for a problem. In my entire life I have never met anyone whose choice of sixth-form college was based on more than looking at one or two options, and then choosing the closest and/or the one most of their friends were going to. The only time this dynamic changed was when certain subjects were unavailable in one place and available in another, a methodology the Tory plan would arguably make harder, since the highest rated subjects under their system are the most commonly taught, which means keeping the subjects around that might actually have affected a choice of school will be less desirable. Unless someone can show me that there are significant numbers of students shopping around for quality sixth-from teaching in specific subjects, then I'm not sure what good the measure will do for anyone but the universities, who already have these things called "grades" they can look at. I'd also like to see some evidence that there are schools using "soft" subjects to boost their scores to the point where their maths teaching looks good when in reality it's shit on toast, and that moreover this isn't trasparently obvious to the local population.
  • The Tories are absolutely right that focusing on the C/D borderline takes up a disproportionate amount of time, at the expense of other children. It's almost as though a league system leads to schools trying to get high in the league. If the Conservatives can offer an alternative metric that involves no numbers or categories (both of which immediately lead to focusing on getting as many children past those number or in or out of those categories), I'd love to hear it. Not that it would make any difference, the very nature of teaching is that some pupils are focused on more than others, whether they be borderline students or borderline sociopaths. It isn't fair, but there it is.
  • As a former comprehensive school student and descendant of a long line of socialists, the problems with attempting to link a school's worth with the universities their pupils go on to study at is immediately obvious. It would, in fact, entrench the problem of class bias in university acceptance (which as I understand it most certainly hasn't gone away), because the argument "if X is such a good school, why does no-one from X ever get in here" would stop being just sophistry and start being sophistry with a stamp of governmental approval (the fact that said government would be the Tories is no fucking surprise).
  • Finally, I am sympathetic up to a point with Smithers' argument that we should just scrap the league tables, because of the damage they do for all sorts of reasons. What I do worry about it what would happen once they were gone. I'm not sure it would lead to teaching being any less of a political football; it seems more likely to my mind that it would simply become an even less well-informed debate than it currently is (if such a thing is possible). I'm also aware that whilst the teaching profession includes a great many people of tremendous talent and dedication, it can't be claimed that everyone is sufficiently self-motivated, and perhaps some form of external pressure isn't actually a bad idea. Whether that could still be effectively provided by Ofsted without the league tables isn't something I'm sure on at all. My point, though, is that granting schools greater autonomy isn't necessarily a great idea, even though each individual school probably thinks it is.

So there you go.

[1] Typing that reminded of the old "art student" vs "science student" debate that was endlessly bandied around whilst I was an undergraduate (well, it was bandied around by the science students, most of the art types were still in bed). An uneasy truce was formed by agreeing that it is both more difficult to fail or to excel in art subjects, allowing a much easier ride for the coasters, and a mountain to climb for those who wanted to reach the top. Whether this theory is true of A-levels (and I'm not sure how true it is of universities) I don't know, but one hopes Ofqual has a more detailed way of considering the situation.

12 comments:

Dan Edmunds said...

Being a further maths veteran I got to do a fairly wide range of different maths modules (4 pure, 4 mechanics, 2 stats and 2 discrete), and certainly in my view stats and discrete were a piece of piss compared with pure and mechanics. Mind you I wouldn’t be surprised if my actual results don’t actually reflect that as while I remember getting 100% in one of the stats modules, I also got 100% in one of the pure modules (I think the highest I got in mechanics was a 97% and can’t really recall discrete) and the comparison of marks between the “easy” and “hard” modules (from memory) was pretty evenly spread (well ignoring the last mechanics module I did which was fairly rubbish, but that is probably explained away by the fact that I did no work for the exam as it couldn’t actually alter my grade at all and I was trying to do well at English). Given that, I wonder if while I might have considered one thing to be “easy” and the other “hard”, in terms of what you actually get in terms of grade they were in fact roughly similar. In which case I guess the system did in fact work, and maybe perception is not necessarily the same thing.

Think I’m just typing my thoughts out load at this stage, but I do find the whole “hard” and “easy” debate interesting. I think there is some merit to putting more weight on “core” subjects, especially if they are something that is in short supply, but at the same time I think there is a lot of good that comes out of these so called “soft” subjects so a system that doesn’t to away with them would be nice to see (don’t know how, maybe you get a higher weighting based on the core, but so many marks are awarded for variety as well…).

SpaceSquid said...

I take the point that one could have comparative difficulty levels that don't actually cross grade boundaries, but I suspect you qualify as an outlier, especially given your results. In my small experience, a student's pure marks are frequently one or even two grades lower than their statistics mark.

"I think there is some merit to putting more weight on “core” subjects, especially if they are something that is in short supply."

In what sense do you mean short supply? People taking them?

Assuming that's what you mean, thn the truth is, of course, that subjects are already weighted, in terms of university admissions. I had to get AAB to get into Durham, and one A had to be maths for instance. So that takes care of weighting so much as you need some subjects more than others to be able to do what you want at university, and what you want to do at university is weighted by what you want to do in life, if you know what that is, or by maximising your chances for doing something, if you don't have anything specific in mind. And, subject to their resources, I'd imagine you'll generally find that as a group, universities offer courses roughly in proportion to how much demand there is for graduates from those courses.

So the impetus for studying a topic like maths is most certainly there. Improved chances to get onto a science-based course at a university, and then into a reasonably high-paying job. If people aren't taking that route,it's at least arguable that it's because they're not interested in the destination, and fiddling around with the intial stages isn't going to help.

The immediate counter to this, of course, is that you can't necessarily trust a 16 year old to choose the best A levels for what they might end up wanting to do, but whether that would change if weighting was introduced, though, I don't know. What incentive would there be to take more weighted subjects, beyond what I've already covered?

Anyway, those are my thoughts from your "short supply" comment, which I may have interpreted wrongly. The idea behind the Tory plan, though, is different. The weighting isn't designed to incentivise students to take "hard" subjects, it's to make school who teach them well stand out. Though even that isn't likely to be the goal, the goal is likely to make schools that offer a wide range of options instead of specialising in the core ones to start looking like they suck.

Dan Edmunds said...

I think by “short supply” you’ve got roughly what I mean, but as well as people taking that subject I think I’m also factoring in quality of teaching, and number of courses offered as an additional factor. Presumably by emphasising the importance of a relatively small number of core subjects this would increase demand for the quality teachers in that area? Hopefully resulting in more teachers working towards those positions (this may be a pipe dream given all teachers are of course paid the same but there may be additional factors involved I – not being a teacher am unaware of).

In respect of your point about “dumbing down” and the fact there are three factors involved (exam difficulty, pupil intelligence and teaching quality – with teaching quality being the one that is generally ignored), I would agree with that and it annoys me when it’s almost automatic that people leap to the dumbing down conclusion. I think however that (as you mention) quality of teaching is a hard judgement to make given either a static or a variable syllabus. One of my biggest complaints (well it’s not really a complaint but something I don’t like) is what I shall term as “teaching the exam and not the subject”. One of the problems with a fairly static syllabus (like A-levels) is the ability to teach kids specifically how to pass an exam, often at the detriment to teaching the subject itself. Things that you know aren’t going to appear on the exam are ignored, things that you know will are taught almost parrot fashion to ensure the kids in question do better than they otherwise would on the exam (obviously simplified – but hopefully you get my point). So yes – over time with a fairly static syllabus you would expect grades to improve, but I don’t necessarily think that’s entirely down to improved teaching, but rather as a result of more targeted teaching – I’m sure there will be some degree of improved teaching as well, but it’s difficult to judge. Of course everything is geared up to provide pressure to do exactly that (the league table is a prime example), as exam results become the be all and end all. In an ideal world I would like to see the subject taught completely independently of the exam, and the exams to be flexible enough that someone who has been well taught and has ability in the subject as a whole will do well in them, and the exams designed in such a way that trying to prepare specifically for them would be to your own detriment. Of course in reality that is pretty difficult to achieve and I suspect any such thing would run the risk of making life very difficult for some teachers / and kids. Don’t really have any kind of silver bullet for that one, just a general feeling that I wish teachers could just teach a subject without concern for the exams, and that the good teachers and good pupils still get rewarded for their efforts.

Tomsk said...

Universities offer courses roughly in proportion to how much demand there is for graduates from those courses

Is there any evidence for this? To me it looks like they offer courses roughly in proportion to how much demand there is from potential students for those courses. Not surprising that they do this really, as it's attracting students that pays the universities' bills. What do they care whether the degree has any worth after they leave?

If 16 year olds weren't sold the lie that all A level subjects are of equal value, they would have a much better chance of making a well-informed decision about which to choose. I'm not at all convinced that weighting is a good idea, but if it forces schools to focus on and recommend the "hard" subjects, that at least would be a good thing.

Dan Edmunds said...

Just as the thought has now crossed my head, I thought I stick it down. How do we actually determine what is a core / hard subject in an unbiased way? I mean my instinct is Maths, English, Sciences and Languages, but then I did Maths, Further Maths, Physics and English so I’m probably more than a little bit biased. I note Media Studies always gets picked on, but no very little about what it actually consists of. Likewise I know Music tends to get picked on, but from what I know my sister does it’s actually a very involved and quite wide reaching subject.

I guess the approach would be to take how well each subject provides support for other subjects, so Maths and English would fit, but the Sciences wouldn’t necessarily. Don’t know, needs some thought anyway, for all I know Media Studies should in fact be emphasised….

SpaceSquid said...

"Is there any evidence for this? To me it looks like they offer courses roughly in proportion to how much demand there is from potential students for those courses. Not surprising that they do this really, as it's attracting students that pays the universities' bills. What do they care whether the degree has any worth after they leave?"

They will care to some degree, because universities are judged on where their graduates go next, though admittedly only in a very general sense, i.e. further study, employment, or neither.

In truth, my (admittedly vague) theory along these lines was mainly based on the massive increase in facilities for teaching law (as most obviously demonstrated by the soon-to-arrive law uber-building that will soon cast its shadow over the Science Site here). It's entirely possible that this means far more people want to be laywers, rather than far more laywers are needed, but I think you downplay the degree to which the latter will be considered by the University. It's also more than likely that a greater demand for courses will follow greater advertising that implies greater demand, so just by following the trends of student application the University is also responding to the job market in any case.

"I'm not at all convinced that weighting is a good idea, but if it forces schools to focus on and recommend the "hard" subjects, that at least would be a good thing."

In my experience recommending appropriate subjects (which isn't the same thing as "hard", nor should it be) is doing fairly well without weighting, but I have no idea on the larger scale. I guess the fear is that schools deliberately recommend easier subjects to boost their overall grades, but this is something I've yet to see in practice.

Tomsk said...

Spielbergo - your approach I think is OK to define "core" subjects, but that's not the same as "hard" subjects, i.e. how easy it is to do well in the subject's A-level. Perhaps you could assess it by comparing performance with GCSE results. So for a given student take their average GCSE performance (in %) over a representative sample of subjects (say maths, english, science, languages, humanities) to establish a baseline, then subtract that from their A-level result. Averaged over all students, that would give an "easiness" score. At least I think it would - I'm sure our host will spot any flaws. :)

Squid - yes, appropriate doesn't always equal hard, but there is at least in theory an incentive (both for students and schools) to choose the easy subjects, given that they are presented as being of equal value. So I think it's fair to assume that easy subjects will be chosen inappropriately much more often than hard ones.

Chemie said...

'I guess the fear is that schools deliberately recommend easier subjects to boost their overall grades, but this is something I've yet to see in practice.'

I have. Every grammar school I know did it, mine included. Until it was removed from tables, nearly all the local grammars made everyone do general studies (with less than 6 hours teaching) because it meant easy grades. My school feely admitted to teaching 'Humanities' GCSE as a coursework heavy way of getting around making everyone do the harder History or Geography. They 'encouraged' people into A-level stats modules unless they were doing physics.

There is reason why so many Etonians do 'Art history', 'Art' and 'Classical civilisation' and girl's public schools are full of 'Home economics'.

Tomsk said...

Our school did the general studies thing too. Unless you did 4 A-levels, it was effectively compulsory, because 3 A-levels didn't quite reach the threshold minimum classroom hours that the school imposed. General studies conveniently made up the difference.

General studies does serve a purpose though - for as long as it exists no one can say with a straight face that all A-levels are equal.

Dan Edmunds said...

We didn’t do general studies. I think they did introduce it a few years after I left (when they switched to the AS type system). Instead we filled in the spare time with the acronym subjects (RE, PE and PSE). Mind you we overall had very little in the way of “soft” subjects on offer at that time (about the only one I think available was Business Studies – and for all I know that might not even be “soft”).

I don’t even know what General Studies actually consists of (going to Google it now), beyond the general disdain people have towards it…

Chemie said...

http://www.thestudentroom.co.uk/showthread.php?t=810355

Some numbers about subject difficulty and desirability

http://www.trin.cam.ac.uk/index.php?pageid=604

The universities have always known, what subjects are hard and valuable. Shame no-one has told the 16 year olds.

SpaceSquid said...

@ Tomsk:

"Averaged over all students, that would give an "easiness" score. At least I think it would - I'm sure our host will spot any flaws. :)"

The immediate flaw I can spot is in it not accounting for the relative ability of intake for each course. You could stratify by subject intake, of course, but even then you run into problems. If the easiness score of maths and media studies are the same, does that mean they are equally difficult, or that an easier A-level is compensated for by less intelligent students taking it?

Having said that, with a bit of fiddling, you might be able to come up with something quite useful. And if it turns out that the average score of those entering media studies is lower and the easyness score ends up being lower as well, you'd have a pretty firm case.

"Squid - yes, appropriate doesn't always equal hard, but there is at least in theory an incentive (both for students and schools) to choose the easy subjects, given that they are presented as being of equal value. So I think it's fair to assume that easy subjects will be chosen inappropriately much more often than hard ones."

That's true, I was just pointing out that this is something students are councilled against already.

@ Chemie:
"I have. Every grammar school I know did it, mine included."

Fair enough; I know pretty much nothing about grammar schools, I must confess.

I'd also forgotten about General Studies, an A-Level so simple I got an A for showing up to the exam. Of course, if everyone uses it, it would make no difference when comparing schools.

"There is reason why so many Etonians do 'Art history', 'Art' and 'Classical civilisation' and girl's public schools are full of 'Home economics'."

There most certainly is, though naturally it is difficult to imagine the Tory plan is intended to address that particular point.