On the cusp of attending higher education
The Guardian on the A-levels mess:
Pupils from disadvantaged backgrounds have been worst hit by the controversial standardisation process used to award A-level grades in England this year, while pupils at private schools benefited the most.
What are A-levels? They’re national exams in the UK that function like a turnstile for admission to higher education. They were canceled this year because of the virus, so instead teacher evaluations were used, but with the Authorities applying a “controversial standardisation process” to the evaluations with the result that a lot of students were downgraded, which amounts to saying “Soz no higher ed for you, off to the factory you go.” There’s a lot of anger.
Private schools increased the proportion of students achieving top grades – A* and A – twice as much as pupils at comprehensives, official data showed.
Gee, what a funny coincidence.
According to detailed analysis published by exam regulator Ofqual, the pattern in England has been similar to but less dramatic than in Scotland, where pupils and schools in disadvantaged areas were marked down the most harshly by the statistical model used to replace exams.
Pupils in lower socioeconomic backgrounds were most likely to have the grades proposed by their teachers overruled, while those in wealthier areas were less likely to be downgraded, according to the analysis.
I don’t know, maybe they have stats that indicate pupils in lower socioeconomic backgrounds are more likely to mess up in those final months, but then maybe that should tell them something about the finality of that turnstile.
For students from disadvantaged backgrounds on the cusp of attending higher education, more than one in 10 of those assessed as receiving C grades by their teachers had their final result lowered by at least one grade, compared with 8% for those from non-disadvantaged backgrounds.
Which translates to a lot of students abruptly shut out of higher education without having had the chance to try. It’s a gruesome situation.
Headteachers and pupils reacted with anger and disappointment, while the Labour leader, Keir Starmer, said “something has obviously gone horribly wrong with this year’s exam results” and suggested a Scotland-style U-turn – accepting all teachers’ recommended grades – should not be ruled out.
Thursday’s results confirm that 39% of teacher recommendations in England were downgraded.
“Parents, teachers and young people are rightly upset, frustrated and angry about this injustice. The system has fundamentally failed them. The government needs to urgently rethink,” Starmer said.
If you’re going to slam the door in people’s faces at least give them the chance to say their piece first.
I suspect at least part of it is the assumption that teachers in the schools that cater to the lower socioeconomic backgrounds are less competent teachers. I don’t know how it is in the UK, but in the US, those jobs are the least desirable from the standpoint of salary, they are perceived as riskier in many ways, and for many teachers, they are seen as less rewarding. Personally, I love teaching students who haven’t had many opportunities and see them suddenly have that ah-ha moment when something clicks, seeing them come out of my class with more than they brought in.
I also, personally, feel the teachers are usually a better source to evaluate how the students will do in higher ed than a single high stakes once-in-a-lifetime test. I think you could use the tests, if you must, but teacher ratings should be weighted higher, not lower. They are the ones who have observed the students through their education, seen them grow (or decline, or stay steady), and experienced their abilities in a variety of ways.
Too many places are downgrading teachers as evaluators and assessors. Yes, I realize teachers can have bias (conscious or unconscious), which is why the tests could be maintained as a helpful corrective, but overall, I think most teachers are better at evaluating their students than impersonal tests.
I have wondered for some time how much of the current denigration of the abilities of teachers (at least in the US; I don’t know if this is happening elsewhere) is related to the fact that teaching is now a female-dominated field. Just like secretary lost a lot of status when it became female-dominated.
In my day job I work for an organization that makes tests of English language proficiency for K-12 (such tests being mandated by federal law). Tests can provide useful data points when used correctly, but they’re just one data point, and shouldn’t be the sole source for high stakes decisions. Teacher judgments aren’t always reliable, but they shouldn’t be dismissed, and if you’re adjusting their grades you’d better have a damn good justification for how you do that.
While we’re on the subject, the “Justice” Department claims that Yale discriminates against whites and Asian Americans. #YaleSoBlack?
Yes to both. From my perspective as a Murkan, the A level system is way too “one chance and one only” to be a good system. I guess it’s the norm – I’ve read some about exams-obsession in Japan and France and if anything they’re even worse – but it’s horribly harsh.
@Ophelia As someone who does not have A levels and now has a PhD, I object a little to the ‘one and done’ idea. It’s harder, no question, but it is not the only path. I left school at 16, got an apprenticeship and did my post-16 qualifications (BTEC instead of A levels) day-release at a local college, then went on to do my degree part-time at university as part of my training. I only re-entered fulltime education as a postgrad.
But take this on board – things may have changed but when I did my GCSE’s (exams taken in the equivalent of 10th grade I think, at age 16 anyway, and separated by subject) many subjects had grade quotas. Yes, quotas. In many of my GCSE’s there was only one A available per school (no A* in those days) for each course. In my Drama GCSE, the outside examiner was not shy about telling us this. I got the A and I was furious. Yeah, it’s nice to have an A but my friend S was at least as good as I was and unlike me she was planning to continue her educational path into performing arts. I felt guilty about it. S deserved an A and needed it more than I did. But it was stupid to have quotas anyway.
@iknklast The Ah ha moment is the best. I used to teach a short lab course as part of a mandatory molecular genetics module at a heavily black university and many of the kids did not want to be there. Some thought they were too cool to care about it.
Until… the moment when the admittedly tedious process of extracting DNA came to an end, they coiled up the DNA molecules out of small tubes onto glass rods and they all shrieked like delighted 4th graders. The surprise that DNA could actually be visible was pure joy. Then came the questions on all the things I’d been talking about on how the process worked and their engagement with it was the best thing ever.
I should add that my nephew has just done a similar thing. He went through an apprenticeship at a well-known quango and they are now sponsoring him to go to university.
Claire, one of my favorite moments was when I took a group of students from North Texas on a field trip into southern Oklahoma. We took tents and sleeping bags, and did an entire camping experience while learning Botany, Ecology, and Geology (it was a multi-disciplinary trip; I did the Botany and Ecology, and a Geology teacher did the Geology). One of the students had never been out of the city, and he was racing around with so much delight at the new sights, it was almost like a kindergartner, but these were college seniors. Watching him race toward a beautiful creek, or stare in awe at a gecko? There’s nothing like it.
I’m a little confused at what the supposed issue is. Is there some sort of grading curve that artificially inflates the scores of students from private schools, such that two students with identical answers get different scores? That would certainly be injustice.
Or is it that students from private schools perform better (i.e., choose more correct answers) than others do? That seems like one reason people pay to send their kids to private schools in the first place.
Nullius,
From what I understand, in the absence of the test, teachers are being asked to assign grades to their students. However, rather than accepting those grades on their face, the governing body has created an algorithm that adjusts those grades based on various criteria. The effect of that algorithm has been this:
In other words, discrimination by algorithm.
@Nullius It’s a bit of a black box, which is part of the problem. Teachers are saying that they don’t feel their views are being properly accounted for and because I assume the “algorithm” everyone is getting upset about is a machine learning method or similar, it is suffering from the same problems other “solutions” do and it’s a black box.
They’re using historical data to train the machine as well as the teachers assessments and the child’s prior performance. As you might imagine, students in poor schools historically perform worse than private schools on average. It’s won’t be clear how the machine has decided which factors are the strongest predictors. It’s not fair to use that to predict an individuals grade and then assign that grade to the student. This is not the correct way to use these kinds of machines.
To use an analogy – there are machine learning tools out there to predict risk of specific cancers. Some people in the medical record database might be identified as very high risk. Doctors use this information to target those people for more scrutiny and additional screening. They don’t tell their patients they have cancer based on the results from the computer.
Claire @ 4 – ah thank you, I always wonder about that. I’ve gathered from the way people talk about it (including in fiction and memoirs and the like) that it is or at least feels like a “use it or lose it” situation. I can never quite figure out why – why not have more than one possible path? I’m glad there is more than one.
Ooooooh, so the exams weren’t held, replaced by teacher-suggested grades modified by a predictive model. Is that it? ‘Cause that sounds unjust prima facie. I can think of many purely arithmetic reasons that could produce unfair outcomes.
Nullius,
Yes, and yes.
To be fair, like everyone these days, testing organizations are struggling to deal with an unprecedented situation. Depending on the nature of the test, there are at least three areas that could cause problems: delivery, proctoring, and scoring. Some organizations are trying to convert to an online test–that was the solution that the College Board tried to implement, and it caused a lot of problems. Other tests, such as the SAT, have pretty much shut down. In our case most of the testing was already completed for the year when the shit hit, but we’re dealing with some unappetizing choices for the coming year.
In the case of the A-Levels, it seems like they went with a hastily-crafted plan without thinking through all the potential pitfalls. Just like so much else these days.
@Ophelia #10 I guess they de-emphasize it because they want kids to think that way, i.e. actually make an effort. It irritates me no end. In fiction, it’s unnecessary information if it’s not relevant to the plot, and would be dead boring. Most Brits who went the traditional route have no idea there are other ways, and if you explain it then, in the classic British way, they immediately assume us BTEC folks are inferior.
This stuff about how they’re assigning grades is really going to hit kids who are dirt poor but ambitious to climb out of the poverty cycle. They’re usually the outliers in a school’s distribution of grades. Their performance will be seriously distorted by any weighting schema for grade assignment.
There should have been a way to test these kids without exposing them to COVID. Most subjects could be done online (and open book) with the copy-paste option blocked for each answer box. For in person exams such as you might do in chemistry, it can be worked around.
In my final year in high school, over the Christmas holidays, some little arse set fire to the school. He started the fire in a room that shared a wall with the fume hoods of the chemistry lab where several volatile chemicals were stored. The resulting explosion plus the fire rendered a whole third of the main school building unusable.
They made a submission to the exam board for alternative tests and swapped out the practicals. Exams were administered in other parts of the school where possible but for the most part we did it at home (take it home, fill it out, return the next day, or for languages, we recorded tapes). Yes we were still attending but in a complicated calendar of days on and off so that all the years (grades) could be in school some of the time. But there was no internet as we know it so we couldn’t email in our test papers. Tim Berners-Lee had only just invented the WWW about 2 years beforehand. But the school and the exam boards figured it out.
Obviously this was a one-off situation but the point is that the alternate exams already existed at the exam boards (MEG and Cambridge I think). They were prepared for an eventuality where a normal test couldn’t take place.
There doesn’t seem to be a great solution to the problem. Forego the tests entirely, and those whose aptitude exceeds their grades get screwed. Not require the tests, and those who don’t take them are viewed as suspicious. Require the tests, and screw those who can’t take them. Those are bad solutions. The “have a computer model predict performance” solution sucks more than all of them.
Like, right off the bat, suppose we have two straight-A students with identical demographics. From what data does the computer possibly recognize that one of them is a diligent student and the other is a diligent genius?
I recall vaguely an essay by Pierre Bourdieu who showed, with the use of statistics, that students who had not been to one of France’s elite schools or who came from ‘humble’ backgrounds, or both, tended to be marked down, mostly, it seemed, quite unconsciously by the very human graders, who I suppose assumed that students with such backgrounds couldn’t be ‘top-drawer’. Now, in Britain, they have computers to do the same, again quite unconsciously. Why have unconsciously-biased human graders, when computers can do it for you? The march of science, I suppose.
By the way, Nullius, out of the blue, do you live in Japan, as I do, or have you lived here? I liked your comments on the rhythms of Japanese, which speakers from stress-timed languages like English in particular find extraordinarily difficult to catch. (It works the other way round, too, of course.)
On thursday Ofqual published a document explaining its methology. It can’t be too complicated a system, the document is a mere 319 pages.
Three hundred and nineteen fucking pages! That’s 13 pages more than a quarter of the first edition of War and Peace, and only one page short of a quarter of a standard KJ Bible, and I daresay harder to get through than either of them.
https://www.theguardian.com/education/2020/aug/14/punishment-by-statistics-the-father-who-foresaw-a-level-algorithm-flaws
My daughter completed High School and went to work. She has worked fast food, banking, and now in a good government job. She was told by her manager that although she was a great worker and applied an enormous amount of thought and analysis to her job, she could never rise any higher without a Degree.
She is happy in her work, loves being a mum as well, and doesn’t want “a Degree” just for the sake of promotion. So, being the good analyst she is, she dug into her department head’s background and asked him “How did you get this high up when you don’t have a degree?”
Love that girl.
Well don’t leave us hanging, what did her manager say?!
Good for her Roj. I hope she goes far.
I was told I was ‘lucky’ to have got a scientists job with only an MSc – and that i’d probably be the last person to do so in the organisation. When I asked why, they said that whenever a position was advertised they had so many applicants that short listing only the best of the PhD’s who applied made life easier. Pointing out that some of the best scientists in the organisations history had BSc or MSc degrees didn’t gain any traction, or brownie points.
It’s not just that it was a hierarchy and a class system that pissed me off, it’s that it was such a lazy reason to implement it that really got me.
I now work in an environment where people are employed for character, personality and a demonstrated ability to learn. Best place I’ve ever worked by far and despite not being perfect (no where is), I’d rate it against any other organisation in the world in our field.
Like Claire I did a BTEC rather than A levels before doing a degree and PhD. I was strongly advised to take A levels instead, but I didn’t listen to anyone back then, either. It was the right choice for me. There are other paths, but they are not always considered as valuable as A levels. My degrees are in Computer Science and the BTEC, being quite practical, was good preparation for that. Not all courses accepted BTEC as valid at the time.
The issue with the grading algorithm is indeed that it adjusts for the past statistical performance of schools and downplays the performance of individuals. It’s grossly unfair. Students can apply to take exams instead later in the year, but they have to pay (I think it’s around £300) which of course also disadvantages students from poorer areas.
Claire:
I think I know that school! Are we stalking each other again?