I’ve written quite a few posts about exams and assessment in the last year. Some of the earlier ones were a bit of a rant; more recently, I think they are more measured. Through this blog and other related activities such as working with the Headteachers’ Roundtable, I’ve been able to meet some of the people from exam boards and OfQual. It is very interesting to talk face to face, to hear their perspective and to discuss the mechanics of the examination system.
My main contention is that teachers, politicians and the public at large need a much better understanding of the mechanics of the system so that there is more awareness of the inherent limitations and margins of error. Of course people are entitled to demand a system that they can have full confidence in…but that doesn’t mean we can wish away or mask the issue of error as if it isn’t there.
Too often the debate descends into conspiracy theory territory. I firmly believe that everyone involved in the system sets out to act with integrity and seeks to do the best they can for students and schools. However, we are all under intense scrutiny pressure and so I also believe that people are motivated to protect and defend themselves, even when they know they’ve made mistakes or where standards are below what they could be. That only serves to undermine confidence rather than build it.
Fundamentally we’d like to think of the exam system as predictable and linear, with clearly defined goal posts and boundaries; in practice it is non-linear; the boundaries are fuzzy and the goal posts are not static. We want our system to be such that students are competing with themselves – simply gaining credit for doing their best on their own terms. However, in reality, our exam system measures how well individuals perform compared to everyone else. This is how it has always been and is how most tests in most schools are conducted too. Essentially, the national exams system recreates many of the processes that teachers use in schools in designing tests, but scales them up to a very large scale. I’m coming to terms with this, starting to see the flaws as an intrinsic aspect of the system we all operate.
And yet, it feels wrong. We want some firm lines in the sand; some benchmarks that won’t move. But just try doing that; all standards are ultimately defined by comparison as I suggest in my most recent post on this subect: Assessment, Standards and the Bell Curve. Basically, if you don’t want to compare students, then don’t ever give them the same test. There are serious concerns and real issues around labeling children as failures… but that is a fault with a wider values system that goes wider than schools and schooling. Exams are technical and procedural; the value we place on their outcomes is something else. You can’t make the high jump fairer or easier or mask the range of performance in any cohort of high jumpers; it is what it is. We need structures that put exams in context, building self-esteem through a celebration of a wider curriculum; a whole education.
However, as I’ve said before, given how much rides on the outcomes from exams, we should do all we can to get them right. I started getting agitated about exams after a few issues emerged at my school over the last two years. See here:
Sour Grapes on Results Day? I’m getting my excuses in early.
Battles with AQA over English A Level Marking have shaken confidence in the concept of standards
I’ve also written more broadly about how much emphasis is placed on assessment data, far beyond it’s capacity to be meaningful, and protested about our tendency to oversimplify data.
The Data Delusion: On average, it’s a bit more complicated
The Folly of Narrow Newspaper League Tables
I’ve tried to give an alternative perspective on how data, assessment and achievement are viewed – to promote some discussion
Again: Assessment, Standards and the Bell Curve and also That Gap isn’t getting narrower: Now What?
I’ve also tried to present some solutions.
In particular, I’ve tried to promote the idea that assessment in the form of exams can and should be redesigned to allow for progressive achievement within a wider framework that gives value to a rounded education:
Thought for the Week: Exams are only part of the Story
Exams Debate is Far Too Narrow: What are the alternatives?
Towards a Proper English Baccalaureate – which links to the work of the Headteachers’ Roundtable on an idealised progressive qualifications system and a more pragmatic ‘ready to roll’ system.
Summing this up and adding some more, my agenda for exam reform would include the following:
1. Policy for national examinations should be taken completely away from election-cycle politics or the assessment market place where Exam Boards seek competitive advantage. A central independent agency that answers to OfQual, should set out the rules for how exams are to be designed, taking account of the nature of various subject disciplines, research evidence and the views of different stakeholders in the profession. The decisions taken would include the issue of modularisation, the weighting and nature of coursework elements and the structure of exam papers. These decisions must not be taken by DFE under the guidance of amateur politicians.
2. Exams results should we awarded in the form of points, not grades. We must remove artificial cliff-edge outcomes. With points would come an understanding of the concept of margin of error. Different assessment processes could be shown to have varying degrees of certainty that could be referenced. For example, 80/100 and 79/100 are neighbouring marks which would be broadly equal within the margin of error of, say +/- 2. However if 80/100 gains A and 79/100 gains B, assumptions will be made that are not true. Grades have got to go because the boundaries are inherently changeable and blurred.
3. Public examinations should form a part of a wider profiling of student achievement through a national English Baccalaureate system, with online transcripts designed to follow students as they progress through the system. We all recognise that exams don’t tell the whole story of a child’s capabilities so we should back that up by creating a system that gives formal recognition to more than just exams. The English Baccalaureate that I have in mind, could be transforming.
4. All schools should be mandated to release teachers to train and serve as examiners, provided that they meet stringent standards. Examining must be conducted by expert people with good judgement so that the qualitative aspect of learning in a subject is given value beyond the limited confines of a rigid mark scheme which leads to the narrow hoop jumping we’re all so tired of.
5. Examination Appeals should be heard by expert people independent of the original awarding body and re-marking should be done blind – ie without seeing the original marker’s marks or comments. It should be recognised that if localised school-specific cohorts generate anomalous results, a local investigation is the only way to establish the true story of achievement, rather than using national level statistical remedies that affect all students equally. If an exam board sets a bad exam, there should be an acknowledgement and redress.
6. Performance tables must include a broad range of measures that are given parity. They should be designed so that perverse incentives to narrow the curriculum and to operate elaborate window-dressing interventions are eliminated. We must strive to educate the public and the media to embrace the complexity of school outcomes information. Single-figure results tables should be outlawed, regarded as unacceptable, tantamount to public deception!
Many of the issues with our exam system can be traced to the undue, obsessive emphasis on measurable outcomes from the system rather than the quality of inputs and processes. That too must change.
Other policies: I would add a strong line on curriculum scrutiny in any accountability process, ensuring that breadth is maximised and I’d move the UCAS system to be a post-A2 process. More on those things another time.
Courtesy of Tom Sherrington at headguruteacher
Comments
No responses to “Exam reform. Another blog manifesto”