Still grading lessons? A triumph of experience over hope

Hope is a good breakfast, but it is a bad supper.
Francis Bacon

To paraphrase Rob Coe’s seminal research, yesterday’s National Teacher Enquiry Network (NTEN) conference at KEGS in Chelmsford was a triumph of experience over hope. just hoping we’re doing the right things is potentially worse than useless: it might be downright damaging.

This was a gathering of teachers and school leaders from a wide range of settings, all of whom are focussed on trying to move from a ‘hopeful’ approach to improving teaching and learning to a more expectant one. Finally there might the first faint glimmers of a new evidence-led dawn.

But, hang on, what happens if we don’t have hope? Surely that’s a pretty bleak outlook?

I’m not so sure. Consider the following examples of hopeful teaching and school leadership:

  • “I hope you all get on to the final task by the end of the lesson.”
  • “I hope behaviour in our school gets better.”
  • “I hope our exam results improve.”
  • “I hope that what we’re doing works.”

Not really all that optimistic, are they?

As Coe says,

[I]f it is true that despite the huge efforts we have made to improve education not much has changed, there are important lessons for us to learn. One would be that effort and good intentions are not enough; we have to work smarter, not just harder. Another would be that we must look carefully at the strategies we have been using to improve, and replace them with some different ones. A third lesson is that a more critical and realistic approach to evaluation may be required. An uncritical belief that things are improving may be comforting, but is ultimately self-deceiving and unproductive.

And one of the pieces of cold comfort addressed at this conference was that provided by the graded lesson observation.

Here are some of the attitudes that you may encounter towards lesson grading:

  • It’s not perfect, but then nothing is.
  • The appraisal system requires that we evaluate teacher effectiveness, so we just have to do it.
  • We may not like it, but it’s here to stay.
  • Grading lessons is best practice. it must be: Ofsted do it.
  • Lesson grades provide a baseline, we can then focus on improvement.
  • It’s easy enough to spot whether learning is taking place.

All these statements embody the hit and hope approach that is so prevalent in our schools. On the contrary, I would argue:

  • Of all the imperfect methods of we could use to evaluate teacher effectiveness, this is the least perfect. They are completely unreliable. We’d get a more statistically valid and less biased assessment if we flipped a coin. Here are the killer stats from the multi-million dollar MET project which utilised way more rigorous training & observation protocols than we do in England: if a lesson is given a top grade by one observer, there’s a 78% chance a second observer will give a different grade. And if a lesson is given a bottom grade, there’s a 90% chance a second observer will give a different grade! Another robust piece of research found that fewer than 1% of lessons judged inadequate are genuinely inadequate; only 4% of lessons judged outstanding actually produce outstanding learning gains. And overall, 63% of judgements will be wrong.
  • Yes, teachers must be held accountable, but is there another way? Joe Kirby argues that there is.
  • It’s not here to stay. I predict that within a maximum of 3 years, Ofsted will no longer be allowed to grade lessons. it may be a lot sooner than that.
  • Ofsted are not interested in improving teaching and learning. They believe (wrongly) that they can measure learning for the purposes of accountability. But we should be interested in fattening the pig, not weighing it.
  • Lesson grades do not provide a reliable baseline. All they reveal are our bias and preferences. Believing in the accuracy and impartiality of lesson grades is inherently dishonest.
  • Actually, it’s impossible to spot whether learning is taking place. As Nuthall says, “Learning does not take place in the here and now of classroom activities.” What we see is performance, or as Coe puts it, ‘poor proxies’ for learning. There is a vast weight of evidence to support this, it’s not just me making it up.

Liam Collins argues persuasively that in a 5 period day, even the best teachers are capable of teaching a full range of lessons from inadequate to outstanding. Anyone can have a bad day, or a good one. He found that the correlation between lesson grades and exam results was poor: teachers who delivered outstanding performances in lessons did not produce outstanding outcomes. And vice versa.

In his most recent Ofsted inspection, Liam explained to the lead inspector that he did not grade lessons. Instead, a member of his senior team would be walking the school, dropping into lessons every lesson, every day. But, said the lead inspector, one of the teachers you have identified as being ‘good’ has just been seen teaching an inadequate lesson. Liam’s response is wonderful: don’t you think we know our teachers better than you could ever hope to know them from a 20 minute observation? Eventually, they concurred, agreeing that his systems for quality assurance made graded lesson observations unnecessary.

Where does that leave the excuse that we have to grade lessons because Ofsted require us to do it?

This is my bottom line: a belief in the validity of lesson grading is akin to a belief in witchcraft. And for all the difference it will make to improving teaching and learning, you might as well be doing Brain Gym.

But don’t despair. We can still use classroom observation to make a difference. The best way forward for any school wanting experience to triumph over hope is to sign up for membership of NTEN. (Please note – I have no affiliation, I get no kickbacks; I just think it’s really good.)

If you’re interested, these are the slides I used in the presentation I gave at the conference.

And here’s a video of me sounding very Brummie
http://youtu.be/8Rt_OXsOnR8

Finally, I owe a huge debt to the work of Robert Coe, Graham Nuthall and Robert A Bjork – these are giants on whose shoulders I’m privileged to stand.

Related posts

Has lesson observation become the new Brain Gym?
Why can’t we tell a good teacher through lesson observations?
Where lesson observations go wrong
Don’t trust your gut: a little bit more on the problem with grading lessons
How can we make classroom observation more effective?
The Cult of Outstanding™: the problem with ‘outstanding’ lessons

29 Responses to Still grading lessons? A triumph of experience over hope

  1. […] via A triumph of experience over hope | David Didau: The Learning Spy. […]

  2. I agree that grading lessons is nonsense but I do think there is a role for lesson observations. I find them useful just to see if the wheels have come off. If you walk into a classroom and some kids aren’t doing what the teacher has asked them to do then this is a bad sign. True, if they are doing what has been asked then you cannot conclude that they are learning but I would suggest its a necessary, if not sufficient, requirement.

    I once observed a struggling teacher and found that she was writing on the board with a green pen which the football lads, who were all sat together on the back row, could not read. I prescribed a black pen and a seating plan.

  3. This is good stuff.

    Rob Coe’s research often points to the efficacy of the kind trendy teaching methods so despised by a number on the echo chamber; meta-cognition, collaboration etc.

    • David Didau says:

      I might be wrong, but I can’t think of anyone who despises meta-cognition – thinking about content is inseparable from writing about content. I think perhaps the bit that some people might ‘despise’ is the idea that meta-cognition is some sort of contentless, domain free skill which can be taught and transferred. As far as I’m aware, no one is really claiming this, are they?

  4. You’ve picked one but you could also include “peer to peer”, “, “group work” collaboration etc do they not conform to the general epithet of “trendy teaching methods”? I realise we are now in the grey area of semantics but they seem to conform to that general mode.

    Not sure that Meta-cognition is thinking about content but rather “knowing about knowing”, which is slightly different.

    I’m not entirely convinced that the executive functions of the brain are in some kind of correlational relationship with memorised knowledge. Clearly some knowledge is essential but I think it’s easy to confuse expertise with thinking. Do expert thinkers become stupid when they think about stuff they know nothing about?

    Do we really know as much as we think we do about this stuff?

    Anyway I liked the article. The little comment about trendy teaching was a bit of a pointless self indulgence really.

    • David Didau says:

      Yeah OK, but I didn’t make a comment about trendy teaching. I was genuinely surprised Coe’s research is being used to endorse such things.

      • No you didn’t make reference to trendy teaching that was me.

        I was thinking aloud about the document you point to, which has an interesting graphic on it, which points to “peer to peer”, “, “group work” collaboration and independent learning (or something similar meta- cognition) etc. Here’s the paragraph

        “More positively, some other strategies look promising: giving effective feedback; encouraging meta-cognitive and self-regulation strategies; using peer tutoring/peer-assisted learning; setting homework (for secondary age
        pupils); promoting collaborative learning; and the use of phonics in learning to read.”

        I agree with it but it sounds a lot like the Constructivist methods debunked by Kirschner (or an attempt at debunking them anyway).

        I agree with Rob Coe. Matt O’ Leary writes well on this subject as well

  5. dodiscimus says:

    I am no fan of Ofsted’s use of lesson observations, nor the way they are often used in school performance management, so this is not a response attacking any of the main points of your post but I worry at the way the educational community has a tendency to get hold of a few, prominent pieces of research, so that they take on a life of their own and get used as evidence in ways that cannot be justified. I’ve blogged briefly about Coe and the MET and Strong, Gargani & Hacifazlioğlu papers ( http://wp.me/p44DHA-Q ). The inaugural lecture by Coe (2013) you link to has only one reference to lesson observation that I can see – it’s says this about the MET research:
    “Their three main sources of evidence (test score gains, student ratings and classroom observation) are shown, if done well, to contribute to a single construct of effectiveness, but also to complement each other with each providing a unique contribution.”
    The MET paper you link to is the discussion of the “technical methods and analyses” – it’s pretty hardcore. The “policy and practice brief” is much more palatable (link’s massive – see my blog or Google it) and it’s main point on observations is that a pretty good level of reliability can be achieved if more than one lesson is observed by at least two different observers. Conversely, one moderately trained observer will be pretty unreliable. This really matters because it suggests that Ofsted judgments about any individual teacher may well be rubbish, but some version of an internal school observation protocol could be designed to be quite reliable. This is not an argument that this is the best option, but it remains an option.
    Strong, Gargani & Hacifazlioğlu (2011) were trying to show that 5 minute observations could be reliable (no lie!). However, their 3rd experiment switched to whole lesson observations but used the CLASS observation tool which is quite biaised to constructivist teaching methods. I’m not convinced this paper is useful.
    Finally, Kirschner, Sweller & Clark (2006) gets a mention in the comments. This paper makes a very strong argument against minimal-guided teaching strategies not the full cohort of constructivist ideas in teaching. So it does attack discovery learning, problem-based learning and inquiry learning but not collaborative learning, groupwork or peer assessment as implied in the comment.
    I hope that’s helpful – it’s not meant to be destructive.

  6. […] Are we any clearer? Ofsted explain what they do and don’t do How can we make classroom observation more effective? Still grading lessons? A triumph of experience over hope […]

  7. […] and usefulness of graded lesson observations of teachers in the classroom in his article “Still grading lessons? A triumph of experience over hope”. He says that, as teachers, us simply hoping that we are doing the right thing is not only […]

  8. […] blogger, David Didau, has written several posts on this subject which you can read here, here, here, here, and […]

  9. […] Hope is a good breakfast, but it is a bad supper. Francis Bacon To paraphrase Rob Coe’s seminal research, yesterday’s National Teacher Enquiry Network (NTEN) conference at KEGS in Chelmsford was a triumph of experience over hope.  […]

  10. […] this prove?  The attack led by David Didau on the conflation of learning and performance, and on grading lessons, seems increasingly irrefutable.  Learning happens when students have to think hard and changes […]

  11. […] troublingly, that performance might not lead to learning. This has led me to the conclusion that grading lessons is wrong and also that much of what we consider to be outstanding practice encourages a focus on rapid […]

  12. […] Watching the watchmen: Is Ofsted fit for purpose? What I learned from my visit to Ofsted Still grading lessons? A triumph of experience over hope […]

  13. […] an awful lot of completely unnecessary stress. Since then I have put together, what I feel is a pretty convincing case on why it is wrong (on every conceivable level) to grade individual […]

  14. […] Teacher Talk: the missing link The shocking mediation of Ofsted criteria by ‘rogue’ inspectors Still grading lessons? The triumph of experience over hope […]

  15. […] 3) Stop judging lessons. The average teacher will teach between 750-800 lessons per year. Sampling 2 or 3 of them is no way to judge effectiveness even if the evidence about the absolute lack of validity and reliability of lesson grading wasn’t so compe…. […]

  16. […] The death knell has sounded for graded lesson observations. Ofsted (at least as far as schools are concerned – FE is another matter. )have drawn a line […]

  17. […] Still grading lessons? A triumph of experience over hope. Hope is a good breakfast, but it is a bad supper. Francis Bacon To paraphrase Rob Coe’s seminal research, yesterday’s National Teacher Enquiry Network (NTEN) conference at KEGS in Chelmsford was a triumph of experience over hope. just hoping we’re doing the right things is potentially worse than useless: it might be downright damaging. This was a gathering of teachers and school leaders from a wide range of settings, all of whom are focussed on trying to move from a ‘hopeful’ approach to improving teaching and learning to a more expectant one. Finally there might the first faint glimmers of a new evidence-led dawn. […]

  18. […] in particular? The first thing we need to think carefully about is the interview lesson. Although we know that there is no reliable way to grade lessons, we still want to have some idea of how a prospective teacher is likely to perform in the […]

  19. […] teach a class of children whom they have never met before. Even though most people now accept the impossibility of reliably grading lessons, we still believe that we can find out a lot about a teacher’s personality: their warmth, […]

Constructive feedback is always appreciated

%d bloggers like this: