The rise of the unscrupulous optimist


“Optimism, n.: The doctrine or belief that everything is beautiful, including what is ugly.” Ambrose Bierce, The Devil’s Dictionary

Education is a project filled with hope. We stand, framed heroically against the setting sun and scan the horizon for new stuff to transform the tired, outmoded, factory clamour of the past and hope – oh, how we hope – that everything will be better. But our forward-looking, progressive stance means we can all too easily miss seeing a landscape littered with failed ideas and the scorched ruins of unscrupulous optimism.

Here are a couple of recent examples for your consideration:

The College of Teaching

Tom Bennett wrote about the increasingly embarrassing failure of the much-trumpeted College of Teaching and the commitment of those invested in the idea to plod listlessly on until the crack of doom in the mad hope that teachers will one day be convinced that what they really need, what will solve all educational ills, is an annual subscription to yet another authoritarian professional body. I remain deeply unconvinced.

Education technology

I wrote a 3 part exploration of how and why true believers in the power of digital technology to transform education take it amiss when anyone is even mildly sceptical. One of the most powerful motivations for embracing the new is, well, because it’s new! The EEF report on digital technology called this ‘the Everest fallacy’. They also pour scorn on the idea that, despite the failures of the past the next big thing is just around the river bend:

After more than fifty years of digital technology use in education this argument is now wearing a bit thin. We need a clear rationale for why we think the introduction of (yet another) new technology will be more effective than the last one. The introduction of technology has consistently been shown to improve learning, the trouble is most things improve learning in schools when they are introduced, and technology is consistently just a little bit less effective than the average intervention. [My emphasis]

This tendency to will improvement and transformation into existence just because we want it to work was the topic of Rob Coe’s 2013 lecture, Improving Education: A triumph of hope over experience. In it he points out the failure of this approach:

Despite the apparently plausible and widespread belief to the contrary, the evidence that levels of attainment in schools in England have systematically improved over the last 30 years is unconvincing. Much of what is claimed as school improvement is illusory, and many of the most commonly advocated strategies for improvement are not robustly proven to work. Even the claims of school effectiveness research – that we can identify good schools and teachers, and the practices that make them good – seem not to stand up to critical scrutiny. Recent growth of interest in evidence-based practice and policy appears to offer a way forward; but the evidence from past attempts to implement evidence-based approaches is rather disappointing. Overall, an honest and critical appraisal of our experience of trying to improve education is that, despite the best intentions and huge investment, we have failed – so far – to achieve it.

I think the way the growth mindset is interpreted and enacted betrays an unscrupulous optimism in many setting: we wish so fervently for all to achieve that where we don’t end up merely mouthing homilies, we fool children with the toxic belief that everyone is special and all should win prizes.

None of this is to say that hope is harmful, just that it can infect our thinking in subtle but pernicious ways. I wrote here about the ‘best case fallacy‘, the beguiling idea that because we all mean well, because we want so much to be right, that all will be well, whatever our endeavour. Roger Scruton calls this, “a kind of addiction to unreality that informs the most destructive forms of optimism: a desire to cross out reality, as the premise from which practical reason begins, and to replace it with a system of compliant illusions.” How delighted we are to cross out reality and dwell in the shining spires of a perfect future where all our happy dreams will be realised… soon.

This is where optimism becomes unscrupulous. The power of positive thinking is the power to blind ourselves to reality. The unscrupulous optimist is a gambler and, like all gamblers, convinced they can beat the odds. Too many in education have calculated the best case and refused to acknowledge the worst. Failure to prepare for the worst is a certain form of insanity.

Instead, we should always seek to dwell in a world of constraints. Whenever we try to change those constraints we should know that the attempt will be difficult and the consequences uncertain. We would always do better to adjust ourselves to reality than expect reality to warp at our whim.

Optimism is a constant temptation – of course we want to believe our ideas will work and our good intentions will win through, but such wishful thinking is dangerous, possibly even poisonous. To be clear, I’m generally an optimistic chap. I tend to act as if things will turn out for the best, and sometimes they do. Even when they do I can look at my three-quarter empty glass as half full. This is a trait I try to fight.

The most unscrupulous of optimists will believe they are somehow capable of overcoming all of the impossibly unpredictable pitfalls and so guarantee a less than optimal outcome. The unscrupulous optimist will read all this* and triumphantly cry, “But I’m a scrupulous optimist!” and continue blithely trailing havoc in their blissful, ignorant path.

Pessimism is a useful trait for teachers. Judicious pessimists know all too well the mess we can make when we hope all will go well, and they actively anticipate – and so often avoid – failure. For those of us who acknowledge our all too human fallibility, here’s a checklist of questions we might use to help us avoid unscrupulous optimism and prepare ourselves for the worst rather than wishing for the best:

  • What is the real root cause of the problem you’re trying to solve?
  • Have you considered other possible reasons for the problem?
  • Have you sought out sources and evidence which contradict your beliefs?
  • Have you allowed for dissenting opinions to be voiced and considered?
  • Have you considered the weight of time, resources and credibility you or others have already sunk into this course of action?
  • Are you viewing the motivations and capabilities of yourself and others realistically?
  • How might groupthink and other social biases be influencing your decisions?
  • Have you encouraged others to criticise and suggest problems with your plans?
  • To what extent is your decision based on an unproven opinion?
  • Might your decisions be anchored by possibly irrelevant information?
  • Do you really understand the data you’re using to inform decision-making?
  • What perverse incentives might you be creating?
  • Certainty can blind us to alternatives. How confident are you that your decisions are correct?
  • What would be the consequences of not taking this course of action?
  • Have any other schools tried this course of action? How many were still doing it three years later? What were the results?
  • Who could you ask to help you spot the biases and flaws in your thinking?

*Actually, the most unscrupulous won’t even bother to read it.

16 Responses to The rise of the unscrupulous optimist

  1. howardat58 says:

    O that governments and quangos may one day ask these questions. This used to be the way of the Civil Service, but they are often bypassed, as politicians have direct influence over the quangos (ofsted).

  2. Polly D says:

    This article made me smile as it rang so sadly true. It’s also grating for children to feel that they have to be permanently optimistic and positive about themselves.

  3. Matt says:

    True, I did not bother to read it. I do appreciate your style of writing, as always, and your counter debate, although optimistically I think you are wrong.

  4. Kapitza says:

    Optimism is bad!

    Pessimism is good!

    Freedom is slavery!

    • John says:

      You are employing a common persuasive technique of using equivalent structure to imply equivalent veracity. It is persuasive at first glance, but that doesn’t make it true. The current research suggests that optimism is more likely to lead to close mindedness than pessimism; that it puts the focus on outcomes rather than on process.
      The idea that positive outcomes are necessarily the result of optimism just isn’t borne out by any research I’ve seen. Pessimism, as the article note, has been shown to lead to things like group think and an unwillingness to consider other options.
      So, you could make your statements a bit more accurate (although still an oversimplified distillation of the facts).

      Freedom is good
      Optimism is slavery
      Pessimism is freedom

  5. teachwell says:

    I am really glad you wrote this as I do think the almost universal rejection of empirical evidence for optomistic ideas has affected behaviour management policies negatively. This is really useful for a post I wish to write this week. Thanks David!!!

  6. Joe Miller says:

    David, I think this post made some good points about expectations. It feels at time like we are looking for something or someone to rescue education. EdTech, personalized learning, micro-school, etc… One part of the post that stuck out to me was the discussion of constraints. I agree we must be well aware of our constraints. In fact, rather than advocating for “out of the box thinking” we should advocate for understanding the limits of the box and designing the best solution. To some folks the idea that we are constrained at all is just plain wrong. Their belief is that we must wholly redesign our system to eliminate constraints. This turns out to be naive. What folks tend to lack is a system (or framework) for testing new ideas for results. Instead, we we implement fast and learn slow (wording borrowed from Tony Bryk). Thanks for post, Joe

    • Kapitza says:

      Hi Joe,

      Intrigued by your comment “rather than advocating for “out of the box thinking” we should advocate for understanding the limits of the box and designing the best solution.”

      Could you give a concrete example?


      • Joe Miller says:


        Here are a couple of examples. A school district I know wanted to discuss and test solutions to deal with low performance among high school math students. One of the barriers to coming up with a testable solution was the insistence among a small group of participants that wanted to explore trying to change attendance patterns to re-introduce more affluent (and higher performing) nearby town that had stopped sending their students to this school and built their own high school. Not acknowledging the design constraint in this case slowed down the discussion about workable solutions. Another school I know was planning to “re-design” the school. In this case they had to acknowledge many design constraints. Here are two: (1) the school included mostly second language learners, but the program for addressing language was not open for change (this was a reality of local politics. (2) While the school was challenged to “re-design” their strategy for success, the measure of success was not changing. In other words, the school was encouraged to make radical changes that would result in increased performance on assessment. The school was not being judged on how engaged students were or how happy parents were.

        Regards, Joe

        • Kapitza says:

          Thanks for your reply, Joe, really interesting. I will reply properly, just a bit snowed under at the mo.


  7. Lucky Pierre says:

    I like the questions you framed at the end. I think you can still be optimistic and ask questions like that and be realistic. The other end of the spectrum are indifferent cynics and pessimists who pooh pooh and scoff at every idea just because it’s been done before and didn’t work (not that I’m saying this is what you are doing- just that there are people out there like this. Also I’m thinking as I type so not necessarily disagreeing with you, just formulating ideas.) Where does that get you though? Just because something didn’t work once, doesn’t mean that’s the case every time right? (that doesn’t mean ignoring the data- it means exploring the variables) Schools are successful when teachers are given autonomy and an environment is created where errors are seen as opportunities to learn- John Hattie’s visible learning studies found this. If you are constantly putting the brakes on because of fears of repeating mistakes of the past, you lose too. Optimism mixed with critical thinking? Can you not have judicious optimism as well?

    • howardat58 says:

      “Optimism mixed with critical thinking? Can you not have judicious optimism as well?”
      Isn’t this what we call “pragmatism” ?

      • Lucky Pierre says:

        Yeah maybe. You could be a pessimistic pragmatist though.

      • David Didau says:

        Judicious optimism is probably OK. Being judicious would mean that you would carefully evaluate a situation and reach a considered opinion. Optimism is a belief that all will turn out fine in the end and I’m not sure that this is ever really all that judicious.

  8. IanH says:

    To put this in the terms that I’ve so often had to use in the classroom; hoping for the best case results, and pinning all future plans on them, can be disastrous for students. We would be misleading our pupils, and letting them down, if we suggested that everyone can be a doctor if they just try hard enough. I’ve been accused of being heartless and giving up on children when I’ve made the point to students struggling to get a B at GCSE science – despite steady effort and lack of trauma, not a single bad result the day after their goldfish died – that maybe medicine or veterinary science isn’t for them. Of course, I’ve always followed this up with guiding them towards other careers that might offer some of the same rewards. But it’s difficult when they’ve come from a careers adviser with a lot more optimism – misplaced and uninformed – than understanding.

    It all comes down to whether optimism is realistic or not – and I don’t think that’s always easy to measure. Previous discussion I’ve seen of grade predictions/targets, where kids understand the likely range of outcome based on previous performance, is perhaps a better way to think about it than “I will do x which will lead to the best possible outcome y”

Constructive feedback is always appreciated

%d bloggers like this: