Why AfL might be wrong, and what to do about it

thinking-conf-bias

Some cows are so sacred that any criticism of them is fraught with the risk of bumping up against entrenched cognitive bias. We are fantastically bad at recognizing that our beliefs are often not based on evidence but on self-interest, and it’s been in everyone’s interest to uphold the belief that AfL is the best thing that teachers can do.

When confronted with ‘others’ who disagree with our most fervently held beliefs, we tend to make the following series of assumptions:

  1. They are ignorant
  2. They are stupid
  3. They are evil

When in the past I have been critical of AfL (or anything else) the most common responses is that I don’t understand it. When I present the incontestable evidence that I do understand it, opponents often treat me as if I’m a bit silly: only an idiot could believe anything so ludicrous and patently untrue. When they accept that my counter-arguments are sufficiently cogent that I prove at least a modicum of intelligence, there are only two remaining propositions: either I am evil, or you are wrong. Of these, it is far easier, and massively less damaging to our sense of self to assume that I must in fact be a bad un seeking to poison children’s life chances with my toxic spewing of such dark falsehoods.

So, can we just assume that we’ve already gone through all that unpleasantness and just engage in the argument I’m presenting?

Of course it’s important not to throw out the baby along with all this dirty bath water. I’m happy to accept that it’s a good thing for teachers to know where pupils currently are, where they need to get to and what they need to do to get there. That bit’s fine. But what of the rest?

images-1

The ‘big idea’ of AfL, as defined by its progenitor, Dylan Wiliam is that we should, “Use evidence about learning to adapt teaching and learning to meet student needs.” This is then broken down into ‘5 key strategies’ discussed by Wiliam in his comprehensive book, Embedded Formative Assessment:

  1. Clarifying, sharing, and understanding learning intentions and success criteria
  2. Eliciting evidence of learner’s achievement
  3. Providing feedback that moves learning forward
  4. Activating students as instructional resources for one another
  5. Activating students as owners of their own learning

Some of these strategies contain real merit if undertaken thoughtfully, but there’s a very real danger that the ‘big idea’ of AfL might be fundamentally, and fatally, flawed. AfL is predicated on the assumption that you can assess what pupils have learned in an individual lesson, and then adjust future teaching based on this assumption.

If nothing has changed in long-term memory, nothing has been learned.
Kirschner, Sweller & Clark

But you can’t. There’s no meaningful way to assess what pupils have learned during the lesson in which they are supposed to learning it. There’s an impressive body of research that tells us that learning is distinct from performance. You cannot see learning; you can only see performance. If we measure performance then we may be able to infer what might have been learned, but such inferences are highly problematic for two reasons: 1) performance in the classroom is highly dependent on the cues and stimuli provided by the teacher and 2) performance is a very poor indicator of how well pupils might retain or be able to transfer knowledge or skills.

So let’s consider the merits of each of the key AfL strategies:

Clarifying, sharing, and understanding learning intentions and success criteria

I’m happy with the basic premise of letting kids know where they’re supposed to be going. Work put into presenting intentions clearly and helping pupils to understand what they are supposed to be learning should always be productive. Too often learning intentions are mechanistic and may actually obfuscate the learning. Time spent on the mechanics of how to share these intentions is fleetingly important: we must simply take care that they do not become ‘wall paper’, but we must take care not to resort to distracting gimmickry.

But I do have a bone to pick with ‘success criteria’. Wiliam recommends sharing rubrics with pupils so that they will know whether they have successfully achieved the learning intention. He explores the merits of ‘student-friendly’ mark schemes concluding,

Student-friendly language can be useful as students are introduced to a discipline, but it is also important to help students develop the habits of mind that define the discipline. and coming to terms with the ‘official’ language is part of that process. P 65

Well, now. My view is that many rubrics are inherently meaningless and lead to some very lazy thinking in teachers and rarely provide anything meaningful to students. In subjects like English this is especially pronounced: what’s the difference between ‘confident’ and ‘sophisticated’ really? Any difference is arbitrary and exam boards are forced to provide exemplar work for teachers to understand the process by which they arrive at their decisions. We are forced into concluding that confident means x whilst sophisticated means y. Instead of wasting their time with vague, unhelpful success criteria, why not spend time deconstructing exemplars and modelling the expert processes we would use to complete a task?

Eliciting evidence of learners’ achievement

Basically, this is all about how we use questions. We make decisions about how and what to teach based on what we know about pupils’ understanding. Wiliam says it is “unprofessional [to move on] without finding out what the students in the class know”, and that questions are “crucially important if we are to improve the quality of students’ learning.” (p77) He points to a study by Ted Wragg which revealed that less than 10% of the questions asked by teachers “actually caused any new learning.” (p 79) But how on earth would we know? All we can infer is that we don’t think they caused any learning, and that’s not the same thing at all.

Wiliam argues that are only two valid reasons for questioning: “to cause thinking and to provide information for the teacher about what to do next.” I’m happy with the first of these reasons and tend to agree with Robert Coe that “Learning happens when people have to think hard.” But I’m not quite so sold on the second reason.

If we ask questions, no matter how well designed, and then use the responses we elicit to decide whether we should ‘move on’ we are in serious danger of missing the point that “as learning occurs, so does forgetting” and that “learning takes time and is not encapsulated in the visible here-and-now of classroom activities.” (Nuthall 2005) If pupils don’t know the answer to our questions then we can remediate. If misconceptions are revealed then we have an opportunity to explore them. But if they answer our questions correctly, it means very little. Just because they know it now, doesn’t mean they’ll know it next lesson or in an exam. A correct answer to a question is, perhaps, the least useful student response we can hope for.

But questions might also be important for stimulating discussion. Wiliam tells us that, “Engaging is classroom discussion really does make you smarter.” (p. 81) If he’s right then it’s certainly incumbent on us to ensure all pupils participate in these discussions. But creating the conditions for effective classroom discussions has very little to do with finding out what pupils know and everything to do with provoking thought. Far from hoping for neat answers to Exit Tickets, maybe the ideal circumstances in which to finish a lesson is for pupils to be actively struggling with difficult concepts. Nuthall tells us in The Hidden Lives of Learners that pupils are unlikely to transfer concepts from working to long term memory until they have encountered them on at least 3 occasions. So, who cares what they know at the end of lesson? Let’s assume that they are highly likely to forget it.

I’ve written in more detail about questioning here.

Providing feedback that moves learning forward

Who could argue against providing feedback that moves learning forward? No one, right? Nobody disputes the fact that feedback is powerful, but perhaps we should be using it a little more sparingly then we have been? For a more detailed discussion, have a read of this post. Wiliam says the following in his conclusions on feedback:

We have seen that in almost two out of every five carefully designed scientific studies, information given to people about their performance lowered subsequent performance…If we are to harness the power of feedback to increase student learning, then we need to ensure that feedback causes a cognitive rather than an emotional reaction – in other words, feedback should cause thinking. (p 131-2)

Activating students as instructional resources for one another

Wiliam argues that the whole purpose of feedback is to “increase the extent to which students are owners of their own learning,” but what does this mean? He argues that ‘cooperative learning’ is “one of the greatest success stories of educational research.” (p 133) But I’m not so sure. He advocates the following as practical techniques for encouraging cooperative learning:

  • C3B4Me
  • Peer evaluation of homework
  • Homework help boards
  • Two stars and a wish
  • Student Reporters
  • If you’ve learned it, help someone who hasn’t

Each of these are about putting the child at the centre of the classroom and moving the teacher to the side. These kinds of techniques have gained a great deal of traction in classrooms. But often at the cost of efficiency. There is always an opportunity cost: If I ask pupils to spend a lesson working things out in groups, that is a lesson I cannot spend teaching them. This has led to widespread condemnation of ‘teacher talk’ and has, I would argue, been instrumental in lowering standards and expectations. If we force children to work cooperatively all the time they will never know enough to produce thinking or work of any quality. Cooperative learning (or ‘independent learning’ as it’s often called) certainly has a place, but it should come after careful explanation and modelling:

screen-shot-2013-07-18-at-10-48-25

 

Read more about this teaching sequence here.

Maybe peer assessment might be a useful strategy for improving pupils’ attainment, particularly in light of Nuthall’s discovery that 80% of the feedback pupils get on their work is from each other, and 80% of that is wrong. If that’s true (and I think it might well be) we need to do something about it. One possible solution is discussed in this post.

Activating students as owners of their own learning

No one can teach you anything, they can only help you learn. And yet, as Wiliam points out:

[O]ur classrooms seem to be based on the opposite principle – that if they try really hard, teachers can do the learning for the learners. This is only exacerbated by accountability regimes that mandate sanctions for teachers, schools, and for districts, but not for students. P 145

Wiliam is convinced that pupils can develop sufficient insight into their own learning to improve it, and I’m inclined to agree. My advice, for what it’s worth is develop pupils’ error checking skills. I discussed in this post how proofreading might be used to force pupils to engage meta-cognitively with their work, and to provide opportunity for the feedback teachers give to be targeted at the point at which pupils are ready to learn. 

More on metacognition here.

I’m not sure whether using traffic lights has much if any impact on developing metacognition, but I can’t see that it does much harm, so if you’re keen, by all means carry on.

In conclusion

Many of the strategies discussed by Wiliam and enshrined in AfL have worth, and if considered carefully enough will have a positive impact on pupils’ progress. I’m a big fan of Dylan Wiliam and have been hugely influenced by his body of work. Just have a look at the Dylan Wiliam tag on my blog! But I’m afraid I have to conclude that the ‘big idea’ of AfL is wrong. Of course, I may well be wrong: I certainly don’t know anywhere near as much as many other people. But I do know that there are hugely powerful cognitive biases at work that will protect us from admitting that our belief systems might be wrong. Staying open to the possibility of error is possibly the hardest but most valuable of human endeavours.

Related Posts

Everything we’ve been told about teaching is wrong, and what to do about it!
Chasing our tails – is AfL all it’s cracked up to be?
AfL: cargo cult teaching
Questions that matter: method vs practice?

51 Responses to Why AfL might be wrong, and what to do about it

  1. johnpearce1 says:

    Dropping into your blog always makes me think! I love your defusers at the start (being called ignorant – stupid – evil) to which I’d add another… “To be honest I haven’t the time to think today and so I’m going to read a bit, bin it and move on, without further thought”. I started to think like that – as I respond from a motorway service station before driving on fast) and realised this is why, as you say, we can never assume learning takes place “in the moment”.

    My obsession at the moment is challenging those who feel they have to, ought to, or (worse) are best when making judgements about the work of others …Because I believe that true learning only makes sense when the learner knows it, can recognise it and move on, forward under their eon steam. So, that is why I default (err) to starting with the learner’s perceptions, ideas… So I want, expect teachers to comment on their teaching (prior to the assumed superior judge doing so) and I’d apply this approach to students… I reckon the real dialogue of learning – the most powerful feedback is of ourselves, by ourselves.. So learning records, diaries, blogs, papers (monologues?) are often really powerful statements of “where we are in our own learning”. We think aloud in this way – we are testing out our thoughts and ideas.

    The problem, in the potential intersections on social media, and maybe the interactions in a classroom, are that there are SO MANY, to which we feel a need, want to respond… So we don’t even bother to try (look at the number of us commenting lurkers of this page) but what spark of new thinking have you unleashed? I know I will not be able to trot out A4L for a while without a little reminder of this David… Thanks… Now about my idea…..

    • David Didau says:

      Thanks John – I’m not even sure that I’m right about AfL being wrong – it rather hinges on whether I’m right about learning being invisible. I’m really hoping that folk will take me on over this and force me to sharpen my thinking.

      But if something can’t stand up to scrutiny then it’s definitely wrong. Questioning assumptions is always worthwhile.

  2. johnpearce1 says:

    In my haste – I missed a crucial point – that a powerful enhancement of and to A4L which picks up your advice and my obsession, would be to – encourage students to describe their learning to themselves in diary, log and discussion ie self- assessment and then undertake analysis of the next steps and the action they need to take to get there… As per Kolb and Boyatsus and, dare I say iAbacus models…

  3. Cazzwebbo says:

    Re proof reading… Currently working in my department with other tutors to adopt the process across other subject areas, not just English. As a result of my comment on your dyslexia post the other week, and Jules Daulbys reply, I’ve found out that we already have ‘balabolka’ installed on ‘mystudybar’ for students but it is underused and tutors don’t have it so it’s probably hit and miss as to whether students really ever touch it. So, our ALS tutor is now going to roll out a series of proof reading classes after Easter, to try to establish the process of self proof reading, peer proof reading and how to use balabolka to help with that too. She’s going to push the learning by mistakes angle and we are talking to subject tutors about doing this too. I’m also going to repeat this often in my classes to reinforce the process, which at the moment isn’t really there. We are going to try to normalise this process for staff and students as a regular way of working. Staff are up for it.

  4. […] Some cows are so sacred that any criticism of them is fraught with the risk of bumping up against entrenched cognitive bias.  […]

  5. A fascinating post which has made me think – much appreciated!

    I’d concur with many of the issues you raise about individual techniques for the strategies… for example, I’ve always believed that sharing learning intentions is critical – if done well, which is pretty rare. Likewise, some tasks designed to ‘activate students as resources for each other’ are a drag on time – so I try to use only those which seem to me to save teaching time (like asking your neighbour simple questions before asking me).

    My big question, however, relates to your central contention. The argument that performance is separate from, and often unrelated to, learning, appears irrefutable; likewise, the case for repetition and the amount of time it takes for something to enter long-term memory. However, the implicit conclusion of this line of reasoning (correct me if I’m wrong) is that the teacher can never know how much students have learned. I’d agree with this in one sense – I don’t believe I have anything much more than a basic illusion as to students’ ‘progress’ (least of all when I try to report this in convenient data form). However, if we don’t assess some proxy for student learning (like understanding at some point in a lesson), what data do we have to inform our planning and feedback? If the answer is that it is acceptable to gain some snapshot impression of student learning at some point, then do not most AfL techniques stand?

    If so, the issue is with the fallacious assumption that competency in one lesson confers retention and ongoing mastery… (AfL is only a culprit inasmuch as it contributes to this – it is not the problem itself).

    If not, I see (although I may be wrong) a void in which teachers can never do anything save plan and proceed with schemes of work using their best understanding of cognitive science and what mastery of their domain represents.

    But I bow to your superior wisdom on this…!

    • David Didau says:

      Ha! Not sure I’m superior in any respect, but least of all in wisdom. What I was trying (failing?) to point out in the post was that some proxies are better than others. Distance between instruction & assessment will (I contend) lead to a much clearer indication of whether pupils have learned. But, if we accept the lessons of the Willingham thesis of memory then we can presuppose that certain strategies are much more likely to lead to learning than others.

      I had the beginnings of an interesting discussion with Rob Coe last night about what might be ‘better proxies’ for learning. His contention that ‘thinking hard’ leads to learning was an attempt to square this circle. But I think there might be others: this post by Peter Blenkinsop is an interesting beginning: http://manyana-education.blogspot.co.uk/2014/03/how-might-we-start-to-see-learning-in.html

      Cheers, DD

  6. Philip Crooks says:

    Just a light hearted comment. The real AFL starts tomorrow evening, when Collingwood play The Fremantle Dockers in Melbourne .
    I really enjoy your posts, and your new book is on order.

  7. Nati López says:

    A lot to consider when aiming to implement AfL ‘properly ‘. I actually completed my PGCE master assigment on this… I don ‘t think many teachers fully understand what AfL is about or use it appropriately. As someone said: AfL in schools is so distorted that we might as well forget about it all together. However, same as levels:’ it’s all we know’.
    Thanks for this post. I’ve learnt so much from reading your post lately that I hope no one thinks you’re evil!

  8. […] David Didau (a.k.a. Learning Spy) observes, AfL is a particularly sacred cow to slaughter. That may be so, but I have had my […]

  9. […] Some cows are so sacred that any criticism of them is fraught with the risk of bumping up against entrenched cognitive bias. We are fantastically bad at recognizing that our beliefs are often not based on evidence but on self-interest, and it’s been in everyone’s interest to uphold the belief that AfL is the best  […]

  10. Simon Porter says:

    I have always been uncomfortable with routinely sharing my learning intentions at the start of the class. My feeling is that if the students understand them, then there is no point teaching the lesson, and if they don’t, then why share them? I also often want what they learn to be a surprise!

  11. AfL is also predicated on the notion of ‘child-centred learning’, which II believe is also flawed.

    • David Didau says:

      Would you like to expand on that Steve?

      It’s striking that the ‘peer assessment’ bit of AfL seems to be based on the fact that cooperative learning is “one of the greatest success stories in the history of education research” according to Slavin, Hurley & Chamberlain (2003).

  12. Concerned says:

    David, thought provoking as always. I’m a big believer in the uselessness of ‘learning objectives’ – just because I tell you to learn something or I say you should learn something doesn’t mean that you will learn it. I’m a huge believer in every question I ask, every task I set, has some idea in mind that I want the students to think deeply about. One example is that I might show the steps of finding the gradient with the ‘mute’ button on, I ask the students to write a set of steps individually about how to find the gradient – this ensures what they think about is how would you explain how to find the gradient. Now I don’t need to say ‘you should be able to find the gradient’ it is clear from what I’ve asked them to do. (Hope I’ve explained that well)

    I also think that you are mistaking ‘co-operative learning’ with ‘group work’.

    • David Didau says:

      Regarding cooperative learning – I was critiquing Wiliam’s recommendations for ‘activating students as learning resources for each other.

      I don’t think deconstruction & modelling are at odds with Bjork – they acknowledge the the brain architecture of experts and novices is different. Bjork would argue that instruction needs to be spaced, interleaved and that ‘practice’ should make more use of the testing effect, no?

  13. Concerned says:

    ‘Instead of wasting their time with vague, unhelpful success criteria, why not spend time deconstructing exemplars and modelling the expert processes we would use to complete a task?’

    In terms of ‘deconstructing exemplars and modelling the expert processes’ – I agree this is needed but surely it is at odds with Bjork’s findings that ‘desirable difficulties’ increase learning?

    Sorry for the double post.

  14. Louise Neve says:

    As Assessment Manager at my school, this was very thought-provoking – thanks.

    I totally agree that any progress measured in a lesson is NOT a measure of actual learning (committing a skill or piece of knowledge to long term memory). Thankfully, Ofsted have also recognised this and now look for “progress over time”.

    However, this does not mean that useful information cannot be gathered by the teacher (or by the pupil’s themselves, if trained) during, or after (via marking), a lesson. The most valuable information I gain is not who “can” do something or who appears to have committed information to their short term memories, but who cannot – those needing another go, or who have picked up misconceptions, or for whom the learning intention is nowhere near being hit! I then either tackle these things there and then, or plan to address it next lesson. This, for me, is true AfL. AfL is NOT summative and was never intended by Wiliam to be – it does not tell you what has been truly “learned” – that is called Assessment OF Learning.

    Without the ongoing lesson-by-lesson assessment I describe above, you cannot give effective feedback which focuses on next steps. And Hattie’s “Visible Learning” meta analysis shows that good feedback is the single most effective “intervention” to increase progress.

    Success criteria? It depends. When teaching a skill, we tend to use “process criteria” – step by step instructions to achieve a goal. If using these, the only thing you can assess is whether the children can and have followed them! Once you think they have committed them to memory, you can then switch to “outcome criteria”: and this is where comparing models and examples to ascertain what features make a “good one” may be useful in generating this list. “You will know you have succeeded if you have……” Finally comes the summative assessment OF the learning: a test, assessment piece or other application task that comes some weeks later and whose intention and success criteria is simply to recall and apply what they learned. Only then will you know if true learning has taken place.

    Perhaps Assessment for LEARNING is in fact misnamed? It is really Assessment for TEACHING: What do I, as a teacher, need to do in the next lesson/ unit?

    • David Didau says:

      Hi Louise – you appear to have fallen into the ‘you must be ignorant’ camp. I can assure you that I’m well aware of pretty much everything that AfL is and claims to be. I’m also clear on Hattie’s claims about feedback. You might find it interesting to read this post which challenges much of what we’ve been told about feedback:http://www.learningspy.co.uk/featured/reducing-feedback-might-increase-learning/

      The problem with your approach is that it wastes a lot of time. Making decisions of what to teach next based on what they can do now, is a bit silly. Finding out who can’t do something in a lesson is irrelevant: assume that NO ONE will learn unless they have encountered new concepts at least 3 times in at lest 3 different ways. And then make use of the Testing Effect rather then wasting time restudying.

      Thanks, David

  15. Louise Neve says:

    Slightly puzzled as to the “you must be ignorant” comment – have you been on the receiving end of a lot of criticism to make you so sensitive?! I was no more implying that you are ignorant, than you are implying that I am, from the evidence you cite in your blog (though I probably am). The evidence I quoted was simply to support my own argument. I obviously didn’t express it clearly, as I largely agree with you.

    I am flummoxed by the 3 times thing though: does this mean NO ONE can learn in less than 3 exposures? What if I am on the “third” exposure? Do I assume they’ve all got it? Or wait to test later? And what if they haven’t got it when I test/assess after 3 exposures? And does this mean I should have rigid unit plans where I teach the lesson, and move on regardless to the next lesson, planning in 3 opportunities to deliver that concept or skill, and then testing at a suitable interval to see if it has stuck? (Therefore no adapting planning from day to day, based on my ongoing assessment?) Sorry – I am probably being dim but I don’t understand (and in case you feel “got at” again, I think this is a deficiency on my part, not yours!)

    • David Didau says:

      Apologies Louise – yes, I’m used to people saying that if I disagree it’s because I don’t understand their position fully.

      The 3x thing comes from Graham Nuthall’s research and suggests that we are unlikely to transfer new concepts from working to long term memory unless they have been encountered on several occasions. My understanding of how we learn suggests that ‘rigid’ unit plans, moving on ‘regardless’ and avoiding adaptive planning (at least in the short term) are all highly desirable. It may well be that well-intentioned practices like formative assessment lead us to make incorrect assumptions about our students ‘progress’ with the result that we neglect what will most benefit them in the longer term. The testing effect has an awful lot of psychological research supporting it as a phenomenon for increasing long term retention and transfer of new knowledge & skills – as such it is hugely underused in classrooms.

      I guess, this requires each of us to examine our own teaching to see where we might best improve. It strikes me that few teachers test students at the outset to see what they already know, and then retest summatively to see how much has been learned, but unless we do this, how will will we know if our teaching is effective?

      • Louise Neve says:

        Thanks for that, I think I am a bit clearer now! Interestingly, I do an initial test at the start of a unit (which I have to say I DO use to inform my planning); then an end of unit one, and then a “site of application” one several weeks later….but only in maths (I am a junior school teacher). I am now wondering if I ought to be doing more of that with, for example, reading and writing units, where I tend to only “test” post-unit, using “in-class” AfL techniques to ascertain starting points at the start of the unit, instead. The Nuthall research sounds interesting – will seek out further reading on that.

        Returning to the subject of feedback, having read Hattie’s work I completely re-wrote our marking and feedback policy to accommodate the levels of feedback and other recommendations of what he found to be most effective practice. The hardest part was separating feedback from praise! For most of us, it was an entrenched habit to comment on effort or attainment in a very general way, both orally and in writing (eg “Well done” “Good work” “Good try” etc) even if this was followed by more specific feedback. We are learning not to do it (because as you know, Hattie suggests that it negates the effect of the specific feedback) but it is a hard habit to break! We changed our marking policy to avoid grading work several years ago, and we also made it against policy to award housepoints for,or on, work. The children certainly engage more with the feedback now that it is not accompanied by grades, points or sweeping praise (or criticism) and are much keener and more willing to improve their work. The policy is also that we do not mark every piece of work with detailed feedback – limiting it to roughly once a week/ once a unit, on average. In between the children have opportunities to self- or peer- assess and make improvements as they learn new things.

        However, as with all aspects of assessment, our practice is a work in progress and I am always seeking ways to improve outcomes for the children, which SHOULD be the ultimate aim of any assessment. (I say SHOULD be, because of course, we are also subject to external “validation” that looks at assessments in terms of their ability to rank schools!)

  16. Concerned says:

    David, I was wondering about the Chimp Paradox and learning. I’ve been thinking a little more deeply and I’ve seen enough episodes in my lessons where a student in the lesson appears to not grasp an idea correctly however the next lesson when we have a quick test upon the recent material (using the testing effect) they seem to have a much firmer understanding. Is this because anything we do in lesson, the chimp is in charge and over time it is the computer that takes over? (hoping you read the book I’m speaking about).

  17. Hi David,

    Like you, I value Dylan Wiliam’s work but have misgivings about always making learning objectives explicit and upfront.

    This is partly because I am a fan of unexpected and open-ended learning, but also because I suspect that deep learning requires the student NOT to know where they are heading the whole time.

    Mary Beard commented recently on how new undergrads at Cambridge suffer when they realize that learning there is NOT about levels and targets and that they are expected to enter a subject and flounder for a while before meaning begins to emerge. I was educated at Cambridge and I remember the panic well! The feelings of being overwhelmed can be intense – and I would argue that those are sometimes appropriate emotions in learning and can indicate that the student is learning at a significant level.

    Certainly, when I began my PhD, I felt those feelings of being overwhelmed and not knowing where I was going all over again. If we are preparing students to learn – for the rest of their lives and not just pass exams – then we need to familiarize them with ‘not knowing’ where they are going for a period of time.

    I also remember Miss Gledhill. Miss Gledhill taught me when I was 11. And I spent a really happy year with her and look back on her as one of my best teachers. And while I value ‘evidence’ in education I also value ‘experience’ and even ‘tradition’. And she said once that one of HER teachers told her that you can know what you intend a child to learn in a lesson but not always what they DO learn. They may learn more from a chance remark you make, an anecdote you tell, or how you treat them than they learn about the use of the semi-colon. That stayed with me when I entered teaching and left me with a desire to travel lightly to my learning objectives. I have them, but always want to admit that a child may also learn something I did not intend. John Dewey (Education and Experience) has lovely phrase for this. He calls it ‘collateral learning’.

    I always enjoy your posts. Many thanks

    Jenny

    • Jason says:

      Hi Jenny, I might be really picky here but from reading a lot of his stuff I haven’t seem Dylan Wiliam advocate making learning objectives explicit and upfront all of the time. There are sometimes when sharing Learning Objectives will spoil the surprise of what they will learn. Just a thought.

  18. […] Twitter exchange. David has written a blog post suggesting that a fundamental point about AfL may be flawed – he’s suggesting that learning is invisible, only performance can be seen and that […]

  19. […] Which implies that a core feature of AfL, peer-assessment, may not be especially beneficial when students lack expertise in a subject (I’m glad to see this slowly leading to a debate about the value of AfL). […]

  20. john says:

    Certainly some great points to reflect on. Be really interesting if Dylan had any thoughts to share having read it himself?

  21. Dylan Wiliam says:

    In his post on “Why AfL might be wrong, and what to do about it” David Didau points out (correctly) that it is impossible to assess what students have learned in an individual lesson. As John Mason once said, “teaching takes place in time, but learning takes place over time” (Griffin, 1989). The ultimate test of any teaching is long-term changes in what students can do (or avoid doing, such as getting pregnant or taking drugs). The problem with such an approach to teaching is that if we wait until we see the long-term evidence, it will be too late. An analogy with automobile manufacturing may be helpful here.

    In the 1970s and 1980s, the major American and European car-makers had smart people designing production processes through which vehicles progressed, and then, and the end of the process, the finished product would be inspected to see if it worked properly. The Japanese, building on the work of W. Edwards Deming (an American) realized that it would be far better to build quality into the manufacturing process. If something was wrong with a car, any worker on the production line had the authority to stop the production line to make sure that the problem was fixed before the vehicle moved along the production process. This approach is often described as “building quality in” to the production process, rather than “inspecting quality in” at the final stage. Another way of describing the difference is a move from quality control to quality assurance.

    Similarly, in teaching, while we are always interested in the long-run outcomes, the question is whether attending to some of the shorter term outcomes can help us improve the learning for young people.

    This is an extraordinarily complex task, because we are trying to construct models of what is happening in a students’ mind when this is not directly observable. Ernst von Glasersfeld described the problem thus:

    “Inevitably, that model will be constructed, not out of the child’s conceptual elements, but out of the conceptual elements that are the interviewer’s own. It is in this context that the epistemological principle of fit, rather than match is of crucial importance. Just as cognitive organisms can never compare their conceptual organisations of experience with the structure of an independent objective reality, so the interviewer, experimenter, or teacher can never compare the model he or she has constructed of a child’s conceptualisations with what actually goes on in the child’s head. In the one case as in the other, the best that can be achieved is a model that remains viable within the range of available experience.” (von Glasersfeld, 1987 p. 13)

    So I agree with David that we can never be sure that the conclusions we draw about what our students have learned are the right conclusions. This is why my definition of formative assessment does not require that the inferences we make from the evidence of student achievement actually improve student learning—learning is too complex and messy for this ever to be certain. What we can do is increase the odds that we are making the right decision on the basis of evidence rather than hunch.

    In terms of the five strategies, I was surprised that David’s post specifically focused on critiquing success criteria. In fact, I actually went back to read what I had written on this in 2011 to check what I had said. Much of the chapter 3 of my book Embedded formative assessment (where I discuss learning intentions) is spent railing against success criteria, and arguing that a shared construct of quality (what Guy Claxton calls a “nose for quality”) is what we should be aiming for, although on those rare occasions when we can spell out the rules for success, we should, of course do so. Michael Polanyi’s work on “Personal knowledge” now over 50 years old, is still the definitive work on this, in my opinion.

    In terms of eliciting evidence (which is definitely not just questioning by the way, as I go to considerable lengths to show), then of course we never really know what is happening in the student’s head but I am confident that teaching will be better if the teacher bases their decisions about what to do next on a reasonably accurate model of the students’ thinking. There will also, I suspect, be strong differences across disciplines here. Asking a well-framed question in science may reveal that a student has what Jim Minstrell calls a facet of thinking (DeBarger et al., 2009) that is different from the intended learning—for example, a student may think that a piece of metal left outside on a winter’s night is colder than the wooden table on which it rests, when in fact the temperature of the two are the same (the metal feels colder because it conducts heat away from the hand faster). You may not get rid of the facet of thinking quickly, but knowing that the student thinks this has to be better than not knowing it.

    As for feedback, there really is a lot of rot talked about how feedback should and should not be given. People say that feedback should be descriptive, and maybe a lot of the time it should be, but people forget that the only good feedback is that which is acted upon, which is why the only thing that matters is the relationship between the teacher and the student. Every teacher knows that the same feedback given to one student will improve that student’s learning but to another student, of similar achievement, will make that student give up. Teachers need to know their students, so that they know when to push, and they know when to back off. There will be times when it is perfectly appropriate to say to a student that this work really isn’t worth marking because they have “phoned it in” and other times then this would be completely inappropriate. Just as importantly, students need to trust their teachers. If they don’t think the teacher knows what he or she is talking about, or doesn’t have the student’s best interests at heart, the student is unlikely to invest the effort required to take the feedback on board. That is why almost all of the research on feedback is a waste of time—hardly any studies look at the responses cued in the individual recipient by the feedback.

    The quote about collaborative/co-operative learning being one of the success stories of educational research comes from Robert Slavin, who has probably done more high-quality work in this area than anyone (Slavin et al., 2003). The problem is that few teachers ensure that the two criteria for collaborative learning are in place: group goals (so that students are working as a group rather than just in a group) and individual accountability (so that any student falling down on the job harms the entire group’s work). And if a teacher chooses to use such techniques, the teacher is still responsible for the quality of teaching provide by peers. As David notes, too often, peer tutoring is just confident students providing poor quality advice to their less confident peers.

    Finally, in terms of self-assessment, it is, of course, tragic that in many schools, self-assessment consists entirely of students making judgments on their own confidence that they have learned the intended material. We have over 50 years of research on self-reports that show they cannot be trusted. But there is a huge amount of well-grounded research that shows that helping students improve their self-assessment skills increases achievement. David specifically mentions error-checking, which is obviously important, and my thinking here has been greatly advanced by working (in Scotland) with instrumental music teachers. Most teachers of academic subjects seem to believe that most of the progress made by their students is made when the teacher is present. Instrumental music teachers know this can’t work. The amount of progress a child can make on the violin during a 20 or 30 minute lesson is very small. The real progress comes through practice, and what I have been impressed to see is how much time and care instrumental music teachers take to ensure that their pupils can practice effectively.

    So in conclusion, David has certainly provided an effective critique of “assessment for learning” as enacted in government policy, and in many schools, but I don’t see anything here that forces me to reconsider how I think about what I call formative assessment. I remain convinced that as long as teachers reflect on the activities in which they have engaged their students, and what their students have learned as a result, then good things will happen.

    References

    Claxton, G. L. (1995). What kind of learning does self-assessment drive? Developing a ‘nose’ for quality: comments on Klenowski. Assessment in Education: principles, policy and practice, 2(3), 339-343.

    DeBarger, A. H., Ayala, C. C., Minstrell, J., Kraus, P., & Stanford, T. (2009). Facet-based progressions of student understanding in chemistry. Menlo Park, CA: SRI International.
    Griffin, P. (1989). Teaching takes place in time, learning takes place over time. Mathematics Teaching, 12-13.

    Polanyi, M. (1958). Personal knowledge. London, UK: Routledge & Kegan Paul.

    Slavin, R. E., Hurley, E. A., & Chamberlain, A. M. (2003). Cooperative learning and achievement. In W. M. Reynolds & G. J. Miller (Eds.), Handbook of psychology volume 7: Educational psychology (pp. 177-198). Hoboken, NJ: Wiley.

    von Glasersfeld, E. (1987). Learning as a constructive activity. In C. Janvier (Ed.), Problems of representation in the teaching and learning of mathematics (pp. 3-17). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Wiliam, D. (2011). Embedded formative assessment. Bloomington, IN: Solution Tree.

    • Martin Harris says:

      Sun Headline: Finding out who can’t do something in a lesson is irrelevant:
      Chris Woodhead responds:
      Testing testing testing.

      Just getting it out of the way so that this excellent discussion continues without nonesense.

  22. […] in March I wrote a post called Why AfL might be wrong, and what to do about it based, largely, on Dylan Wiliam’s book Embedded Formative Assessment (If you haven’t […]

  23. I apologise if this point has already ben made.

    I find that the best type of AfL used in my classroom (NQT, so limited experience) is AfL that allows the learner to ‘understand’ and ‘know’ that they have learned something. If it is purely an observation made by the teacher then are we leaving the most important person out of the equation?

  24. […] struck me recently in the debate between David Didau and Dylan WIlliams was the latters view that actually the two were discussing something other, or […]

  25. […] you’re not already aware of my critique and Dylan Wiliam’s defence of formative assessment I do recommending getting up to speed […]

  26. […] you’re not already aware of my critique and Dylan Wiliam’s defence of formative assessment I do recommending getting up to speed […]

  27. […] you’re not already aware of my critique and Dylan Wiliam’s defence of formative assessment I do recommending getting up to speed […]

  28. […] those who don’t know, I wrote a post earlier in the year in which I set out my ideas on why AfL might be wrong. Generously, Dylan took the time to set out a defence and we subsequently agreed to thrash out our […]

  29. […] it out) is one of the most widely-followed educational blogs in the world.  In March, he wrote this blog, proposing that Assessment for Learning might be wrong.  Dylan Wiliam duly responded, and I […]

  30. […] those who don’t know, I wrote a post earlier in the year in which I set out my ideas on why AfL might be wrong. Generously, Dylan took the time to set out a defence and we subsequently agreed to thrash out our […]

  31. […] Why AfL might be wrong, and what to do about it. Some cows are so sacred that any criticism of them is fraught with the risk of bumping up against entrenched cognitive bias. […]

  32. […] Why AfL might be wrong, and what to do about it 12th March 2014 – 6,732 […]

  33. […] feedback: is less more? Why AfL might be wrong Getting feedback […]

  34. […] it is for teachers to change their planning in light of pupils’ performance, especially if AfL is wrong. It’s also worth reminding ourselves that just because feedback tops the charts on what […]

Constructive feedback is always appreciated

%d bloggers like this: