Is there a case for summative assessment?

I’ve written a lot on the importance of formative assessment recently and feel pretty clear in my own mind of its efficacy. In contrast I see summative assessment as existing only as an external measure of success or failure. I know it exists, and I know it’s fairly important to my students’ life chances. It’s also one of the primary means by which my professional practice is judged, so I’d better take notice of it. This seems like a necessary evil, not something to be celebrated.

Cristina Milos and Jennifer Borgioli have challenged this view and asserted that actually, summative assessment has its place in the classroom and fulfils a useful purpose.  Cristina says that summative assessment “need not be constructed in terms of pass/fail” and Jennifer suggests that “quality assessment is about balance” and using the right tool and the right time. She also says that the problem with using formative assessment to evaluate whether learning has occurred is that it is “unethical to evaluate a learner while they are still learning”.

Now, I consider myself to be an open minded chap, so it seemed reasonable to check out what’s been written on the subject to see if there is merit in their views.

Firstly, let’s define our terms. Summative Assessment I mean the formal testing of what has been learned in order to produce marks or grades which may be used for reports of various types. This is different from Formative Assessment, in which the emphasis is on on-going assessments of different types used to judge how best to help pupils learn further. Have I got this right? I’d be very grateful if anyone would like to (politely) set me straight.

Research suggests that while summative assessment can be motivating for some students, it has a negative effect on others:

  • After summative assessment, low-achieving pupils had lower self-esteem than higher-achievers, whereas there had been no correlation between self-esteem and achievement before
  • Repeated practice tests reinforce low self-esteem of low achievers
  • When results of summative assessment are presented as primarily relating to individual pupils the negative effect on low-achievers is more pronounced than when the results are for evaluation of school or authority standards.
  • Secondary age low-achievers may deliberately underperform in summative assessments because they are failing anyway
  • Summative assessments can be limiting for the most able
  • “Big bang” tests cause anxiety in pupils, especially girls and widen the gap between high and low achievers’ motivation
  • Summative assessment promotes “extrinsic” motivation, in which pupils respond to the promise of some kind of reward rather than “intrinsic” motivation in which they perform because they are interested and want to do the work
Summative assessment can also produce problems for the curriculum and with teaching:
  • The curriculum can be narrowed by “teaching to the test” which can take away from curriculum content
  • It can produce distortion in terms of teaching techniques
  • Summative test questions may not be framed in the same way as those preferred for formative assessment
  • Teachers can spend a lot of time on summative assessment which does not directly improve pupils’ learning
  • Teachers sometimes adopt a more didactic “transmission” style of teaching which disadvantages those who don’t respond well to it
Now, I accept that many of these problems can be planned for so that their effects are minimised if not avoided all together. Here are some of the ways to avoid some of the pitfalls:
  • Intrinsic interest in tasks can be encouraged
  • Students’ awareness of learning goals rather than test performance goals can be developed
  • A wide range of types of understanding can be included in summative assessment
  • Some formative assessment evidence may be included in summative reports
  • Peer- and self-assessment could be included in summative records
  • Tests don’t need to be formal written assessments
  • The comparison of individual pupils on the basis of scores can be avoided
  • Summative tests can be placed before the end of a teaching block so that there is some opportunity for follow-up based on the results, and even reassessment
  • Summative judgements can be made on the basis of a variety of tests (varied both in form and content)
  • Students could carry forward lessons from assessments even into the next school session (eg in the form of a copy of their school report)
  • Feedback can be given to pupils in terms of the learning goals rather than just a test mark
  • Tests might be devised to assess separate elements of the course separately
  • In practising for summative assessments, pupils can make up and answer their own questions.  (Research has shown this to be an effective strategy)
  • Tests can be timed according to pupil readiness rather than leaving them to the end of the block of work
  • Summative assessment can be presented realistically, as being limited
  • Tests can provide evidence for evaluating courses and teaching approaches
  • Whole-school discussion of such assessment principles can be helpful
The problem is though that these measures, whilst all very worthy, only provide a way to cope with something that is essentially damaging. Look how tentative the language is: the potential harm of summative assessment can, could, might be ameliorated by conducting it in a more formative way. If that’s the case, why bother? Why not simply ensure that all assessment provides opportunities for formative feedback which enable the teacher to redirect teaching and the learner to make progress?
Maybe I’m being dense but I really struggle with the idea the summative assessment is useful for anything other than providing data on students’ (and teachers’) success and failure. I’m not being bombastic and I like to think that I don’t deal in “vague rhetoric” as Cristina accuses me. In fact I’m desperately keen to be shown the error of my ways and will respond positively and graciously to any comments. Over to you.

18 Responses to Is there a case for summative assessment?

  1. William Lau says:

    I think we can get around the challenges of Summative assessment by offering a “current grade” and a “potential grade”. Under the potential grade, you write small steps which the student needs to take in order to bring them closer to their potential grade. The latter is a form of formative assessment, by providing feedback which helps them improve.

    The key thing is (as Ken Spours would put it) a focus on “AND” instead of “VERSUS”. A balance of Summative AND Formative instead of the age old Summative VS Formative.

    Ipsative assessment is also worth throwing into the mix. Measuring someone against their past achievement to see how much they have grown. The simple formular of:
    1)what are you trying to achieve?
    2)Perform the action.
    3)Check if that action helped-are you getting closer to your goal?
    4)If not, try a different technique/improved action. If it works, carry on what you were doing.
    5)Re-check, did that new action help? If yes, carry on. If no, go back to step 3 and keep cycling…

    • learningspy says:

      Wil thanks for this. Yes, really like the ‘personal best’ ethos of ipsative assessment – one of my main concerns with grades is that they allows kids to measure themselves against each other – much better to measure against yourself.

      I’m not really in the business on Vs for it’s own sake – what I’m struggling with here is that summative exists. I get why we think it has to exist, but does it really? What would Ken’s education revolution view be?

      If I absolutely have to talk about target grades, I share students’ statistical spread of grades from which their target is derived. It’s much more motivating to see these figures that to feel condemned to only being able to get a B/C/D. I think targets fix mindsets whereas showing % likelihoods encourages growth.

  2. William Lau says:

    It’s unfortunate that we have become so reliant on grades. But realistically, it’d be near impossible/impractical to get rid of summative assessments and grades, certainly at KS4 and KS5. The main reason for this is because universities and to an extent, employers still look at grades. It’s a stepping stone to an interview and from there, obviously they’re really only assessing competencies/experiences gained.

    Interestingly, when I went to Malmo in Sweden on a task force visit, we found that students were not graded for most of KS3. They barely had exams/tests.

    They did have national tests in Year 11, but the grades did not mean anything. i.e. They were graded, but it did not determine which 6th form/college they would go on to. Their 6f/college placement was based solely on the teacher assessed grade. Students knew what grade they were working on, as their teacher told them. Many knew what was required to move up to the next grade. Headteachers, the LEA, the government just trusted teachers to be honest. Many of them were. Although a recent Guardian article found some evidence contrary to what we saw, this occasional grade inflation was particularly evident in free schools (http://www.guardian.co.uk/world/2011/sep/10/sweden-free-schools-experiment).

    I still think Sweden have got some things right though, they play the “long game”. They don’t measure students every month or even every year, they know that learning takes time. You cannot rush the kids through assessment (especially not via rote learning to show progress in summative grades). Their PISA results are just above the UK, so they’re not doing too bad. They could be doing better mind. I think, if you do not measure often, how do you know that a child is making any progress, nevermind good progress? How do you know they are meeting their potential. Too much laissez faire kind of teaching without the rigour of summative testing is not the answer (http://www.economist.com/node/21529013?fsrc=scn/tw/te/ar/thehorsebeforethecart). An over-emphasis on grades and summative grades is not the answer either.

    Can we get rid of summative assessment, probably for KS3, yes. Can we get rid of it all together, probably not. We still need a measure, students crave that measure, they want to know what grade they’re on, even in Sweden they do! It’s the only way they know how to improve.

    Closing thought. Imagine watching a season of football matches in which you did not pay any attention to the scores, just who played well and who had improved. From watching who plays well, could you ascertain who wins the league and who comes second and third? Probably not. We’re obsessed with ranking-that’s not a good thing. But we do need to assess QUALITY in all areas of life, we need these measures. Watching a game of football and not paying attention/caring about the scoreline means there would be no focus, no need to (score/meet) goals. I don’t think this would benefit the players and I don’t think this would benefit students or teachers.

  3. ceolaf says:

    I’ve said this elsewhere.

    There’s another purpose for summative assessment, one that — admittedly — has little to do with teaching and learning.

    The schools are accountable to the communities they serve, including the families they serve. As Leithwood has asked (I think it was Leithwood), what form does that account take?

    Well, how can schools report to those to whom they are accountable on the progress of their work?

    This is a valid question. That account — in some form acceptable to those accountors — is a valid purpose.

    You see, school should not serve teacher or even students. Schools serve communities, and public schools serve the public. And are accountable to the public.

    So, what is a case for summative assessment? Purely summative assessment?

    A case is that the that public (or the community serve by a private school) have a right to know what is being taught, how students are progressing and/or what they are learning. Not only does the public pay for the schools, but the schools exist to serve the public interest. As with any other public institution, the public has the right to get an account of the work the schools do.

    Further, I would suggest that parent (or guardians) of students also have a right to an account. I would further suggest that parents can far far better understand their own children’s learning and/or progress if they have a understanding of context. That includes curriculum — probably not just standards — and the progress of peers. Norm based reporting can be quite helpful for parents, in that regard. Summative assessment seems reasonable.

    Mind you, this is not a defense of our current test. This is not a defense of longstanding grading practices, be they numeric and letter. I would far prefer narratives full of analysis and anecdotes for parents than a bunch of grades. I would far prefer more classroom and school level testing (i.e. as the NAEP cannot provide scores for individual students), to allow for far better tests.

    Summative assessment? If you think about the purpose of schools — not just of education — then it is not hard at all to make a case for summative assessment.

    • learningspy says:

      Of course you’re right. The rights of ‘stakeholders’ are our justification for exams.

      I absolutely agree that schools serve the whole community and would perhaps go further: schools are a force for civilization. We exist to produce right thinking, productive members of society. Isn’t a summative judgement on this kind of thing a little unhelpful? And meaningless? We have made schools’ accountability about qualifications – this doesn’t have to be the case. It’s a de facto position that we subscribe to without really thinking about whether is should be this way.

      You point out that the public pays for schools. Well, in purely economic terms we get a lot more out than we put in. Maybe the public should be well pleased with the bargain they have struck? The public should be aware that not everything that counts can be measured and not everything that can be measured counts.

  4. (Page says 4 comments but only 2 are showing up so I apologize if I’m repeating what someone else has said.)

    Below is my thinking on summative assessment. It is informed by the writing of Wiggins, Stiggins, Martin-Kniep, Marzano, Brookhart and my own experiences as a teacher, student and a staff developer. Additionally, for the last ten years, my personal research question has been some variation on: “Can we quantify learning? Can we reduce learning to a number?”

    Assessment can occur at 3 moments of time: before (diagnostic), instruction (formative), or after instruction (summative). While assessment can serve a variety of purposes, be done by a learner, with peers, or facilitated by an adult, take a variety of formats, and be used to inform decisions in multiple ways – options for timing are limited. So summative assessment are those assessments that occur after learning has occurred. The phrase does not describe a type of assessment (e.g. test), a purpose (e.g. grades), or the driver for the learner’s action (i.e. intrinsic vs. extrinsic). As a result of its timing – after learning – comes its function: to demonstrate the learning that has occurred and to evaluate if learning has occurred. Caveat 1: This is not to imply we’re ever done learning or you can know all of “math”. Rather, there reaches a point in every learning cycle when it’s either time to move on or to put your learning cards on the table and show that your hard work has paid off (more on this below).

    For me, to talk of a system without summative assessments is to talk of a system with perpetual piano scales, and no recital. Debate preparation and research, with no actual debate. Of writing multiple draft copies, and never submitting the final for publication. Of learning how to be a diver, but never taking a diver’s certification test. Dress rehearsals without opening night. Being in a perpetual state of preparation, without the payoff.

    Multiple trades (e.g. cosmetology, auto mechanics, construction) require mastery of minimum competencies through a 2-part summative assessment (theory and practice). Without that assessment, a tradesman or woman cannot demonstrate to an employer that they’ve mastered the basics of the trade. The summative assessment serves two purposes – for the apprentice to show off what they’ve learned and for the master’s to have a common understanding of the minimum skills of those joining their ranks.

    Making the case for summative assessment is making a case for learning to have a purpose – even if that purpose is only learning for the sake of learning. Sometimes, we learn stuff just because we think it’s cool and there’s absolutely no practical application to our lives, nor will there ever be. (I took “Fashion in the Movies” as an undergrad – a fantastically challenging course that did nothing but threaten to lower my GPA. I’m neither in the movies nor fashion but man, that was a cool course.)

    In one of your tweets, you said formative assessments can be used to evaluate student learning and while on one hand, I would agree – than an assessment task can be used to both inform and evaluate, @russgoerend reminded me this morning that FA is really a process, not a series of products. Additionally, to steal a quote from Pellegrino, Chudowsky, and Glaser (2001): “In fact, it would be accurate to say that, in general, a specific assessment is neither formative nor summative—it all depends on how the information is used. … This noted, it is also true that assessments can and perhaps should be tailored to … be used for either formative purposes or summative purposes but not both.” So, I do not think we can evaluate student learning using products, performances, or processes generated during the formative assessment process. In other words, we know it would be unfair for a student who thinks they’re practicing for a debate to be told that, in fact, they’re actually debating and lost due to a weak argument.

    During formative assessment, the student is still playing scales, not at their recital. But there will come a point when parents fill the audience, shoes are shined, the lights are dimmed and the learner gets on stage to say: “Check it out folks, this is what I can do. This is what I’ve learned.” Meanwhile, in the wings, the student’s teacher is watching, evaluating their performance. Making the decision if they’re ready to tackle the Tychovski with Level 2 or if they should stick with Chopsticks and Level 1.

    Caveat 2: The teacher is most likely taking some notes for feedback e.g. soft shoulders, banged keys when playing A sharp. These features speak to the fact that assessment is a part of a larger whole that includes feedback, instruction, reflection, etc.

    Caveat 3: The teacher is making a judgment about the student’s learning but chances are that the recital isn’t the end all, be all for entry into Level 2. Additionally, it’s very possible that in my example above, the little piano player gets a wicked case of nerves and boots all over the keys, skewing the teacher’s judgment of their performance. This is an example of why a single assessment measure should not be used to make large-scale decisions about a student’s performance. In many cases where summative assessment present a negative or punitive connotation, it’s often because that’s the ONLY assessment being used to make evaluation decisions about a student’s abilities (this rule and many others and expressed in the National Council on Measurement in Education’s “Code of Fair Testing Practices in Education”) or they make up the bulk of the assessments the student encounters. This rule is broken by thinks like a bar exam, but in K-12 education, it’s pretty critical.

    For every instance of schools or classrooms with balanced, quality assessment systems there is probably one is that out of balance, relying too much on summative (students are in a perpetual state of performance with no room to make errors without penalty), ignoring diagnostic (teacher is teaching content because it occurs in the curriculum without knowing if the students already know the content) or focusing so much on formative that they are slowly going insane due to the lack of sleep. It is possible to anecdotally speak to abuse of assessment or claim that teachers are doing the wrong thing. However, anecdotes does not data make and discarding a component of a system because some teachers are struggling to incorporate it correctly does not make sense. I maintain that is about using the right tool at the right time, about creating an assessment system that is differentiated, as authentic as possible, balanced, and reasonable.

    • learningspy says:

      Jennifer, thank you for this well reasoned and enlightening explanation – I feel much clear on you position now. I’m pleased with the idea of that depending on one type of assessment would force learners into perpetual states of wither practice or performance – I’ve never really seen it quite as starkly before. Butt does perhaps seem a bit reductionist. When I qualified to be a teacher I was summatively assessed as having achieved a minimum level of competency. As a qualified teacher I now receive regular formative assessment to ensure that I am up to snuff. I don’t feel like I’m in a state of permanent practice – I feel like I ‘perform’ every day but that each of my performances is reflected on and used to ensure that I always improving. Could the way students are measured be more akin to this? You give the examples of cosmetologists, auto mechanics & construction workers also needing to achieve “common understanding of the minimum skills of those joining their ranks”. Maybe – but don’t we acquire mastery by doing and getting feedback? Plus I would take real issue with anyone who claimed that the only way to learn a musical instrument was to pass exams – I can cite hundreds of marvellous musicians who took an alternative (more formative?) route.

      I love the fact that you chose to study fashion in the movies, it does sound fun. But how on earth is this linked to the statement that “Making the case for summative assessment is making a case for learning to have a purpose – even if that purpose is only learning for the sake of learning.” If the purpose of learning is its own purpose why does it need to be assessed. You said yourself that there was absolutely no benefit for you in this course being assessed so what was it? Why didn’t you just do it for its own sake? You remind us that FA is a process rather than a series of products. Yes. Absolutely. So is life! So is fashion in the movies and playing trombone and teaching. I feel depressed at the thaught that these things can only be considered meaningful if someone says we’ve passed.

      Also, I’m fascinated by the ideas “that assessments can and perhaps should be tailored to … be used for either formative purposes or summative purposes but not both.” I’m not sure I understand why. Being as summative assessment (exams) is forced upon us, I would always make use of the results to provide as much formative feedback as possible. Why is this a bad thing? Or have I misunderstood?

  5. Justin says:

    WOW!

    She also says that the problem with using formative assessment to evaluate whether learning has occurred is that it is “unethical to evaluate a learner while they are still learning”.(hope this works)

    I never thought of it that way before. As a pre-service/future teacher I am still making up my mind on the battle between formative and summative assessment.

    For me at the moment, I can see the uses of both. The end of the day, Stakeholders in the students lives be it, parents, governments etc. need to see results on performance. Unfortunately this is a necessary evil that is placed not only schools, but throughout the corporate/business world. My work in the corporate world has shown that results and knowledge is measured.
    Students when they go into the business world need to see the importance of assessment in real terms as it effects so many things like, pay grade, position etc.

    So where do I lie right now? Thoroughly confused, I think only experience will help solve this one.

  6. Dr. Timony says:

    I cannot believe that we have these discussions at all. There are many ways to assess and they all serve particular functions. (I will skip the part where I make corrections to the original post but the definitions and criticisms are faulty.)

    Assessment Prohibitionists often fight against assessment tools because they are misused. As Beckett said and I often quote–“there he goes, blaming his shoes for the problems of his feet.”

    In many (MANY) other cases, those who claim that they do not “assess” still do so but they do it haphazardly which insults the students and the profession.

    Proposed “solutions” to summative assessment are not solutions at all but a conflagration of missteps that assume the worst from any teacher who would dare plan an assessment in the first place. Silliness.

    If anti-testing individuals want to pick a real fight that most rational peoples can rally behind it would be this: Create meaningful assessments that represent actual teaching and prevent admins, politicians, and others from misusing the results.

    • learningspy says:

      Dr Tim – I did ask for polite contributions. We can all quote Beckett out of context to support our points. Might generations of crippled Chinese woman have some justification for blaming their shoes for the problems of their feet? Ill fitting shoes are uncomfortable and damaging to out feet (especially children’s feet) so let’s take them off! How far would you like the metaphor extended?

      I have never claimed not to assess – only an idiot would think this a good idea. However this does not mean that tests are particularly useful or desirable. The consensus view is that we need them and I have to work within the system. Therefore (to a point) I teach to them.

      The really annoying thing is that “anti-testing individuals” come up with “meaningful assessments that represent actual teaching and prevent admins, politicians, and others from misusing the results”. It’s called Assessment for Learning. If only the pro testing lobby felt able to treat teachers as professionals and trust their judgements there might be no need for tests.

  7. Cristina says:

    I think Jennifer made the entire case for both types of assessment (and, oh, she could have turned that into a blog post in itself – it s an exemplary piece of logic, reasoning and expertise).

    Aside from the reasons she mentioned (some of which I do not want to reiterate although they were in my mind too when I replied to your tweets), it would be an interesting question to ask: why do some educators feel uncomfortable when summative assessment is discussed? Please note, again, that SA does NOT equate grades or competition – I am definitely not referring to the American education system. SA is a PART of a learning continuum – it comes at reasonably selected cycles of learning.

    I can only speak from my experience as a teacher of ESL (elementary school students). I DO use formative assessment (anecdotal records, rubrics, self-evaluation, peer-evaluation etc), but students need to see their “learning” at the end of longer periods of time. Although they get feedback daily on how/what they do (n.n. knowledge and skills), certain parts of the curriculum need to be reviewed and assessed summatively. You simply cannot do it otherwise because the curriculum is “taught”(read “uncovered”) gradually. That is when SA helps: they can identify WHAT they actually are left with after a period of learning and so they can set goals for themselves and improve in the future.

    Basically they place themselves on a CONTINUUM of learning and reflect on a piece entitled “I thought I knew…but I need to improve on…”. They decide what are the next steps – as each learns at his/her own pace.

  8. learningspy says:

    Thanks Cristina. As far as I can make out my difficulties seem to be purely semantic.

    If, as you and Jennifer insist, summative assessment is any kind of performance (play, recital, football match) then OK, I’m all for it. I had never encountered this definition before: you live and learn. I have always assessed in ways similar to the ‘continuum of learning’ you describe but have always referred to this as AfL.

    Next time I’m tempted to pontificate on the problems with testing/exams I’ll choose my words with more care.

  9. Jennifer Borgioli (@DataDiva) says:

    One quick point to clarify – you cannot ascertain if an assessment is diagnostic, formative, or summative just by looking at it. Players may hold a scrimmage football match so the coach can diagnosis players’ strengths. A football match may be started and then stopped when the coach realizes that several players are making the same mistake and provide formative feedback. Additionally, a football match may be a summative culmination of what the players learned during a week-long clinic. So in other words, summative refers to the timing of the assessment, not the design.

    Grant Wiggins and Rick Stiggins are two great authors around the big picture of assessment. As are Susan Brookhart and Giselle Martin-Kniep.

    • learningspy says:

      It’s great that Wiggins & Stiggins write on the same subject – I love rhyming authors.

      Yes, I had understood your point about timing, although I see by your use of the word scrimmage that you’re talking about a slightly different game.

  10. […] wrote a few months ago wondering about the point of summative assessment and was persuaded that it has it’s place. Mid course, at the point of writing, where learning […]

  11. […] Is there a case for summative assessment? […]

  12. […] doing good things? September 2011 What can engineers teach us about assessment? September 2011 Is there a case for summative assessment? September 2011 The joy of marking October 2011 Feedback: it’s better to receive than to […]

  13. […] Is there a case for summative assessment? […]

Constructive feedback is always appreciated

%d bloggers like this: