What Is The Impact Of SLA Marking On Student Attainment In Science?

As part of the Extending Influence and Impacting Others module of the PGDE via Teachfirst, second year trainees have to carry out an action-research-style project on an area of their choosing. This is Jenny's summary of the evaluation she has carried out in Science over the past two terms.

By Jennifer Scott

In my final module for my PGDE, I recently investigated the research question: What is the impact of detailed SLA marking on students’ attainment in science? Here I will outline the investigation, in the hope of sparking some discussions around marking across the school! Motivation: My main motivation for this investigation was that day-to-day discussions with other teachers suggested that our written feedback in Science was not having the impact that it ‘should’. Therefore, I wanted to investigate whether the approach to SLA marking we were using in Science could be improved. Literature summary (in brief): The EEF has a wide-ranging Marking Review including several suggestions. Some which we have already embedded into the marking policy itself are: to provide specific, actionable feedback; and to allow students time in lessons to respond. There is also a word of caution: if students are producing “superficial responses” then the impact of marking is likely to be smaller. A lack of motivation appears to be a significant factor in whether students respond in sufficient depth to their actions One potential cause of this could be their mindset: as described by Dweck (2006) (as well as on the poster in every classroom!), students with a ‘fixed mindset’ are less likely to believe that they can improve their ability, whereas students with a ‘growth mindset’ believe that they can improve with effort and practice, and thus are more likely to make the most of SLA feedback. Also, Henderson and Harper (2009) noted that whilst teachers generally viewed assessments as formative, students viewed them as summative, and therefore were less motivated to improve their knowledge on a given subject after the assessment. A suggestion from the EEF which we have not yet embedded is to distinguish between “mistakes” (caused by carelessness) and “errors” (caused by lack of understanding).

Methodology and Results: Part 1: Surveys to collect teacher and student views All Science teachers completed a survey on their views of science SLA marking, as did 83 Year 9 students. The results bore out five key messages about our current SLA marking: A high proportion of students do not complete their SLA responses fully; More than 80% of students feel they understand the purpose of SLA feedback (despite only 50% of teachers thinking they did!), but they tend to frame it negatively e.g. “to see what we got wrong” (as opposed to “to see how to improve” etc.). This provides some evidence of a lack of student ‘growth mindset’;

• Consistency in marking methods is present between teachers, but teachers are not sure if these methods have a positive impact on students;
• Teacher workload: if the feedback is not having a positive impact on students, time spent marking feels wasted.
• Accessibility of actions: teachers felt that content often needed to be retaught in order for students to respond to actions, while students equally felt that if they did not know the answer in the test, they still would not know the answer for the SLA. Some teachers have already been working to improve this by providing students with specific resources to use when completing their actions.

All these points served to reinforce my motivation and informed some trial changes made in part 2. Part 2: Comparing a ‘normal’ SLA, no SLA, and a new SLA I chose to carry out this investigation with my two Year 9 classes in parallel. After a regular end-of-topic test, each class received feedback before being given a ‘retest’. This took place in two stages (at the end of two consecutive topics). 9Z4 were the ‘control’ group, completing the usual style of SLA after going through a few ‘frequently missed’ questions with the class, before completing the retest the following day. They did the same process in both stages. This consists of a Strength I identify, a specific Literacy target, and 2-3 specific Actions, which are usually questions or tasks which ask students to consolidate upon or expand upon areas of misconception I had picked up from the marking. 9X4, on the other hand, did not complete an SLA in stage 1: they only went through frequently missed questions as a class, before immediately completing the retest. In the second stage, 9X4 completed an SLA with some trial changes, before completing the retest the following day. Neither class was given much warning of the retest, to increase the likelihood that any improvement was due to the feedback and not extra study! The main change in the updated version of the SLA involved students completing a ‘review table’ of all questions, noting why they lost marks and giving an explanation/correction (example seen below). This is strongly weighted towards student metacognition rather than personalised teacher feedback. I tried to incorporate a teacher-directed action as well, but it felt like too many different tasks for students to complete within the SLA lesson. The surprising results of the two stages can be seen below: there was no statistically significant difference in the percentage of students improving their score, regardless of whether they had completed the SLA or just gone through some questions (stage 1). It should also be noted that the average percentage score improvement was roughly the same at ~13%. Equally in stage 2, there was no significant difference in percentage of students improving between the current and updated SLA methods. This could be for various reasons, not detailed here for brevity.

Conclusions and looking forward: Overall, it can be said that the current SLA marking in the science department ‘works’ (for most students; in the short term), in that it does allow students to improve their knowledge on the given topic. However, it does not give a significantly different improvement in students’ attainment than simply going through the most frequently missed test questions with the class. This raises the following questions:

• How can we ensure that students engage fully with marking?
• How can we frame end-of-topic tests more formatively, so that students are more likely to see the importance of working to improve on them?
• What is the best approach to marking end-of-topic tests to ensure they have maximum impact on student learning?

I would be very interested to hear others’ views on the questions posed here, how they apply within their own subjects and any suggestions in response (I have a few of my own)!

References:

EEF (2016) A marked improvement? Education Endowment Foundation. Available at: https://educationendowmentfoundation.org.uk/public/files/Presentations/Publications/EEF_Marking_Review_April_2016.pdf.

Dweck, C. (2017) Mindset - Updated Edition: Changing The Way You Think To Fulfil Your Potential. Hachette UK. Available at: https://play.google.com/store/books/details?id=ckoKDQAAQBAJ.

Henderson, C. and Harper, K. A. (2009) ‘Quiz Corrections: Improving Learning by Encouraging Students to Reflect on Their Mistakes’, Physics Teacher, 47(9), pp. 581–586. doi: 10.1119/1.3264589.

Bitesize Research: Understanding The Impact Of Sport On Student Self-Control And Attainment

By Jaya Carrier

I hope everyone has had a great week so far! With the excitement of Sports Day coming up, I was interested in looking at a study from the University of Cambridge in PLOS One in May 2021 that looked at how sports and exercise impact young people’s self-control, and therefore their attainment too.This was a study that looked at the data of 4,000 students across the UK. Objectives: To understand the impact of sports and exercise on student self-control and attainment. Summary: The key findings were:

• Those students who do more physical activity are more likely to have stronger self-regulation and that this, in turn, has a positive impact on academic outcomes
• Physical activities found to have the strongest impact are ball sports and swimming. This positive impact on self-regulation and on academic attainment is even more pronounced in disadvantaged children
• One recommendation from the author was to ‘rethink’ COVID catch-up discussions in line with these findings - maximising classroom time needs balance, as play and sports need to also be prioritised.

How does this impact me and my practice?:

Some reflection questions arising from this that might be helpful to consider are:

• How might I support students I work with to participate in more physical activities? What positive impact might this have on them?
• How might I draw on the experiences of sports or exercise in supporting students to self-regulate in other settings?

If you would like to discuss this further with me - please let me know! I’d be delighted to open up these discussions and conversations.

Bitesize Research: How Types of Anxiety Affect Student Well-being And Performance

By Jaya Carrier

I hope everyone has had a great week so far! For this week’s bitesize research, I was interested in looking at a study from the Journal of Youth and Adolescence in March 2021 that looks at different types of student anxiety, and the relationship these have to psychological wellbeing and academic performance. This was a study that took place in China, and looked at 715 students. Objectives: To understand how five different anxiety symptoms (including generalised anxiety, separation anxiety, social anxiety, school anxiety, and panic disorder) develop in young people and how these impact psychological wellbeing and academic performance in young people. Summary: The key findings were:

• Students with generalised and social anxiety reported the lowest level of psychological well-being. Their life satisfaction and self-esteem levels were the lowest, while they displayed the highest levels of depressive symptoms and suicidal ideation.
• Students in the group with predominant social anxiety showed the highest academic achievement.
• The group with school anxiety and congruent high anxiety displayed the lowest academic achievement.

How does this impact me and my practice?:

Some reflection questions arising from this that might be helpful to consider are:

• What different types of support might students with different anxiety symptoms need?
• What sorts of strategies have I tried with students showing different anxiety symptoms? How effective have they been? How can I continue to refine them?

If you would like to discuss this further with me - please let me know! I’d be delighted to open up these discussions and conversations.

Book Review: Black Box Thinking

 By Charlotte Nicholas

I recently finished reading ‘Black Box Thinking' by Matthew Syed and I cannot recommend it highly enough to everyone. If you have seen me anytime in the last few weeks you no doubt will have heard me jabbering about some of the ways that it has inspired me. 

WARNING! - The next part of this post will include some spoilers!

The premise of the book is to use ‘black box thinking’ a term used by Matthew Syed as an approach to failure. The first comparison that he draws upon is between the aviation and the medical industries. The book uses anecdotal evidence to give stark contrasts between different industries and their approach to failure. In the first examples of the book, Syed compares a routine operation that went wrong for several reasons; failure for a subordinate to challenge a superior, the surgeon's loss of the concept of time and inability to perform an emergency tracheostomy. The woman unfortunately died.

 In contrast to this, an aviation accident that occurred in the 1970s suffered a failure due to similar issues concerning hierarchy and the concept of time. Syed highlights the differences in the way both incidents were dealt with afterwards. In the medical example, euphemisms were used to cover up the mistakes, a selection of phrases we’ve all heard before -- ‘complication’, ‘accident’. It was only after the husband of the deceased called for an investigation into her death that avoidable mistakes were found. In contrast to this, in the aviation accident, the plane was fitted with a black box that recorded not only the mechanical issues with the plane but also the conversations of the crew members. Not only did this lead to changes in plane engineering but new processes and procedures that factored in the psychology of high-pressure situations, even a new process to get around the notion of challenging superiority. In the aviation industry, it is claimed that thousands of lives are saved due to this approach to error, that it is a way to progress and that crucial lessons can be learnt from mistakes. 

The book also includes sections that focus on the way in which closed feedback loops, fixed mindsets, cognitive dissonance and confirmation bias have a negative impact on the way that we deal with failure, mistakes and innovation. One of the points that really resonated with me is that literally all innovations/ inventions in the world have been created as a response to something not going well or working properly and the only way to overcome this is through trial and error with trial just as essential as error. Put it simply, we have to fail in order to get better at something or find a solution. 

You’re probably thinking that this is all riveting stuff if you want to learn about how James Dyson invented the Dyson dual cyclone but how does it relate to our teaching? Andy McHugh from the ‘Teaching and Learning Guru’ discusses how he had read the book and how it has informed his teaching practice. I will summarise the crux of the article where McHugh discusses an example teacher named ‘Steve’ who had traditionally got excellent A-level results until one year he didn’t. Steve couldn’t explain the ‘failure’ (all kids still passed). The students were no weaker than the previous year. However, the students were different, the same data would not lead to the same results when Steve had assumed that they would. Steve had been using assessment data to monitor his students' progress but hadn’t considered how data might indicate progress but does not necessarily guarantee it. In his practice he had used rigorous assessment, assessment questions from exam boards and was a seasoned examiner so was a good judge of student responses. However, Steve had focused entirely on improving skills and techniques, what he found was that the kids simply didn’t know the content. He asked for some of the scripts back from the exam boards and found that each student had a very specific reason why they hadn’t done so well. 

Student A - had suffered a bereavement in the family. 

Student B -  had poor attendance and had plagiarised in-class assessments and homework.

Student C - had been consistently given detailed feedback from Steve. 

This shows us that as teachers we often use data to inform our judgement of what to plan and teach but where this falls down is that we sometimes don’t consider the specific and individual issues that students face, the story behind the data as it were. 

How could I apply this to teaching?

After the popularity of his book, Syed was asked how he might apply the lessons from his book to the teaching profession. He suggests the following three things: 

‘1 – get the language right in the classroom. Praise for effort, rather than for outcome or talent, but that doesn’t mean endlessly saying, ‘Well done, you worked hard’. To create a really great variety of interactions you need a rich language that will orient learners towards learning.

2 – foster more collaboration in the staff room, so that if particular teachers are struggling with an area of their work, they feel they can ask for help and feedback. Perhaps they can video one of their lessons and talk it through afterwards with a mentor.

3- adopt a motto in the school that articulates a growth mindset in some way that students will relate to. They’re just three of a whole array of things, but those would be a good start.’

What are the key takeaways?

From this my key takeaways that I am going to try and ingrain in my practice are:

• We don’t need to cover up our mistakes but face them head-on and work out how we can learn from them. Not only do we need to do this in our own practice but attempt to engrain this way of thinking into our students. This links nicely to Divisha’s mantra of ‘Improve not prove’ that she has introduced to the Humanities department this year. 
• We need to teach the students resilience, students need to see mistakes and failure as something that will benefit them in the long run even if it doesn’t feel so good at the time. 
• We need more balance between accountability and learning. Students need to be responsible for their own learning and deal with their own mistakes accordingly. 
• We need to consider the story and faces behind the data, data does not guarantee progress and results.
• We need to let the kids fail! With increased pressure on teachers to get good results, we are under pressure to get students to rote learn and run the risk of exam factory-esque teaching. Whilst this helps students pass their GCSEs, what are we really teaching them about the real world? They need to learn how to deal with mistakes so that in the big wide world they are resilient, resourceful and pragmatic. 

I hope this post has made you think a little bit about how you deal with mistakes and failure and how you might encourage students to become a ‘Black Box’ thinker. 

I’ll leave you with a quote from Syed to end on: 

 ‘Black Box Thinkers have a mindset directed at learning and always trying to improve their performance. In schools, they would be teachers and leaders who look for those marginal gains that can help them improve what they do so that students leave the school with all the skills and attributes they need to change the world.’  

Thanks for reading! 

If you’re interested in reading more about this, please see the links below: 

https://www.teachwire.net/news/black-box-thinking-matthew-syed-on-what-schools-can-learn-from-the-aviation  

https://www.youtube.com/watch?v=-r0avhWk-xk&feature=emb_imp_woyt 

https://www.youtube.com/watch?v=MmVCYqs3mko 

https://www.teachingandlearningguru.com/black-box-thinking-for-teachers/ 

Bitesize Research: How Observation Feedback Impacts Teacher and Student Learning

  By Jaya Carrier

I hope everyone has had a great week so far! For this week’s bitesize research, I was interested in looking at a study by Song et. al published in March 2021 in the Journal of Research on Educational Effectiveness which looked at teacher professional learning in relation to teachers gaining feedback about their practice. It also looked at impacts on student achievement. This was a large-scale, randomised trial that took place over two years in the US with middle schools. Objectives:

To understand - 1) the impact of giving teachers feedback and facilitating discussion on their practice following observation of that teacher’s practice 2) the impact of working with teachers to look at student achievement in comparison to similar students in similar areas Summary:

The key overall themes are:

• There was a significant improvement in classroom practice following classroom observations, discussions and feedback
• There was a very small improvement in student achievement

How does this impact me and my practice?:

Some reflection questions arising from this that might be helpful to consider are:

• When I’ve been observed and have taken part in discussions about the observation, what did I do to ensure I made the most this powerful tool?
• What characterises the feedback have I found helpful for my practice?
• How often am I observing the practice of others and offering feedback to support their improvement?
• With respect to student achievement, comparing student achievement to those in similar schools and with similar backgrounds is something that previously we were able to do with Progress 8 results. Without this metric being available due to the changes to the examinations in 2020 and 2021, what other opportunities can I take to engage in this kind of analysis of student achievement?

If you would like to discuss this further with me - please let me know! I’d be delighted to open up these discussions and conversations.

How To Make Lessons More LGBT+ Inclusive

 By Holly Youlden

As well as (finally) having some warmer weather, June also brings with it Pride, celebrating the LGBT+ community! Having loved the recent Channel 4 series ‘It’s A Sin’ starring Olly Alexander, I found a recent article in Vice, where Alexander discusses his new book of essays on the future of LGBT+ rights. In the interview, he focuses explicitly on his experiences of homophobia and lack of LGBT+ visibility during his secondary school education, and the impact this had during his early adult life. Going to school is one of the most ‘normal’ things a young person can do, and this reminded me of the important contributions we can make in helping to normalise LGBT+ lives and, as the wonderful Chris Chimonas summarised for me, “creating an alternative narrative to negative stereotypes'' which our young people are often exposed to.

So the next question is how can we, as teachers, use Pride month as an opportunity to celebrate positive images of the LGBT+ community?

As ever at WA, we’re lucky enough to have some excellent examples of what this could look like and how you can make your lessons as LGBT+ inclusive as possible in this month, and beyond! 

Think about subject-specific applications

Chris has put together a fantastic selection of subject-specific examples here to get you started about how you can add an LGBT+ focus to your lessons. Are there any LGBT+ figures you can reference? Is there any LGBT+ historical context that is relevant to what you’re discussing? Could the case study, poem or example you’re using feature characters who are LGBT+? Discussing ideas between colleagues and departments is a great way to get some inspiration. Educate & Celebrate also have a great bank of lesson resources that could be used this month. One example I thought was particularly interesting was this French lesson aimed at high-ability KS4 or KS5 students and our very own English department put together this great set of LGBT+ authors to use in lessons. 

Weave inclusivity into your dialogue

As teachers and tutors, we are used to having conversations with students about a huge variety of topics. In order to ensure that we create the open and inclusive classroom environment we strive for, it is important the language we use is supportive and terms used are correct. Make sure you are confident in your sexual orientation and gender identity lexicon by using some of the very easy resources from Educate & Celebrate here. Think about how you can use a strong understanding of simple definitions to challenge misconceptions from students in a way that demonstrates inclusivity in every conversation we might have in lesson.

Think about the broader curriculum picture

As Charlotte Nicholas (History) mentioned in her recent BLM blog post, one of the bigger challenges is how we ensure we have a genuinely diverse, inclusive curriculum, rather than tokenistic ‘add-ons’. This article talks about how we can move away from stand-alone lessons on an LGBT+ figure and towards creating lessons that challenge heteronormativity using Banks’ tiers of integration. This four-tiered approach offers educators steps to help implement multicultural content as individual courses and programs or, ideally, across the curriculum. 

More ideas for how to support LGBT+ students can be found here. 

Please let us know if you see any great examples of LGBT+ positive activities in lessons as we’d love to hear them! 

Bitesize Research: Understanding Why Cold Calling Is Effective To Assess Learning

 By Jaya Carrier

I hope everyone had a great half term, and that this first week back is going well. For this week’s bitesize research, I was interested in looking at Tom Sherrington’s take on cold calling in the classroom from February 2021. Objectives: To understand why cold calling is so effective as AfL, and how it might be effectively implemented in a classroom setting. Summary:

The key overall themes are:

• Cold calling supports inclusivity in the classroom wherein everyone’s contributions matter
• Cold calling supports accountability in the classroom - there is no opt-out
• Students are made to think routinely by way of cold calling - it causes a cultural shift
• Cold calling should be the default position for questioning in all classrooms but can be used as part of a wider repertoire of questioning techniques too
• Sherrington notes that cold calling is really powerful in remote learning as well as in-person.
• Sherrington also offers a number of strategies in the article to support with cold calling as effectively as possible

How does this impact me and my practice?: Some reflection questions arising from this that might be helpful to consider are:

• How much do I use cold calling in my classrooms? Do I use it as the ‘default mode’? What works well in using it?
• How useful are Sherrington’s cold calling strategies in my classroom? What ones can I add or develop in my practice?