This year, due to the ongoing impact of the coronavirus (COVID-19) pandemic, many assessments for vocational and technical qualifications (VTQs) and other general qualifications had to be adapted or did not go ahead. COVID-19 caused different levels of disruption across the country, so many colleges, schools and training providers were not able to teach all the necessary qualification content.
Vocational qualifications are frequently modular and assessment typically happens throughout the course of study. They also tend to have a high proportion of teacher assessment. Some assessments are necessary to confer occupational competence or are 'Licences to Practice'. This diversity meant that it was not possible to prescribe a single approach to either adaptations or teacher-assessed grades (TAGs).
We asked awarding organisations (AOs) to balance carefully mitigating the effects of COVID-19 with the need to provide fair and reliable results that allowed learners to progress. In many cases, VTQ results have been determined using normal assessment methods. Some learners will have carried forward centre assessment grades (CAGs), from 2020, for some units or components. Some results have been determined through adapted assessments, including remote invigilation. Other results have been decided wholly, or in part, using TAGs like those used for A levels and GCSEs.
TAGs had to be based on evidence of a learner's achievement. Teachers drew on a range of different types of evidence to inform their professional judgement about a learner's result. AOs advised on the process teachers and tutors should follow when determining TAGs and how to check that decisions were evidence-based and consistent. Most learners had already completed assessments, many of which had been marked and, in some cases, moderated earlier in the course. Teacher judgements had to be reviewed by other subject teachers and assessors, as well as the relevant head of department or head of centre before TAGs could be submitted to AOs.
Many AOs reviewed centres' quality assurance procedures upfront, while others adapted their normal verification or moderation processes to provide centres with additional support throughout. Some AOs required centres to confirm that they had complied with their internal quality assurance arrangements, but other AOs confirmed this through their monitoring and quality assurance processes.
After TAGs were submitted, AOs conducted their own quality assurance processes. AOs took different approaches to this, depending on their qualifications and the part that TAGs had played in the final result. Some AOs did this through an extension of their normal centre monitoring, some asked centres to provide rationales for any results that looked unusual and also looked at work from centres selected at random, others looked at a sample of learner evidence, and many conducted a combination of these activities.
Results this year have only been possible due to the hard work of everyone involved, not least learners themselves. We have placed trust in the system, and especially in teachers and tutors. They have done everything possible to help learners achieve their qualifications and progress in their education or career.
Learners who think there has been a mistake with their grade can choose to appeal via their school or college. Learners who are unhappy with their grade can talk to their centre about taking an assessment instead.
2021 results
This year we have carried out a range of analyses to support understanding of results.
Outcomes
We primarily focus on the issuing of top grades, rather than the issuing of passing grades, as the percentage pass rate in our dataset is usually close to 100% in all years, as fail grades are often not reported.
In every year the cohort of VTQ learners differs quite substantially with regard to their numbers, their ages, their prior attainment, and the centres and centre types they attend. The general ability of cohorts of learners might also have changed over time. The qualifications vary according to factors such as qualification size and types of assessment.
This year we saw an overall increase in the volumes of Level 3 entries for Applied General qualifications, other VTQ performance table qualifications and 'other general' qualifications. This follows a similar increase in 2020. This is primarily due to a rise in entries for Applied General qualifications.
We did not see any substantial changes to grade distributions across Level 3 VTQs, suggesting general stability across 2018, 2019, 2020 and 2021.
In terms of the proportions of top grades being issued, the patterns of changes are varied. In many Level 3 qualifications the proportions of top grades being issued has increased.
However, for some Applied Generals, the proportion of top grades has slightly decreased, following an increase between 2019 and 2020 (7.1% in 2019, 19.9% in 2020 and 19.1% in 2021).
Equalities
AOs looked at student work from a sample of schools and colleges. They found no evidence that teachers' judgements were biased in favour of one group of students or another.
We conducted equalities analyses to check whether the gaps between the average results of different groups of students, which are observed in normal years, have changed this summer. We compare the extent to which the relationship between results and student background variables in 2019 and 2020 are similar in the 2021 outcomes. In doing so, we take account of the prior attainment of the students. This included a consideration of gender, ethnicity, free school meal eligibility (FSM), the Income Deprivation Affecting Children Index (IDACI), and special educational needs (SEND).
The analyses show general stability in the differences in outcomes for students with different protected characteristics compared to 2019 and 2020.
There are some small changes in the results of some groups of students, which are observed in normal years, which may reflect the impact of COVID-19. We did note that males became slightly less likely to achieve top grades compared to females in 2021. Males were on average 1.5% less likely to achieve top grades than females in 2019, which increased to 4.2% in 2020, and increased again to 5.9% in 2021. We also saw this in 2020 relative to 2019. For Applied General qualifications there was also a small increase in the attainment gap over time for learners assigned to the 'very low' prior attainment group. The learners assigned to the 'very low' prior attainment group were on average 2.7% less likely to achieve top grades compared to their middle ability peers in 2019. This difference increased to 6.8% in 2020, increasingly slightly to 8.6% in 2021. A small increase in the attainment gap over time between high and medium prior attainment learners was also observed.
Regional results
We conducted analysis to identify if any differences in the attainment of top grades appear to exist across different regions. We found that the degree of change for each region between 2018, 2019, 2020 and 2021 is relatively consistent, accounting for the fact that a degree of variation is always to be expected. The changes occurring in 2021 from previous years for any one region do not seem out of the ordinary, relative to other regions, suggesting a picture of stability across regions.
