High stakes online assessments: A case study of National Benchmark Tests during COVID-19

Abstract

Owing to the COVID-19 pandemic, paper-based delivery of the National Benchmark Tests (NBTs) was not possible during the 2020 testing cycle. The NBTs, being a large-scale national assessment project, did not have alternative options, other than to offer the tests online. Moving these high-stakes tests online meant that certain considerations had to be considered to retain the credibility and security of the tests, without compromising the validity and reliability of the scores. Digitising the paper-based NBTs required an innovative, flexible and robust solution, which promotes fairness and ensures the quality of testing is maintained, while in many ways remains comparable to the paper-based implementation. To deliver the NBTs online, the following important considerations needed to be addressed: test security and integrity, test candidate identification processes, the prevention of dishonest behaviour, test scheduling and timing and technical support. The online testing solution chosen integrates the following aspects: it 1) enables all candidates to take the same test at the same time; 2) ensures the quality and similarity in experience of test delivery for all candidates as far as possible; 3) prevents candidates from accessing other applications and devices during the test; 4) enables proctoring before, during and after the tests to encourage appropriate behaviour similar to that expected during paper-based tests; 5) provides live support to assist candidates to deal with technical challenges and to guide them through the test sessions and 6) processes and presents data and scores in the same way as for the paper-based tests. In this article, we analyse the integration and complexity of the online NBTs solution, the opportunities and challenges associated with this form of delivery and reflect on test candidates’ and the team’s experiences. We discuss components of online assessment and wish to argue that this is also relevant to high-stakes course assessments. This case study should help to refine the scope of further research and development in the use of online high-stakes assessments.