Lessons learned : post-review procedures for electronic assessment

Background: Assessment plays a crucial role in university teaching, as it usually determines the course of study or the attainment of an academic degree. In veterinary education, exams carry a significant level of responsibility, certifying students’ competence to practice as veterinarians. Consequently, it must be ensured that the assessment procedures uphold a high standard of quality. In this context, electronic examinations are an increasingly integral part in veterinary education, as quality assurance measures can be implemented efficiently with this form of examination.

The University of Veterinary Medicine Hannover (TiHo) employed a system from an external service provider to conduct electronic examinations, which was initially introduced in 2008 for summative exams. The service provider later developed a new system, firmly incorporating quality assurance measures for the assessment process, including automated post-analysis of exams, which was adapted to TiHo’s requirements. The TiHo began to use this newer platform in 2017.

Summary of Work: The research focused on post-review measures aimed at evaluating, ensuring and improving the quality of examinations and items. Various parameters were identified that facilitate a quick assessment of an individual test post-execution, for example the reliability (Cronbach’s α), distribution of results as well as item and distractor analysis. Additionally, a comprehensive, comparative evaluation of all exams conducted since 2008 was carried out in order to pinpoint strengths and weaknesses in the process and to be able to further develop the procedure. For this purpose, characteristic values of the assessments, items and item formats were recorded and the log data was analysed. As a result, the number of items per exam, the proportion of reused items, time limit per item and the selection of item formats were defined as factors, whose adjustment can lead to an improvement in quality. Notably, multiple choice type A and image-based formats in particular have proven to be of high quality.

Take Home Message: A well designed post-review process adapted to the needs and procedure of the educational institution helps to ensure quality in the electronic examination process. In addition to the post-review of the exams and items following exam administration, an overall evaluation of all exams and procedures is essential to identify potential for improving the assessment procedure.