Digital clinical assessment – the new normal?

By Sheik Abdul Malik

It would not be an understatement to say that the COVID-19 pandemic has affected every single student's lives over the past 12 months. But what about those students for whom social distancing is simply not possible and in-person interaction is a necessity?

In an unprecedented move, the General Medical Council (GMC) decided to make significant changes to examinations taking them from the clinical to the virtual environment. Can pioneering such assessment formats at the postgraduate level in a field that has been reliant upon patient-centric face-to-face care allow for accurate determination of clinical ability?In a 2021 paper submitted to the Journal of Oral and Maxillofacial Surgery [1] researchers considered the implementation of unique virtual assessment methods that would not only allow surgical and medical trainees the ability to continue enhancing their career progression but also prevent the proliferation of the coronavirus in the confined surgical spaces that are typically used to carry out such assessments.

However, it was not just the GMC that turned to the virtual world as a place to host such vital examinations. Assessments such as Membership of the Royal College of Psychiatrists (MRCPsych) and Membership of the Royal College of Obstetricians and Gynecologists (MRCOG) among others turned to online platforms to verify medical and surgical knowledge, forgoing the traditional demonstration of practical clinical skills.

Digital Disparity

Whilst other disciplines have utilised the online sphere to conduct assessments over recent years, this is a relatively new phenomenon within the field of medicine, where maintaining people-focused practice is key to its rigour. As it is unlikely that the typical interpersonal methods of communication standard to operating theatres and hospitals are likely to return for students any time soon novel virtual assessment methodologies are being increasingly utilised. Whilst adapting written examinations for an online platform is not without its challenges, it is something that has been done before by medical schools [2]. Despite this approach being an effective way to reduce the transmission of viruses such as COVID-19, regulators like the GMC have faced logistical difficulties in ensuring fairness and checking for potential sources of cheating. More broadly, online examinations have several disadvantages, some highlighting the various disparities between candidates.

Assessing the Problems

The high costs of creating and implementing digital assessments coupled with the potential technical issues associated with this can be costly. Additionally, specialised training may be required to ensure ease of use of the new technology. Furthermore, it is also the candidates that can be negatively impacted by transitioning to an online method of examination. Students who lack the adequate resources to purchase high-quality computing facilities can face barriers to participation. However, it is not merely financial considerations that can disadvantage students, A lack of practical assessment of surgical skills and patient examination can leave candidates feeling under-equipped to deal with real-world clinical problems.

Conversely, there are potential advantages to facilitating online assessment for postgraduate students in the medical and surgical fields. Those who reside internationally can participate without the need for travel which has sustainability implications too. A virtual examination removes the need for a physical venue space that can be economical. Furthermore, candidates can be in a familiar and comfortable space when sitting their exam and this may allow them to perform more effectively. Vitally, combined, the lack of international travel and sitting exams in a confined space with others would not be a contributing factor to the spread of COVID-19.

Taking the Clinical into the Virtual

The Royal College of Psychiatrists (MRCPsych) has gone a step further and taken their clinical examinations online too in a bid to minimise the negative effects of the pandemic. Other institutions have continued to run their face-to-face assessments with stringent social distancing measures in place and the universal use of facemasks.

One thing that should be considered as a distinct advantage is that the pandemic has facilitated the need for increasing advanced digital literacy as well as refined verbal and non-verbal communication cues – a distinct asset in the medical arena.

Despite this, there is one thing that is still vitally important for candidates to engage in, even in a digital world and that is practice. Continued use of a computer for extended assessments can impact concentration, however, by following techniques such as maintaining hydration and taking regular breaks, these effects can be minimised. Additionally, adequate practice in advance of online assessment can help to gain familiarity with a new technological interface and put the candidate at ease on the day. Whilst it is not feasible to eliminate all technical issues or distractions when taking an online assessment, it is possible to still complete it to the same standard as that of an in-person exam.

The Future of Assessments

Whilst it is impossible to fully predict what medical and surgical assessments of the future will look like for postgraduate candidates, we will likely be living with this pandemic for many years ahead. Whether clinical, patient-focused examinations can be facilitated fully online is still to be decided but one thing is for sure, many written assessments will continue to be held online. This provides a unique opportunity for students to adapt to a new way of performing and developing their digital skills. Clinical practice in the future could move further into the online sphere leading to new technological developments and improved patient care.

FAQ

Q: How do virtual assessments compare to traditional in-person assessments in terms of effectiveness and student satisfaction? Are there any studies or metrics that show the impact of virtual assessments on learning outcomes and exam performance?

A: Virtual assessments have been a subject of much debate regarding their effectiveness and the satisfaction they provide to students. While some studies suggest that virtual assessments can offer comparable outcomes to traditional methods, the student voice indicates a mixed response. Factors such as ease of access, flexibility, and the ability to take exams in a comfortable environment are often cited as benefits. However, concerns about technical issues, the lack of direct interaction, and the potential for increased anxiety also emerge. The effectiveness of virtual assessments often depends on the discipline, the design of the assessment, and the support provided to students. Metrics on learning outcomes and exam performance tend to vary, with some studies indicating that well-designed virtual assessments can lead to outcomes similar to those of in-person exams, while others suggest the need for further refinement to match the effectiveness of traditional methods.

Q: What specific technologies and platforms are being used to facilitate these virtual assessments in medical and surgical fields, and how do they ensure the integrity and security of the examination process?

A: In the medical and surgical fields, various technologies and platforms are being used to conduct virtual assessments. These include video conferencing tools, virtual simulation software, and secure online examination platforms that mimic the structure and content of traditional exams. To ensure the integrity and security of the examination process, these platforms often incorporate features like secure login procedures, surveillance to prevent cheating, and encryption of data. The student voice highlights the importance of clear instructions and support in using these technologies effectively. Ensuring the fairness and reliability of the exams is paramount, with ongoing efforts to improve the technology used to better replicate the conditions and challenges of in-person clinical assessments.

Q: Given the necessity of practical skills in medical education, how are institutions planning to address the gap in hands-on clinical experience that cannot be replicated virtually? Are there any innovative solutions being developed to simulate or substitute this aspect of medical training?

A: The gap in hands-on clinical experience due to the shift towards virtual assessments is a significant concern in medical education. Institutions are exploring various innovative solutions to address this challenge. Among these are virtual reality (VR) simulations, augmented reality (AR) applications, and advanced computer-based simulations that offer interactive experiences mimicking real-life clinical scenarios. These technologies allow students to practice procedures and decision-making in a safe, controlled environment. Additionally, hybrid models combining virtual learning with limited, carefully managed in-person sessions are being developed to ensure students receive essential hands-on experience. The student voice plays a critical role in these developments, with feedback being used to refine and improve the simulation experiences to make them as close to real-life practice as possible. The goal is to ensure that despite the constraints imposed by the need for remote learning, students can still develop the practical skills crucial for their future careers in healthcare.

References

[Source Paper] R. Ellis, R.S. Oeppen, P.A. Brennan, Virtual postgraduate exams and assessments: the challenges of online delivery and optimising performance, British Journal of Oral and Maxillofacial Surgery, Volume 59, Issue 2, 2021, Pages 233-237, ISSN 0266-4356,
DOI: 10.1016/j.bjoms.2020.12.011

[1] Ahmed Elzainy, Abir El Sadik, Waleed Al Abdulmonem, Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University, Journal of Taibah University Medical Sciences, Volume 15, Issue 6, 2020, Pages 456-462, ISSN 1658-3612,
DOI: 10.1016/j.jtumed.2020.09.005