6 Ways to Support Unknown Observers in Observational Assessments

It is really useful to be observed and assessed by professionals when in practice – but frequently these observers will not be registered on the systems of the institution that is managing the practice observation process. For example, a university medical school may be managing their students’ observed assessments but it is usually not practical for them to maintain the details of every medical professional who may provide students with feedback in the placement hospitals and practices.

However it is important to be able to rely on the results of these observational assessments and we are often asked how the results of such assessments can be verified where the observer is unknown to the system. And how can we improve on the use of paper forms which can also be lost, damaged, tampered with, be illegible and which do not have any in-built verification process?

There are techniques which can be introduced to enable this ongoing flexibility and value of using “unknown” observers and we list here some of the approaches which could or should be considered in any service solution

• At the end of the assessment – design the system so that it asks for the observers email address. Explain on the form against this question that they will receive confirmation from the system using the email address provided.
• When the observer completes their required input, lock the assessment form so that the student cannot amend the information.
• When the completed observation is received by the host system, send an email to the observer confirming details that they have observed the individual in practice and attach details of their completed assessment. Request notification from the observer if the observer does not recognise the assessment or the student.
• Review reports on completed observations and ensure that sufficient variety of observers are being used for each individual, especially on critical competency areas.
• Ensure that technical measures are in place for catching and resolving incorrectly entered email addresses—for example, provide reporting on “bounced” emails sent to non-existent email addresses, and ways of manually approving particular email addresses.
• Optionally, enable the observer to enter a signature using the mobile device being used for the assessment. This provides a double inspection option, creates an impression that the assessment is dangerous to forge, and where a regulatory body has validated your course based on a paper-and-signature assessment process, means that you don’t have to go back and revalidate your processes. Mobile devices can handle such signature boxes quite well, especially when an inexpensive capacitive stylus for writing on touch screens is available.

We have also found that some institutions would like their “unknown” observers to be able to engage more closely with the assessment system for the individuals that they have observed and there is sometimes the requirements to consider the following additional options:

• Enabling the observer to be registered onto the system automatically using their email address as identity, providing them with a password so that they can view restricted detail about that students’ progress.

The overall effect of supporting the above processes is that it can provide a robust process for enabling the use of unknown observers in practice settings.

As we continue to work with a wide range of medical schools worldwide we will continue to share our experiences through this series of Top Tips.