A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care

Bakhshi-Raiez, F.; de Keizer, N.F.; Cornet, R.; Dorrepaal, M.; Dongelmans, D.; Jaspers, M.W.M.

doi:10.1016/j.ijmedinf.2011.09.010

« PreviousInternational Journal of Medical Informatics
Volume 81, Issue 5 , Pages 351-362, May 2012

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care

F. Bakhshi-Raiez
- Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands
- Corresponding author at: Academic Medical Centre, University of Amsterdam, Dept. of Medical Informatics, P.O. Box 22700, 1100 DE, Amsterdam, The Netherlands. Tel.: +31 020 566 5596; fax: +31 020 691 9840.
N.F. de Keizer
- Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands
R. Cornet
- Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands
M. Dorrepaal
- Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands
D. Dongelmans
- Department of Intensive Care, Academic Medical Center, University of Amsterdam, The Netherlands
M.W.M. Jaspers
- Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands

Received 25 May 2011; received in revised form 30 September 2011; accepted 30 September 2011. published online 26 October 2011.

Highlights

► User-based usability evaluations of an interface terminology. ► Usability is evaluated on five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. ► Detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. ► The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology is defining for its usability.

Abstract

Objective

To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System.

Design

Observational study with user-based usability evaluations before and 3 months after the system was implemented and routinely used.

Measurements

Usability was defined by five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. Qualitative (the Think–Aloud user testing method) and quantitative (the System Usability Scale questionnaire and Time-on-Task analyses) methods were used to examine these usability aspects.

Results

The results of the evaluation study revealed that the usability of the interface terminology fell short (SUS scores before and after implementation of 47.2 out of 100 and 37.5 respectively out of 100). The qualitative measurements revealed a high number (n=35) of distinct usability problems, leading to ineffective and inefficient registration of reasons for admission. The effectiveness and efficiency of the system did not change over time. About 14% (n=5) of the revealed usability problems were related to the terminology content based on SNOMED CT, while the remaining 86% (n=30) was related to the terminology application. The problems related to the terminology content were more severe than the problems related to the terminology application.

Conclusions

This study provides a detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology are defining for its usability. Carefully crafted domain-specific subsets and a well-designed terminology application are needed to facilitate the use of a complex compositional interface terminology based on SNOMED CT.

Keywords: Terminological system, Interface terminology, Usability evaluation, SNOMED CT