Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis

Saitwal, Himali; Feng, Xuan; Walji, Muhammad; Patel, Vimla; Zhang, Jiajie

doi:10.1016/j.ijmedinf.2010.04.001

« Previous Next »International Journal of Medical Informatics
Volume 79, Issue 7 , Pages 501-506, July 2010

Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis

Himali Saitwal
- School of Health Information Sciences, The University of Texas Health Science Center at Houston, Houston, TX, United States
- Corresponding author at: School of Health Information Sciences, The University of Texas Health Science Center at Houston, 7000 Fannin Street, Suite 600, Houston, TX 77030, United States. Tel.: +1 832 573 4613; fax: +1 713 500 3929.
Xuan Feng
- Arizona State University, Department of Biomedical Informatics, Phoenix, AZ, United States
Muhammad Walji
- School of Health Information Sciences, The University of Texas Health Science Center at Houston, Houston, TX, United States
Vimla Patel
- School of Health Information Sciences, The University of Texas Health Science Center at Houston, Houston, TX, United States
Jiajie Zhang
- School of Health Information Sciences, The University of Texas Health Science Center at Houston, Houston, TX, United States

Received 21 September 2009; received in revised form 1 April 2010; accepted 13 April 2010. published online 10 May 2010.

Abstract

Background

Many Electronic Health Record (EHR) systems fail to provide user-friendly interfaces due to the lack of systematic consideration of human-centered computing issues. Such interfaces can be improved to provide easy to use, easy to learn, and error-resistant EHR systems to the users.

Objective

To evaluate the usability of an EHR system and suggest areas of improvement in the user interface.

Methods

The user interface of the AHLTA (Armed Forces Health Longitudinal Technology Application) was analyzed using the Cognitive Task Analysis (CTA) method called GOMS (Goals, Operators, Methods, and Selection rules) and an associated technique called KLM (Keystroke Level Model). The GOMS method was used to evaluate the AHLTA user interface by classifying each step of a given task into Mental (Internal) or Physical (External) operators. This analysis was performed by two analysts independently and the inter-rater reliability was computed to verify the reliability of the GOMS method. Further evaluation was performed using KLM to estimate the execution time required to perform the given task through application of its standard set of operators.

Results

The results are based on the analysis of 14 prototypical tasks performed by AHLTA users. The results show that on average a user needs to go through 106 steps to complete a task. To perform all 14 tasks, they would spend about 22min (independent of system response time) for data entry, of which 11min are spent on more effortful mental operators. The inter-rater reliability analysis performed for all 14 tasks was 0.8 (kappa), indicating good reliability of the method.

Conclusion

This paper empirically reveals and identifies the following finding related to the performance of AHLTA: (1) large number of average total steps to complete common tasks, (2) high average execution time and (3) large percentage of mental operators. The user interface can be improved by reducing (a) the total number of steps and (b) the percentage of mental effort, required for the tasks.

Keywords: Electronic Health Records, Cognitive Task Analysis, Distributed cognition, UFuRT, GOMS, KLM