Randomized controlled trial of an automated problem list with improved sensitivity
Received 1 November 2006; received in revised form 10 December 2007; accepted 10 December 2007. published online 30 January 2009.
Abstract
Purpose
To improve the completeness and timeliness of an electronic problem list, we have developed a system using Natural Language Processing (NLP) to automatically extract potential medical problems from clinical, free-text documents; these problems are then proposed for inclusion in an electronic problem list management application.
Methods
A prospective randomized controlled evaluation of the Automatic Problem List (APL) system in an intensive care unit and in a cardiovascular surgery unit is reported here. A total of 247 patients were enrolled: 76 in an initial control phase and 171 in the randomized controlled trial that followed. During this latter phase, patients were randomly assigned to a control or an intervention group. All patients had their documents analyzed by the system, but the medical problems discovered were only proposed in the problem list for intervention patients. We measured the sensitivity, specificity, positive and negative predictive values, likelihood ratios and the timeliness of the problem lists.
Results
Our system significantly increased the sensitivity of the problem lists in the intensive care unit, from about 9% to 41%, and even 77% if problems automatically proposed but not acknowledged by users were also considered. Timeliness of addition of problems to the list was greatly improved, with a time between a problem's first mention in a clinical document and its addition to the problem list reduced from about 6 days to less than 2 days. No significant effect was observed in the cardiovascular surgery unit.
Department of Biomedical Informatics, University of Utah, School of Medicine, Salt Lake City, UT, United States
Corresponding author at: University of Utah, Department of Biomedical Informatics, 26 South 2000 East, HSEB Suite 5700, Salt Lake City, UT 84112-5750, United States. Tel.: +1 801 581 4080; fax: +1 801 581 4297.
ABSTRACT