Functionality test for drug safety alerting in computerized physician order entry systems
Received 3 March 2009; received in revised form 12 January 2010; accepted 12 January 2010. published online 10 February 2010.
Abstract
Purpose
To evaluate the functionality of drug safety alerting in hospital computerized physician order entry (CPOE) systems by a newly developed comprehensive test.
Methods
Comparative evaluation of drug safety alerting quality in 6 different CPOEs used in Dutch hospitals, by means of 29 test items for sensitivity and 19 for specificity in offices of CPOE system vendors. Sensitivity and specificity were calculated for the complete test, and for the categories “within-order checks”, “patient-specific checks”, and “checks related to laboratory data and new patient conditions”. Qualitative interviews with 16 hospital pharmacists evaluating missing functionality and corresponding pharmacy checks.
Results
Sensitivity ranged from 0.38 to 0.79 and specificity from 0.11 to 0.84. The systems achieved the same ranking for sensitivity as for specificity. Within-order checks and patient-specific checks were present in all systems; alert generation or suppression due to laboratory data and new patient conditions was largely absent. Hospital pharmacists unanimously rated checks on contra-indications (absent in 2 CPOEs) and dose regimens less than once a day (absent in 4 CPOEs) as important. Pharmacists’ opinions were more divergent for other test items. A variety of pharmacy checks were used, and clinical rules developed, to address missing functionality.
Conclusions
Our test revealed widely varying functionality and appeared to be highly discriminative. Basic clinical decision support was partly absent in two CPOEs. Hospital pharmacists did not rate all test items as important and tried to accommodate the lacking functionality by performing additional checks and developing clinical rules.
aDepartment of Hospital Pharmacy, Erasmus University Medical Centre, Rotterdam, The Netherlands
bDepartment of Internal Medicine, Erasmus University Medical Centre, Rotterdam, The Netherlands
cInstitute of Health Policy and Management, Erasmus University, Rotterdam, The Netherlands
Corresponding author at: Department of Hospital Pharmacy, Erasmus University Medical Centre, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands. Tel.: +31 10 703 3202; fax: +31 10 703 2400.
ABSTRACT