Functionality test for drug safety alerting in computerized physician order entry systems

Abstract 

Purpose

To evaluate the functionality of drug safety alerting in hospital computerized physician order entry (CPOE) systems by a newly developed comprehensive test.

Methods

Comparative evaluation of drug safety alerting quality in 6 different CPOEs used in Dutch hospitals, by means of 29 test items for sensitivity and 19 for specificity in offices of CPOE system vendors. Sensitivity and specificity were calculated for the complete test, and for the categories “within-order checks”, “patient-specific checks”, and “checks related to laboratory data and new patient conditions”. Qualitative interviews with 16 hospital pharmacists evaluating missing functionality and corresponding pharmacy checks.

Results

Sensitivity ranged from 0.38 to 0.79 and specificity from 0.11 to 0.84. The systems achieved the same ranking for sensitivity as for specificity. Within-order checks and patient-specific checks were present in all systems; alert generation or suppression due to laboratory data and new patient conditions was largely absent. Hospital pharmacists unanimously rated checks on contra-indications (absent in 2 CPOEs) and dose regimens less than once a day (absent in 4 CPOEs) as important. Pharmacists’ opinions were more divergent for other test items. A variety of pharmacy checks were used, and clinical rules developed, to address missing functionality.

Conclusions

Our test revealed widely varying functionality and appeared to be highly discriminative. Basic clinical decision support was partly absent in two CPOEs. Hospital pharmacists did not rate all test items as important and tried to accommodate the lacking functionality by performing additional checks and developing clinical rules.

Keywords: Medical order entry systems, Clinical decision support systems, Drug therapy computer-assisted, Safety management, Sensitivity and specificity