ICU-Customised Lab Value Ranges Reduce False Positive Alerts
Novel EMR interfaces using adjusted reference ranges customised for specific patient populations may ameliorate this problem. In a study published by PLoS One, researchers sought to compare the accuracy of abnormal laboratory value indicators between two EMR systems (novel vs. traditional) using as a reference standard the judgment of board certified critical care physicians about which data they would like to be alerted given patients’ particular clinical situations.
The study was conducted at a tertiary care academic medical centre. Laboratory data from patients consecutively admitted to any of four ICUs (medical, surgical, mixed and trauma) during a two-day period were recorded. For each patient, researchers simultaneously took screenshots of the two EMR interfaces’ laboratory sections at a random time point during the day of admission. A document was created for each participant containing all available lab results and the patient's problem list, omitting information on "interface-origin(s)" and "abnormal indicators" of either interface.
The patient documents were sent to two mutually blinded critical care experts, who marked the values about which they would like to be alerted. All disagreements were resolved by an independent super-reviewer. Based on this gold standard, the investigators calculated and compared the sensitivity, specificity, positive and negative predictive values (PPV, NPV) of customised versus traditional abnormal value indicators.
Patient demographics including age, gender, admission diagnosis, severity of illness (APACHE III), ICU/hospital length of stay and mortality along with mechanical ventilation status were queried from a previously validated ICU database (Multidisciplinary Epidemiology and Translational Research in Intensive Care [METRIC] ICU Datamart).
Descriptive patient data were summarised as median and inter-quartile ranges (IQR) or percent (number) as appropriate. For each EMR interface, the authors calculated sensitivity, specificity and predictive values of abnormal value indicators as compared to the gold standard. For each measure of accuracy, the authors determined exact binomial confidence intervals and used two-sample proportion tests to compare them across the two EMR interfaces.
Results and Discussion
The study included 37 patients with a total of 1,341 laboratory values reported in the traditional EMR interface, of which 754 values (56 percent) were also displayed in the novel interface. The percentage of laboratory values indicated as abnormal was 42 percent (559/1341) in the traditional interface compared to 26 percent (195/754) in the novel interface.
Experts had fair agreement (kappa = 0.39) classifying 71 of all 1,341 (5.3 percent) values as abnormal. One of these 71 (1.4 percent) “truly abnormal” values was not displayed in the novel interface (serum Hydroxybutyrate). Compared to the traditional interface, the novel interface had a similar sensitivity (77 percent vs 85 percent, P = 0.22) and negative predictive value (97.1 percent vs 98.6 percent, P = 0.06) but had a higher specificity ((79 percent vs 61 percent, P<0.001) and positive predictive value (8 percent vs 11 percent, P<0.001).
The novel interface selectively omitted almost half of all available laboratory data and amongst all displayed values marked 38 percent less values as abnormal compared to the traditional interface. The main effect was a substantial reduction in the number of values indicated as abnormal by the interface but considered normal by critical care experts (false positive alerts), which was reflected by the higher specificity and positive predictive value of the novel interface.
The finding that the standard reference ranges in the traditional interface produce a large fraction of false positive notifications (37 percent) is consistent with reports from other studies evaluating alerts in ICU settings more generally.
Reference ranges for laboratory values adjusted towards a population of critically ill patients based on expert consensus decreased false positive alerts (increased specificity and positive predictive value) without affecting the fraction of false negative alerts (unchanged sensitivity and negative predictive value). This may reduce alert fatigue among healthcare providers.
Substantial disagreement among critical care experts about which laboratory values should be indicated as abnormal is a major limitation for both developing and testing such customised reference ranges.
Image Credit: Wikipedia
Published on : Mon, 29 Sep 2014
Print as PDF
FeaturesSV 300 is a state-of-the-art ventilator that’s simple to configure, easy to operate and versatile in use. It equips with extensive ventilation modes that can treats pediatric and adult patients with all acuity levels at ICUs and Intermediate Care.With...
Always in sight, always in mind Features Mindray believes the best way to predict the future is to create it. The revolutionary BeneVision N22/N19 is designed to optimize user experience by satisfying all your clinical demands. With visionary-stimulating...
FeaturesMindray BeneVision Central Monitoring System is a powerful and scalable solution providing for continuous, real-time surveillance across networks large and small. The system can display patient information from networked monitors, wireless transport...
The HAMILTON-C6 represents a new Generation of high-end ventilators. The combination of modularity, ease of use, mobility, and advanced features allows you to individualize your patient's ventilation therapy: - State-of-the-art ventilation modes for adult,...