Assessing Early Warning Score Performance
According to a new study published in the journal Resuscitation, vital sign measurements can be treated as if they are independent – multiple observations can be used from each episode of care – when comparing the performance and ranking of EWSs, provided no EWS includes age.
"Using multiple observations from each episode of care does not significantly change the ranking of EWSs compared to using only one observation from each episode, as long as no EWS includes age. This is in spite of observed dependence between vital signs observations collected during the same episode of care," say researchers from the University of Portsmouth, Portsmouth, UK.
EWS systems allocate points in a weighted manner, based on the derangement of a predetermined set of patient vital signs variables (eg, blood pressure, heart rate, temperature) from an arbitrarily agreed “normal” range. The points for each variable are summed and the total is used to inform a change in the patient’s vital sign monitoring schedule and/or trigger a call for expert help at the bedside.
The study analysed data from adult patients (aged 16 years or older) admitted to Portsmouth Hospitals NHS Trust on or after 25/05/2011 and discharged on or before 31/12/2012. The authors used a population of 1,564,143 consecutive vital signs observation sets collected as a routine part of patients’ care. Each vital signs measurement set contained: pulse rate, breathing rate, systolic and diastolic blood pressure, temperature, SpO2, the inspired gas (eg, oxygen or air) at the time of SpO2 measurement, and the patient’s conscious level. Conscious level was recorded as alert (A), responds to voice (V), responds to pain (P) or unresponsive (U).
The authors compared 35 published EWSs for their discrimination of the risk of death within 24 hours of an observation set using (1) all observations in the authors' dataset, (2) one observation per patient care episode, chosen at random, and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode.
The authors compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order.
Results and Discussion
In the study period, there were 64,285 episodes of care with admission on or after 25/05/2011 and discharge on or before 31/12/2012, where the patient was aged ≥16, the patient was not discharged alive on the day of admission and one or more observations were taken during the last 24 hours of the stay. Associated with these episodes of care were 1,395,941 observation sets (mean 21.7 observation sets per episode)
There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the UK National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24 hours. This supports the findings of earlier studies in which multiple observations were used from each episode of care.
AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS). EWSs that include age, the observation selection method can be important in determining the rank order of EWSs.
Overall, the findings of this research suggest that vital signs and derived EWS values for EWSs that do not include age can be treated as if they were independent (even though the interclass correlations demonstrate that there is within-episode dependence). Therefore use of multiple observation sets from a single episode in assessing the performance of EWS systems does not appear to bias the ranking of EWSs, as long as no EWS includes age.
Image Credit: UK Engineering and Physical Sciences Research Council
Published on : Fri, 27 Mar 2015
Print as PDF
The HAMILTON-C1 neo is a versatile neonatal ventilator that combines invasive and noninvasive modes with the additional options of nCPAP and high flow oxygen therapy. The integrated turbine allows it to be operated independently of a compressed air supply....
The HAMILTON-C3 ventilator is a modular high-end ventilation solution for all patient groups. Offering a number of unique features, the HAMILTON-C3 is one of the first ventilators featuring the “Ventilation Autopilot” INTELLiVENT-ASV®. The HAMILTON-C3’s...
NovaPort cannulas: The safe access to the vascular system. NovaPort® twin double lumen cannulas are specifically developed for veno-venous vascular access. NovaPort one single lumen cannulas are specifically developed for the percutaneous connection (KU)...
Patient Dedicated Arterial Blood Gas Analyzer Frequent measurement of arterial blood samples is an important component in the effective management of patients in the critical care environment, particularly those that are unstable. Point of care or...