ICU Management & Practice, Volume 19 - Issue 1, 2019

This article focuses on the clinical and practical application of current available cloud-based data analysis to benchmarking in real-time and to optimise clinical care in the ICU.



The ICU is a highly technological environment where each patient data generates thousands of data-points per day. However, most of this data is usually wasted thus missing the opportunity of using this data to understand patient profile and improve outcomes. For many years intensivists have used individual patient data to monitor and follow organ failure severity and trajectory with widely used scoring systems such as the SOFA score (Ferreira et al. 2001) and aggregate ICU data using information on physiology and patients characteristics to generate severity of illness and prognostic scores such as the SAPS and APACHE scores that do not add to the management of individual patient (Salluh and Soares 2014) but can be used to evaluate global severity of illness of a population and produce estimates of efficacy of the ICU through standardised mortality rates. In recent years the fast development of electronic medical records, interoperability, connectivity with medical devices, cloud-based systems and streaming analytics brought advanced information with near-real-time analysis to the bedside aiding clinicians to manage patients based on data and to manage ICUs and their quality and performance using descriptive analysis, advanced prediction models and strategic benchmarking tools. Although not fully implemented, it is clear that the current technology allows point of care assessment of key performance indicators and is the cornerstone of data-driven management.


You might also like:How to provide better intensive care?


Data-driven management in the ICU

What kind of data improves ICU performance and patient outcomes

There is currently a plethora of data in the ICU and exploring it in-depth can be a labour-intensive task. Currently, several institutions have devoted time and resources to data-science departments in order to generate models that can help evaluate their patient population, and use of resources and outcomes. It is clear that either older methods (i.e. logistic regression, data mining techniques) or newer ones (i.e. machine learning, deep learning, super-learner algorithms) are widely available and increasingly used. A high degree of expertise, as well as access to high-quality and highly granular data through robust interoperability, is essential (Rush et al. 2018; Komorowski et al. 2018; Gehrmann et al. 2018). However, there is also data that is simpler, easier to obtain and can generate rich and very useful insights for measuring and improving quality of care and patient outcomes. With the use of core data or a minimal dataset comprised of patient characteristics (diagnosis, comorbidities), complications within the first day of ICU admission (physiologic derangements, limited lab data, use of invasive devices) and ICU related resource use and complications, robust, reliable and rich information can be easily generated.


Currently national registries such as the NICE registry in the Netherlands are providing ways to use actionable indicators on antibiotics, pain and transfusion management fusing the audit (adherence to best practice) measured at each ICU with a system that generates ways to improve adherence to the best current evidence in the form of “tool-boxes” (Kallen et al. 2018; Lange et al, 2017). A large real-world data project from Epimed, a cloud-based analytics for quality measurement and ICU performance (Zampieri et al. 2017) is currently implemented in more than 800 ICUs in six countries where physicians at the point of care (through computers or mobile devices) can have access to real-time information on key quality metrics (Rhodes et al. 2012), risk-adjusted outcomes and adherence to prevention of adverse events and infection and use this information as target for quality improvement initiatives.


In addition, all this information can be compared, and ICU benchmarking has made substantial progress in recent years (Salluh et al. 2018). Traditionally, benchmarking has been divided into categories of process, performance, and strategic benchmarking. It also can be performed within the same institution or as external benchmarking. For ICUs, benchmarking should use standardised measurements to allow comparison of performance between intensive care units and if feasible involve risk-adjusted measures such as the standardised mortality ratios that allows ICUs to compare their mortality rates with other ICUs with different profiles in a fair way. Or it could also involve risk-adjusted length of stay or resource use as a way to measure its efficiency regardless of the ICU profile (Soares et al. 2015; Rothen et al. 2007). Although imperfect, severity-adjusted mortality rates should be used preferentially associated with processes of care and compliance as they can offer an alternative approach to benchmarking, providing actionable data (Table 1).


How I use data to manage my ICU


Despite the worldwide adoption of electronic health records (EHR), few institutions are making full use of its entire potential using highly granular and sequential data acquired in EHR to improve the quality of care. The EHR in the ICU is a powerful source of information generated by healthcare professionals and consolidates data from patient monitoring systems, bedside equipment (i.e. infusion pumps, dialysis machines), and other hospital IT solutions. The emergence of Big Data and data-driven healthcare presents both tremendous opportunities as well as unprecedented challenges (Sanchez-Pinto 2018; Pirrachio et al. 2018). Recent studies demonstrate the potential for the use of data to reducing health care costs while improving quality of care, through the development of clinical decision support of general ICU patients and sepsis cases (Vellido et al 2018; Pirracchio et al 2018), better risk assessment and clinical profiling of ICU patients through machine learning (Vranas et al 2017) as well as identifying actionable targets for improvement of process of care in QI initiatives and registries (Soares et al 2016).


Most organisations are now in the phase of identifying patients through clinical or financial risk profiles. Urgent action is needed to improve the recognition where it is possible to have the greatest outcome with the resources available. For this purpose, the use of currently available solutions that enable predictive data analytics at the bedside can help to identify specific at-risk populations and target those individuals to optimise clinical care or improve ICU staff profile and skills.


Real-world use of analytics to evaluate the performance of my ICU


The recent progress of EHR as well as advanced Patient Data Management System (PDMS) for the ICU and cloud-based analytics allows us to apply the 40-year-old Donabedian principles in near-real time. Basically, there are three types of indicators based on Donabedian categorisation (Donabedian 1978):


  1. Structure: describes the organisation, facilities, and staff. Usually describes aspects that can be improved by increasing investments.
  2. Process: describes the process of care between the caregiver and the patient. Usually depends on care aspects being more easier to change.
  3. Outcome: describes the outcome, frequently at the patient level. Ultimately, the indicator is more important.


The first step is to define the indicators that should be monitored. Second, guarantee the achievement and storage in a clinical database. Several examples of databases are available. Third, compare the results to other ICUs (Guidet et al. 2016).


After measurement and benchmarking, some domains will require changes. One important consideration is to evaluate the ICU results globally and for specific groups and conditions (e.g. sepsis, cardiac surgery, onco-haematological patients). Considering as an example, a mixed ICU with a poor performance for patients in mechanical ventilation (i.e. high rates of ventilator-associated pneumonia - VAP, delayed and long weaning periods, high rate of tracheotomy and longer than expected LOS) compared with other ICUs with the same characteristics. As an ICU director, the mission is to ensure the quality of data input and analyse the data. Once the problem is detected, a meeting with clinical champions in the unit must be done. The aim is to find actionable indicators and start change! It is important to share the information with the team to understand the root cause and create a plan to fix the problem. In this hypothetical case, the team concluded that high rates of VAP were related to VAP prevention bundle non-compliance, especially daily sedation vacation. As an action, a sedation protocol was developed, and its application was ensured by daily measurements with electronic checklists and a dedicated and well-trained multi-professional team. At the same time, non-invasive ventilation use could be revisited or a weaning protocol planned. A practical approach is presented in Figure 1.



Real time process of measured care

Both expert opinion and medical societies recommend that ICU LOS should be measured and compared as it represents a proxy of ICU effectiveness. Because it is easy to measure and reproducible, it is considered a good marker of resource use and is employed in a risk-adjusted way to obtain the efficiency matrices for ICU as depicted in Figure 2 (Rothen et al. 2007). Although ICU physicians are good at predicting mortality, even experienced intensivists are unable to accurately predict LOS at admission for both those patients who will experience short and long LOS (Nassar and Caruso 2016). In the last years, several models for ICU LOS prediction have been reported. However, a recently published systematic review found that none of those models completely satisfy requirements for planning, identifying unexpected long ICU LOS, or for benchmarking purposes. The authors recommended that physicians using these models to predict ICU LOS should interpret them with caution and use them for benchmarking, but not for individual patient assessment (Verburg et al. 2017).



One important caveat about these models is that the time period between when the sample is taken, and results are generated must be shortened considerably to be used in predictive equations (Zimmerman and Kramer 2017). Recently, the Epimed Monitor System® embedded a tool that collects data on LOS and provides clinical guidance for future admissions (Figure 3). For each diagnostic category and from demographic information, a LOS estimate is calculated by the system in the first 24 hours from admission; however, instead of simply reporting the predicted LOS (which create a bias by “pressuring” the staff to discharge the patient from the unit), the algorithm indicates whether landmark LOS have passed (in percentiles) and provides an individualised risk of prolonged LOS (Zampieri et al. 2017). Using this management tool is possible to share the information early and in real time with all the staff involved in the care of the patient, set specific goals and predict ICU occupancy rate for the few next days.



A pragmatic model of this application tool is depicted in Figure 4. As an example, a 49y man was admitted in ICU and presented the diagnostic of community-acquired pneumonia. He had a history of a class II NYHA cardiac failure, hypertension, and chronic atrial fibrillation. During the first 24hs of ICU admission, the patient developed septic shock and respiratory failure being supported with vasopressors and invasive mechanical ventilation. The software predicts a median LOS of 8 days with a risk of prolonged LOS between 33-66%. This information is shared with the team during multidisciplinary clinical rounds, and each professional optimises relevant measures to improve the outcome of this patient (i.e. respiratory therapist focus on weaning process and early mobilisation, nurses in delirium prevention and family communication, a clinical pharmacist in medication reconciliation). As a manager, it is possible to use this information to negotiate with healthcare insurance.





Data-driven management applied to ICU allows not only an evaluation of ICU performance but has other conveniences including implementation and monitoring of clinical protocols, optimisation of patient flow, and better planning and transition of care and discharge.



APACHE Acute Physiology and Chronic Health Evaluation

EHR Electronics Health Recorder

ICU Intensive Care Unit

LOS Length of Stay

NICE Netherlands Intensive Care Evaluation

NYHA New York Heart Association

SAPS Simplified Acute Physiology Score

SOFA Sequential Organ Failure Assessment

VAP Ventilator Associate Pneumonia


Conflict of interest

Dr. da Silva Ramos reports no conflicts of interest. Dr. Salluh is founder and shareholder at Epimed Solutions®, the provider of a cloud-based healthcare analytics and performance evaluation software.


Donabedian A (1978) The Quality of Medical Care. Science, 200:856-864.


Ferreira FL, Bota DP, Bross A (2001) Serial Evaluation of the SOFA Score to Predict Outcome in Critically ill Patients. JAMA, 286(14):1754-8.


Gehrmann S, Demoncourt F, Li Y et al. (2018) Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PLoS One, 13(2):e0192360.


Guidet B and Nguyen Y (2016) Benchmarking. In : Guidet B, Valentin A, Flaatten H, eds. ICU Quality Management in Intensive Care A Practical Guide. Cambridge University Press, pp 212-220.


Kallen MC, Roos-Blom MJ, Dongelmans DA et al. (2018) Development of actionable quality indicators and an action implementation toolbox for appropriate antibiotic use at intensive care units: a modified-RANS Delphi study. PLoS One, 13(11):e0207991.


Komorowski M, Celi LA, Badawi O et al. (2018) The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med, 24(11):1716-1720.


Lange DW, Dongelmans DA, Keizer NF et al. (2017) Small steps beyond benchmarking. Rev Bras Ter Intensiva, 29(2):128-130.


Nassar Jr AP and Caruso P (2016) ICU Physicians are unable to accurately predict length of stay at admission: a prospective study. Int J Qual Health Care, 28(1):99-103.


Pirracchio R, Cohen MJ, Malenica I, et al. (2018) Big data and targeted machine learning in action to assist medical decision in the ICU. Anaesth Crit Care Pain Med, pii: S2352-5568(18)30216-9.


Rhodes A, Moreno RP, Azoulay E, et al. (2012) Prospectively defined indicators to improve the safety and quality of care for critically ill patients: a report from Task Force on Safety and Quality of the European Society of Intensive Care Medicine (ESICM). Intensive Care Med, 38(4):598-605.


Rothen HU, Stricker K, Einfalt J et al. (2007) Variability in outcome and resource use in intensive care units. Intensive Care Med, 33(8): 1329-1336.


Rush B, Celi LA, Stone DJ (2018) Applying machine learning to continuously monitored physiological data. J Clin Monit Comput, 2018 Nov 11. doi: 10.1007/s10877-018-0219-z;


Salluh JIF, Soares M (2014) ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care, 20(5):557-65.


Salluh J, Chiche JD, Reis CE et al. (2018) New perspectives to improve critical care benchmarking. Ann Intensive Care,8(1):17.


Salluh JIF, Soares M, Keegan MT (2017) Understanding intensive care unit benchmarking. Intensive Care Med, 43(11):1703-1707.


Sanchez-Pinto LN, Luo Y, Churpek MM (2018) Big Data and Data Science in Critical Care. Chest, 154(5):1239-1248.


Soares M, Bozza FA, Angus DC, et al. (2015) Organizational characteristics, outcomes, and resource use in 78 Brazilian intensive care units: the ORCHESTRA study. Intensive Care Med, 41(12):2149-60.


Vellido A, Ribas V, Morales C, et al. (2018) Machine learning in critical care: state-of-the-art and a sepsis case study. Biomed Eng Online,17(Suppl 1):135.


Verburg IWM, Atashi A, Eslami S, et al. (2017) Which Models Can I Use to Predict Adult ICU Lenght of Saty? A Systematic Review. Crit Care Med, 45:e222-e231.


Vranas KC, Jopling JK, Sweeney TE, et al. (2017) Identifying Distinct Subgroups of ICU Patients: A Machine Learning Approach. Crit Care Med, 45(10):1607-1615.


Zampieri FG, Soares M, Borges LP, et al. (2017) The Epimed Monitor ICU Database®: a cloud-based national registry for adult intensive care unit patients in Brazil. Rev Bras Ter Intensiva, 29(4):418-426.


Zimmerman JE and Kramer AA (2014) A history of outcome prediction in the ICU. Curr Opin Crit Care, 20:550-556.