ICU Management & Practice, Volume 19 - Issue 4, Winter 2019/2020

An AI-enabled ICU is coming in the not-too-distant future, but it requires strong partnerships between clinicians and engineers.

Spoiler alert. The short answer to this question is yes! 

Artificial Intelligence in Medicine” has been taking place biannually for the past 28 years (Patel et al. 2009).

What is new, however, is the cross-industry exponential growth in awareness of, and interest in, AI over the past decade. In addition to the stimulus provided by our ability to generate, gather, organise, store and access enormous amounts of digital data, the growth of AI in medicine has been facilitated by three major developments:

  1. The proliferation of electronic medical records (EMRs) is the most obvious manifestation of the use of AI in medicine. Although EMR adoption is visible, by far the largest growth in the healthcare field is occurring in the realm of digital imaging and genomic sequencing. The wealth of data available has driven a need for innovation in the analytics space, while simultaneously fueling AI development which is highly dependent on the availability of large quantities of training data to produce reliable algorithms.
  2. Advanced analytic methods demand significant computational resources. Increasing standalone computer power combined with the availability of state of the art cloud computing services from providers such as Google and Amazon puts the necessary computational resources to get started in AI within reach of anyone who is interested. The impact of this has been felt most obviously in the consumer space but in medicine, this resource is increasingly being applied to the enrichment and analysis of the glut of medical data flowing from #1.
  3. Data transmission methods using mobile technologies such as 5G, smartphones and consumer wearables are advancing rapidly. These technologies enable in situ data capture/analytics, data sharing, knowledge delivery, synchronous and asynchronous communication and extended reality interactions with profound implications for traditional healthcare delivery models.
However, because of patient privacy issues, healthcare presents significant barriers to entry for those outside the health system firewall. Those driving innovation in the three areas outlined above have mostly remained outside of healthcare. Because of the firewall, AI development has started as a cottage industry run largely under the direct or close supervision of the healthcare stakeholders that collect and store the data. Efforts through this approach have, to date, produced little in the way of meaningful impact on patient outcomes. For example, despite an explosion of AI-related academic output, a recently published systematic review shows “no performance benefit of machine learning over logistic regression for clinical prediction models” (Christodoulou et al. 2019).

Things are about to change. At this time we are witnessing the beginning of a revolution in healthcare AI. The rise of interest in healthcare from non-traditional stakeholders is palpable. Silicon Valley big technology companies (Google, Apple), hardware manufacturers (Philips, GE, Siemens), integration/consulting firms (Deloitte, Lockheed Martin, Leidos), employers (Amazon, Walmart), venture capital executives, and a myriad of experts in the financial and intelligence communities looking for new business opportunities are determined to enter the field and will drive innovation in the areas of advanced data analytic techniques and AI development. The evidence that the interaction between Big Tech and healthcare is happening now is all around us. At the beginning of 2019, it was reported that nearly 80% of healthcare executives said their organisations are exploring and investing in big data analytics and AI (

Despite the investment, there are important reasons why we should continue to be cautious about the claims made for AI in healthcare in general, and the ICU in particular.

  1. Garbage in-garbage out: Data used for training AI do not provide a perfect representation of the patient and rarely contain mechanistic insights into disease or health. Data are generated as a side effect of caring for patients or for payers’ purposes. Diagnostic tests such as laboratory analyses for example, are ordered because of a clinical suspicion of some problem, to help the clinician resolve diagnostic uncertainty, or to monitor the impact of a treatment decision. In this situation, associations are easy to identify but causality is elusive and rarely “discoverable” within the data. This leads to a fundamental problem for this first generation of data scientists engaging in healthcare AI development – mechanistic understanding of critical illness takes time to acquire. AI models developed without mechanistic understanding embedded into them, will fail to breach the threshold of usefulness for a clinician.
  2. Inconsistent evaluation and validation and absence of clinical trials: The first generation of AI algorithms mostly fall into the category of “developed and validated on MIMIC II” or some other flavour of publicly available data. The area under the receiver operator characteristic curve, true and false positive rates, sensitivity and specificity are often reported. Thus far, follow-on prospective evaluation and validation of the safety and performance of these AI algorithms in real world operating conditions are largely absent. Clinical trials have not taken place and regulation is dismissed as stifling of innovation. All other diagnostic tests, devices and therapeutic interventions follow a relatively standard evaluation and regulation pathway. For AI to be embraced, it will also have to demonstrate real world operational safety, reliability, and efficacy.
  3. Implementation of science and stakeholder engagement: We work within complex adaptive systems that have evolved over generations to care for critically ill patients. What we have in place in the ICU now is a collection of people, processes and technology that largely serves our patient population well. Lack of stakeholder engagement and a limited understanding of the socio-technical environment into which AI will be implemented severely limit the impact and sustainability of AI. If we fail to engage the stakeholders in a discussion about the risks and benefits of these disruptive technologies, we could cause widespread unintended harm and leave our patients worse off than they are in the current system.
  4. Alert fatigue, information overload and burnout: With data accessibility, multiple alerts, reminders or scoring systems may be easily produced and deployed rapidly. Instead of minimising cognitive burden, however, there is more demand on bedside providers to respond to this information. The jump from “no data” to “all data” places an additional burden on clinicians. The development of user-friendly interfaces and rigorous testing are required to minimise alert fatigue before deploying these tools to clinical practice.
  5. Privacy and trust: There is a growing suspicion surrounding big tech companies and the monetisation of personal data. Leaks, narrowly focused CEOs, security breaches, misuse of data, a culture of over promise/under delivery (anyone remember Theranos?) undermine public trust, and make new partnerships between health care organisations and AI innovators challenging. Technology companies need to cede control to healthcare providers if the full potential of partnership is to be realised.

In 2012 we published an article “The hospital of the future - building intelligent environments to facilitate safe and effective acute care delivery.” This described an alignment of people, processes, technology and incentives to serve the interests of the patient (Pickering et al. 2012). We would like to revisit some of technologies in an attempt to demonstrate how we might harness the developments in AI for the benefit of patients and providers while avoiding some of the potential harms. Our prediction for the near future is that three AI-based ICU tools might be transformational:

Control Tower Platform

The modern EMR adds to information overload by overwhelming EMR “inboxes” and generating unnecessary alerts (nytimes.ccom/2019/11/01/health/epic-electronic-health-records.html). Clinical Control Tower is a newly-developed central alert-screening and implementation system developed at Mayo Clinic. The concept behind Clinical Control Tower is to serve as a centralised non-life-threatening alert and prediction “cockpit.” This unified screening system is managed by a designated capsule communicator or “CapCom,” analogous to the US National Aeronautics and Space Administration ground-based astronaut who maintains contact with astronauts during space missions. The CapCom in the healthcare context is the clinician responsible for screening incoming alerts and notifications. As no alerts have 100% accuracy it is essential to perform initial validation of notifications before activating specific workflows with bedside providers.

When the CapCom decides that an alert is valid, he or she communicates “down to the ground” to a bedside clinician and guides them through necessary and recommended tasks. Each step may be captured electronically in the control tower application. Workflow and actions are captured and analysed using a feedback loop tool. Deviations from intended care processes may be identified. Control Tower is a tool designed to minimise errors and information overload in hospital practice (Figure 1).

Computer Vision

Platforms such as Control Tower will help deal with data management and representation, but will not change the fact that a significant portion of a clinician’s time is spent on data entry to computers.

Computer vision is an area of AI development with a goal of enabling computers to gain high-level understanding from videos or digital images. Image reasoning and computer vision may be applied to healthcare environments to enhance diagnostic processes and optimise and automatise workflows. But computer vision alone will not be able solve challenging clinical scenarios. For example, computer vision cannot distinguish anaesthetised patients from patients who are simply sleeping. Adding information from the environment (patient location, time of day) and EMR (medications given, orders) could augment camera data and elevate such systems to powerful clinical and workflow tools. The possibility for automatisation is truly enormous (Figure 2).

Voice Recognition

The efficiency of human-computer interaction is greatly enhanced by high-performing voice recognition software. Chatbots and voice-activated computer interfaces (e.g. Alexa, Siri) are increasingly prevalent and increasingly reliable in everyday life. Such developments have not, as yet, been widely embraced in healthcare, but one can envision a future in which AI responds to physician or nurse voice command to change the rate of an infusion pump, order a medication or test, answer a clinical question or provide a diagnosis or prognosis.

Artificial intelligence will play a significant role in the ICU of the future not as a standalone tool, but as part of a smart ambient environment (Dybowski et al. 1996; Keegan et al. 2011; Fauw et al. 2018; Nemati et al. 2018; Parreco et al. 2018).

To be able to develop such tools, researchers require access to new widely available databases of clinical and non-clinical information. Connecting EMR data with clinically meaningful labels will help produce clinical tools that are based on causality. Augmenting EMR data with environmental and non-clinical data will enable researchers to build algorithms for public health and pre-hospital care.

An AI-enabled ICU is coming in the not-too-distant future, but it requires strong partnerships between clinicians and engineers.

Key Points

  • The growth of AI in medicine has been facilitated by three major developments: electronic medical records, cloud computing services, and mobile technologies.
  • Because of patient privacy issues, healthcare presents significant barriers to entry for those outside the health system firewall.
  • We are witnessing the beginning of a revolution in healthcare AI: nearly 80% of healthcare executives said their organisations are exploring and investing in big data analytics and AI.
  • For the near future, three AI-based ICU tools might be transformational: control tower platform, computer vision and voice recognition.


Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B (2019) A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol, 110:12-22.

Dybowski R, Gant V, Weller P, Chang R (1996) Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm. Lancet, 347.

Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med, 24.

Keegan MT, Gajic O, Afessa B (2011) Severity of illness scoring systems in the intensive care unit. Crit Care Med, 39:163-9.

Moor J (2006) The Dartmouth College Artificial Intelligence Conference: The Next Fifty Years. AI Magazine, 27(4):87.

Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG (2018) An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU. Crit Care Med, 46.

Parreco J, Hidalgo A, Parks JJ, Kozol R, Rattan R (2018) Using artificial intelligence to predict prolonged mechanical ventilation and tracheostomy placement. J Surg Res, 228.

Patel VL, Shortliffe EH, Stefanelli M, Szolovits P, Berthold MR, Bellazzi R, Abu-Hanna A (2009) The coming of age of artificial intelligence in medicine. ArtifIntell Med, 46(1):5-17.

Pickering BW, Litell JM, Herasevich V, Gajic O (2012) Clinical review: the hospital of the future - building intelligent environments to facilitate safe and effective acute care delivery. Crit Care, 16(2):220.