Sepsis is a critical healthcare problem, associated with high mortality rates, and considerable financial and resource burden. At the end of 2021, new Surviving Sepsis Campaign international guidelines were published. Although updated guidelines have incremental updates bringing some new insights into geographic and gender diversity, haemodynamic management, early administration of steroids, antimicrobial choice, and post-ICU care, there are no real game-changers for either the diagnosis or treatment of sepsis and septic shock (Evans et al. 2021). Timely recognition of sepsis is the foundation of improved patient survival. There are opportunities to improve this in multiple areas.
Public Awareness and Pre-Hospital Recognition of Sepsis
Knowledge about sepsis has increased but remains lower than awareness of other more lethal conditions such as acute myocardial infarction or stroke (Jabaley et al. 2018). Members of the public may know the word “sepsis” and some of the signs but they are not always aware of the devastating consequences of organ failure if prompt medical intervention is lacking. Unlike, cardiac arrest or stroke, the early signs of severe sepsis are relatively non-specific and may be of gradual onset making it difficult to educate the public about triggers for medical intervention.
In-hospital Recognition and Treatment of Sepsis
Once the patient enters a health system, we have more tools at our disposal but still encounter delays or failure of diagnosis and treatment with significant negative impact on patient outcomes, health costs and resource utilisation.
In the last 15 years, there have been many efforts to use Electronic Medical Records (EMR) data for the early recognition of sepsis. Multiple systematic reviews on sepsis surveillance showed no breakthrough progress on that for the past 15 years in hospitalised patients (Alberton et al. 2017) and emergency departments (Hwang et al. 2020). One early “sepsis sniffer” demonstrated sensitivity of 48% and specificity of 86%, and a positive predictive value of 32% (Herasevich et al. 2008) and was based on established sepsis definition criteria. Later tuning of the algorithm improved performance with sensitivity of 80% and a specificity of 96% when applied to the validation cohort (Harrison et al. 2015). In parallel, Artificial Intelligence (AI) and Machine Learning (ML) in medicine have resulted in a new generation of AI-based sepsis detection/prediction models. Despite their potential, the prediction performance of those models is suboptimal in the real world compared to the controlled development environment, where they do about as well as traditional rule-based system using Systemic Inflammatory Response Syndrome (SIRS) criteria. For example, a prospective study of a sepsis detection algorithm utilising 160 clinical features relevant to sepsis achieved a sensitivity 65% and specificity 88% (Yuan et al. 2020). Recent prospective validation of a widely implemented sepsis prediction model from a commercial EMR vendor, similarly demonstrated poor performance. In this example, the system failed to identify 67% of patients with sepsis while simultaneously generating an alert for 18% of all hospitalised patients (Wong et al. 2021) undermining the primary intent of alerting, which is to call attention to a developing situation and to drive a behaviour or response that averts or mitigates the negative impact of that event. However, even in situations where machine learning algorithms demonstrate good predictive performance, clinician response to alerting is limited and results in minimal changes of patient care and outcomes (Giannini et al. 2019).
What can we learn from more than a decade of active sepsis surveillance research?
- EMR Data: The way we capture and store clinical data in electronic form through the EMR is not ideally suited to the development of predictive models. Infrequent and delayed data capture or charting presents a major barrier to the performance of real time predictive models. Most physiological charting does not occur in real time. This is particularly true during busy times when patient care takes precedent over charting. Unfortunately, this is precisely the time when automated detection of sepsis may be most useful and is a major contributor to poor performance and impact of alerts in the working environment.
- Continuous reliable automated data capture is not common: Time is of the essence in sepsis. Given the broad differential in the early stages of disease, it is essential that predictive models reliably identify the point where sepsis is highly probable as early as possible. For this, a model needs data at a frequency greater than commonly captured with usual workflows. Hospital based nurse observation data is not often captured at high frequency - every 4-6 hours would not be unusual in the average general care area. When combined with inconsistent data charting practices these organisational factors can lead to missing or very delayed data with significant downstream delays impeding the performance of model based sepsis alerts. These constraints are not accounted for by models developed on static retrospective datasets.
- Pre-test probability: Compounding all of this is the fact that most predictive models are trained to recognise sepsis using diagnostic labels found in the EMR that have been generated by clinical staff. This means that most sepsis algorithms depend on data gathered when sepsis is suspected by clinical staff. An excellent example of how this impact model performance is lactate measurement. Lactate is measured most often when sepsis is suspected by a clinician. This has two implications for performance in the setting of a patient with sepsis. In the first, the clinician suspects sepsis, orders lactate and the sepsis alert triggers, calling the attention of the clinician who already suspects sepsis to the fact that sepsis may be present - annoying and distracting; in the second, the clinician does not suspect sepsis, does not order lactate and the alert doesn’t trigger – the clinician and algorithm are both blissfully unaware of the fact that the patient has sepsis until some other events intercede. These model development choices dramatically decrease the impact of alerts on clinical outcome.
- Testing and reporting the usefulness of an alert: Most sepsis alerts are tested for diagnostic performance (sensitivity, specificity etc.) but almost never for action following an alert. Rarely are the impact on diagnostic performance or treatment choices reported. This obscures the essential value of the alert and underrepresents failures in essential follow-up tested for diagnostic performance (sensitivity, specificity etc.) but almost never for action following an alert. Rarely are the impact on diagnostic performance or treatment choices reported. This obscures the essential value of the alert and underrepresents failures in essential follow-up actions that can lead to clinically meaningful delays in recognition and intervention.
- Prediction lead time and risk-benefit analysis: Every diagnostic or treatment intervention for sepsis carries a risk or cost. Prediction is valuable only if the risk of early action outweighs the risk of delay while waiting for certainty. The inflection point for such risk-benefit analysis is very poorly understood for predictive models of sepsis. If this is layered on top of the other contributors to suboptimal performance of sepsis alerts outlined above, the probability that such alerts will generate unnecessary interventions or be ignored is very high.
As we discover all the ways not to build and deploy sepsis sniffers, are there approaches or combinations of approaches we can take to deliver truly useful alerts?
- Failure to rescue alerts: An approach that has been pioneered in security systems and more recently applied to clinical sepsis is the failure to rescue model of alerting. In this approach alerts are considered meaningful only if they prompt useful actions that otherwise would be missed or delayed. For this to be effective, the alerting platform monitors both system state and system processes. Alerts are issued only when there is a mismatch between the state and expected process. In the case of sepsis, this would correspond with an alerting platform that detects the condition of sepsis, confirms it with a clinician and then monitors for expected events or actions such as timely administration of antimicrobials. Notifications are only generated when expected actions are not detected. This acts as a guardrail for the quality of care delivered once the condition of sepsis is confirmed to be present.
- Ambient data cues: Clinicians utilise all their senses when evaluating the patient. Many of the visual and tactile cues clinicians use to make decisions are not recorded in the EMR and are not readily available to analytic models. Advances in computer vision could be applied to this problem. Computer vision models could be trained to monitor patients 24/7 and recognise important visual clinical cues as they emerge. Applying machine learning algorithms on such data may be able to achieve recognition of emerging sepsis performance similar to that of an expert clinician. This approach could be extended to other sensors such as accelerometers or sound.
- Physiological waveforms: As continuous physiologic monitoring becomes more common outside of traditional high acuity environment such as the ICU, the potential for high frequency data capture increases. This can potentially reduce dependence on EMR charted data and address some of the sepsis detection algorithm performance issues associated with delayed data availability. The application of machine learning to the problem of false alerts and noise is a particularly fertile area for improvement that may accelerate their application to sepsis monitoring.
- Alert delivery: Using real time locations services and accelerometer data, information on computer network activities and other augmented data from providers could be used to deliver alerts directly to the right person in the right location and position to act. The acceptance of such an approach to providers is unclear and merits further study.
- Control tower: There is no doubt, that sepsis screening tools should be implemented systematically and hospital-wide to improve surveillance and treatment of sepsis for all patients. With the recent interest in telepresence, there is an opportunity to integrate sepsis surveillance and management into clinical control towers under centralised supervision. This will help to mitigate some of the challenges associated with implementation and training of large numbers of staff in new processes of care.
Conflict of Interest
Mayo Clinic and Dr Herasevich hold the USA patent on sepsis sniffer technology - sepsis monitoring and control (US 8527449B2). Technology licensed to Ambient Clinical Analytics - https://ambientclinical.com.