Improved SEP-1 Compliance Using AI-Supported Tools

Artificial intelligence (AI) plays an evolving role in optimising sepsis management and SEP-1 compliance. Despite these advances, barriers persist, including alert fatigue, disparities in model performance across diverse settings, and regulatory hurdles in AI deployment. By integrating predictive models with structured team-based interventions, healthcare institutions can improve early recognition, enhance sepsis bundle adherence, and ultimately improve sepsis-related outcomes.

Introduction

Sepsis remains a major global healthcare challenge, leading to significant morbidity, mortality, and economic burden. Defined as a life-threatening organ dysfunction caused by a dysregulated host response to infection, sepsis accounts for nearly 20% of all global deaths (Singer et al. 2016). Despite advances in critical care, standardised protocols, and bundled care approaches such as the Centers for Medicare & Medicaid Services (CMS) SEP-1 measures, sepsis related outcomes remain suboptimal due to variability in adherence, challenges in early recognition, and delays in timely intervention (Gesten and Evans 2021).

The SEP-1 performance metric was introduced by CMS in 2015 as part of a national effort to improve sepsis care. It mandates a structured, protocol-driven approach requiring timely administration of antibiotics, lactate measurement, fluid resuscitation, and haemodynamic monitoring (Quality Measures Fact Sheet BPCI Advanced and Quality Background on Severe Sepsis and Septic Shock n.d.). However, despite its widespread adoption, SEP-1 remains a topic of debate, with studies showing mixed results regarding its impact on patient outcomes (Gesten and Evans 2021). Some reports suggest improved compliance with SEP-1 is associated with lower mortality, while others indicate that rigid adherence to the measure may not translate into meaningful clinical improvements. Additionally, implementing SEP-1 presents logistical and operational challenges, including the need for multidisciplinary coordination, extensive documentation, and data abstraction, which can divert resources away from direct patient care.

In recent years, artificial intelligence (AI) and machine learning (ML) have emerged as promising tools to enhance sepsis detection and management. AI-driven algorithms can analyse vast amounts of clinical data to identify early signs of sepsis, predict deterioration, and guide individualised treatment strategies (Santacroce et al. 2024). AI has the potential to automate sepsis screening which can provide clinicians with information to optimise antibiotic stewardship and reduce unnecessary interventions, thus complementing traditional sepsis management frameworks. However, integrating AI into clinical workflows remains a challenge, requiring robust validation, interpretability, and alignment with existing quality measures such as SEP-1.

Studies have demonstrated that structured multidisciplinary approaches, such as Sepsis and Shock Response Teams (SSRTs), significantly improve adherence to sepsis bundles and reduce mortality (Grek et al. 2017). Effective sepsis care requires coordination among emergency physicians, intensivists, infectious disease specialists, pharmacists, and nursing staff to ensure timely recognition and intervention. However, achieving multidisciplinary alignment remains a persistent barrier, with factors such as inconsistent sepsis definitions, variable institutional protocols, and competing clinical priorities affecting consistent implementation and/or utilisation.

This review explores the evolving landscape of sepsis management, focusing on the role of AI and multidisciplinary collaboration in optimising SEP-1 metric performance and our experience in improving SEP-1 compliance. By leveraging AI-based predictive tools and structured team-based approaches, healthcare institutions can enhance early recognition, improve compliance with sepsis bundles, and ultimately reduce sepsis-related morbidity and mortality. Addressing the challenges of change management and aligning diverse stakeholders will be critical in bridging the gap between guideline-driven sepsis care and real-world clinical practice.

Our Experience

Mayo Clinic’s efforts towards developing an algorithm for timely detection and treatment of severe sepsis (aka sepsis sniffer) started in the early 2000s. Preliminary results from a two-month-long prospective implementation showed a sensitivity of 48% and specificity of 86%, and a positive predictive value of 32%, which warranted further improvement in the model (Herasevich et al. 2008). Further adjustments were made to improve model performance and a 2017 paper by Harrison et al. (2015) highlighted this algorithm, which reported 80% sensitivity and 96% specificity when tested on a validation cohort. From the perspective of operationalising a streamlined workflow to ensure compliance with sepsis bundles within the early hours of sepsis alert, the concept of Sepsis and Shock Response Team (SSRT) was adopted by a group of multidisciplinary experts from the Mayo Clinic (Grek et al. 2017). With the help of a quality improvement methodology, they demonstrated how some high-impact interventions led to dramatically improved adherence to the Surviving Sepsis Campaign guidelines. Post-implementation of their team-based interventions, the rate of compliance with all elements of the bundle improved from 0% to 51% at 6 months (Grek et al. 2017). In a subsequent publication, Mayo Clinic experts demonstrated that the adoption of a two-step machine-human interface (combined utilisation of the ‘sepsis sniffer’ tool and SSRT) left a significant impact on early sepsis care. The computerised sepsis sniffer algorithm yielded a sensitivity of 100% (95% CI, 99.13-100%) and a specificity of 96.2% (95% CI, 95.55-96.45%) in identifying sepsis in the ED. SSRT was appropriately activated in 34.1% of patients meeting severe sepsis or septic shock criteria (Bansal et al. 2018).  Moving forward, the impact of multiyear implementation of the Sepsis Sniffer tool, expanded to MICU and ED, and was re-evaluated in an expanded cohort of 1950 adult patients. Interestingly, the alert system was found to have a sensitivity of 79.9%, specificity of 76.9%, positive predictive value (PPV) 27.9%, and negative predictive value (NPV) 97.2%. Moreover, the alert system was not associated with improved sepsis care bundle compliance or in-hospital mortality among patients with severe sepsis or septic shock (Lipatov et al. 2022). This probably underscores the importance of a human element to achieve improvements in tangible outcomes.

Despite positive outcomes in sepsis management, the organisation has historically underperformed in SEP-1 metrics. With CMS adding SEP-1 into the Hospital Value-Based Purchasing (HVPB) in 2026 (2024 as the first reporting year), underperformance could lead to significant financial penalties (How to Prepare for CMS’s New Sepsis Requirements n.d.). Previous interventions including education, direct feedback to clinicians, nursing engagement, practice alerts and development of smart phrases did not produce significant improvement. The goal of the Sepsis Tools Implementation Project (STIP) was to develop, validate, and implement electronic health record (EHR) tools for the early detection of sepsis and provide real-time reminders to clinicians to improve SEP-1 compliance through appropriate therapy and documentation. A core team was established including representatives from the emergency department (ED), critical care, and informatics. Endorsement of the project was required through various local and enterprise committees prior to planning meetings to outline workflow processes and align resources. Initial discussions and coordination with stakeholders began in the Fall 2022 with go live in March 2024.

The Sepsis Timer and Checklist

An essential component of STIP involved developing and implementing an electronic timer and checklist and getting those tools into the workflow of the clinicians caring for potentially septic patients. Previous improvement efforts alerted clinicians to metric opportunities months after the case. The development of a dynamic, patient specific, automated checklist and timer allows clinicians to have real time insight into performance on the measure potentially leading to improved compliance and documentation. The timer, displayed on storyboard in the patient chart, is paired with a checklist containing colour coded banners for each element of the measure to encourage completion of all elements of the sepsis bundle within SEP-1 defined time constraints. The banners appear and change colour based on data in the EHR including lab results, vital signs and flowsheet documentation and use of key exclusion smart phrases. The banners also contain hyperlinks and documentation suggestions to guide clinicians in their efforts to meet the SEP-1 bundle elements. For example, administering fluid resuscitation when indicated or documenting contraindications is a component of the SEP-1 septic shock bundle that is frequently missed. The banner calling for fluids is only presented when shock criteria are met and the banner contains the acceptable language for documentation of the fluid exception when the measure would call for a fluid bolus, but that fluid would be detrimental or harmful to the patient. The patient specific, dynamic banners with helpful links and suggestions for documentation have been key to successful clinician engagement with the checklist.

The ED Workflow

In the ED, a workflow was developed that incorporated the pre-existing passive Our Practice Advisory (OPA) for sepsis to the automated, patient specific, dynamic checklist and timer (Figure 1). The passive OPA was updated an active alert that would start the timer and checklist if the ED provider indicated that they believed sepsis was possible for the alerted patient. The ED providers receive an interruptive alert asking for a response regarding the likelihood of sepsis. They can pause the alert once if they need more information. The next time it is presented, they must indicate whether sepsis is possible or not. If they select no, the practice alert is dismissed. If sepsis possible is selected, the timer and checklist start and are displayed for all clinicians entering the patient’s chart.  

Figure 1. Diagram demonstrating sepsis workflow in ED

Our experience with SEP-1 management included missed opportunities to complete required time-based tasks related to the handoff process from the ED to the inpatient unit or intensive care unit (ICU). The checklist display persists allows across shifts and physical locations allowing continuity and clarity in communication of the possibility of sepsis and status of each task required to meet the SEP-1 measure.

Hospitalised Patient Sepsis Workflow

While the vast majority of patients who fall into the SEP-1 denominator present through the ED, those who become septic after hospitalisation or after direct admission present a challenge in early detection. It is this population that benefits most from AI integration such as predictive modelling. This project involved training and implementing a sepsis predictive model, telemedicine clinical screen, and initiation of the timer and checklist. The sepsis predictive model continuously monitors all hospitalised patients and generates a score every 15 minutes. If the predictive model score meets the high likelihood threshold and the patient is located in select hospital units, the patient is eligible for screening. To minimise alert fatigue and predictive model score bias, a clinical screen is performed by a highly skilled telemedicine monitoring unit prior to notification of the care team. If the clinical screen is negative, additional screening requests are suppressed for 24 hours. If the clinical screen is positive, the timer and checklist are activated and the screening requests coming from the predictive model scores are suppressed for 7 days. The telemedicine unit clinician orders a lactate panel and blood cultures and executes a warm handoff to the primary treatment team who assumes responsibility for using the checklist to ensure all bundle elements are completed. Alternatively, they may elect to cancel the lab orders and document an alternative explanation for the lab abnormalities or vital signs that triggered the elevated risk score. Telemedicine support is critical to minimise the time it takes to identify a high-risk patient and initiate appropriate therapy, particularly when resources are limited. Figure 2 details the hospitalised patient workflow.

Figure 2: Diagram demonstrating hospitalised patient sepsis workflow. ECC - Enhanced Critical Care refers to the telemedicine unit 

Additional Tools/Resources

Initiation of the timer/checklist adds the patient to a list that mirrors the active banners on the checklist. This enables a centralised resource to monitor all patients with an active timer and provide real-time feedback to nursing and providers. In our facility, this task is performed by our sepsis coordinator who is a Clinical Nurse Specialist (CNS). The coordinator has a detailed understanding of the core measure specifications for sepsis which enables effective communication with clinical teams providing real time education and intervention to improve SEP-1 performance. The project team also created a dashboard and registry to facilitate case review and research opportunities.   

Findings/Results

We chose not to implement the predictive model in the ED because sepsis recognition is prompt in this setting. Additionally, we were unsure how sensitive the predictive model would be in the ED setting given limited amounts of data. As engagement with the checklist expanded, clinicians wanted to be able to start the timer/checklist without having to rely on the OPA or predictive model. An order was created to allow clinicians to use the tools as soon as they identified potential sepsis. The STIP implementation established workflows to rapidly identify potentially septic patients and gave clinicians a visual aid to monitor compliance in real time. The documentation suggestions in the checklist banners significantly improved our performance on our most frequently missed element. This combination of tools has increased compliance on the SEP-1 measure at our organisation to an average of 65% in 2024 (Figure 3). We foresee a continued and sustained improvement as a result of this intervention because it gives our clinicians real timeline of sight into compliance while they are still able to make an impact.

Figure 3. Quarterly SEP-1 compliance rates over time. MCF: Mayo Clinic Florida; FL: Florida

Key Lessons Learned

To support similar efforts being undertaken by other organisations, we have captured suggestions that may support SEP-1 documentation, education, and predictive model integration. 

• Multidisciplinary collaboration and clear communication were crucial to the implementation process. The STIP team included members from information technology, clinical informatics, data scientists, clinical abstractors, physician and nursing clinical champions and external EHR experts. Strong technical support and planning were essential to the project’s success.
• Building a data pipeline was critical for capturing SEP-1 data elements to support monitoring, analysis, and quality improvement. This effort included creating a new order, developing a provider dashboard, establishing a sepsis registry, and engaging IT staff to resolve data issues.
• Augmenting the capabilities of the predictive model required integrating new SEP-1 metrics into the EHR. The multidisciplinary team played a key role in ensuring seamless data integration and display.
• Identifying champions across various clinical roles and locations helped secure buy-in, streamline education, and improve communication. Direct communication between data teams and clinicians facilitated efficient problem-solving.
• Utilising insights from external organisations provided a valuable blueprint, saving time and improving implementation efficiency. Although some modifications were made, learning from their experiences helped streamline the process.
• Monitoring post-go-live issues was essential, with the Clinical Nurse Specialist and telemedicine nurse manager leading reviews. Physician champions kept teams informed and served as communication conduits.
• Including all levels of clinicians in the educational process ensured successful adoption, as APPs, residents, and fellows are often the first to respond to patient status changes. Proper training enabled them to place sepsis bundle orders and complete required documentation efficiently.

Sepsis Predictive Tool Development

Sepsis remains a key focus area for AI integration within clinical practice and studies have identified distinct areas of sepsis-related care that could be streamlined with the implementation of artificial intelligence – early prediction, sepsis diagnosis, mortality prediction, and personalised sepsis management (Bignami et al. 2025). Attempts at developing an early sepsis detection tool date back to 2011 when Sawyer et al. (2011) and Nelson et al. (2011), in two separate, single-centre prospective studies, reported that implementation of a real-time computerised sepsis detection algorithms resulted in improved frequency and timeliness of sepsis-related interventions. In 2012, Hooper et al. (2012) in their single-centre randomised trial utilised an automated SIRS monitoring system to facilitate early detection of sepsis in ICU setting but the study failed to demonstrate any significant improvement in outcomes. Also in 2012, Jones et al. (2016) began the Sepsis Early Recognition and Response Initiative, incorporating a screening tool into the EHR at 15 facilities.  

Machine learning models have been developed to utilise pre-extracted data as well as continuously measure real-time clinical variables such as vital signs, laboratory results, and pre-existing comorbidities, to identify patients who are at increased risk of developing sepsis or septic shock. Giannini et al.’s (2019) random forest-based machine learning model comprised 587 variables and was trained on more than 160000 patients but was able to offer a sensitivity of 26% although the specificity was 98%. Only a minimal clinical impact was noted during its utilisation in sepsis management, with a marginal increase in lactate testing and fluid administration. Lauritsen et al.’s (2020) attempt at developing a deep learning model was a step forward wherein a combination of convolutional neural networks (CNNs) and Long Short-Term Memory networks (LTSM) were used to extract and learn temporal patterns in patient data, which subsequently was used to predict the onset of sepsis. The model demonstrated an AUROC of 0.856 while predicting sepsis 3 hours earlier and an AUROC of 0.753 at 24 hours before sepsis onset. Explainability of machine learning models is another important feature that allows users to identify key contributors for a specific prediction score, in other words, enabling physicians to address key abnormalities prior to clinical decompensation. Deep Artificial Intelligence Sepsis Expert (DeepAISE) model was quite distinctive in its ability to not only predict sepsis with high accuracy (AUROC 0.90) but also report interpretable risk factors on a real-time basis, which can be acted upon (Shashikumar et al. 2021). Sepsis Immunoscore, which is the first FDA-approved AI algorithm for sepsis prediction utilises 22 clinical parameters to predict sepsis within 24hrs of hospital admission with an AUROC of 0.85 in derivation and 0.81 in external validation sets (Bhargava et al. 2024).

Being able to predict sepsis-related mortality is another key component of prognostication, which allows us to optimise treatment strategies, operationalise necessary resources, and potentially improve clinical outcomes through timely and personalised interventions (Bignami et al. 2025). Various machine-learning models (MLM) have been tried and tested to optimise predictive performance – random forest (RF) model, deep neural network (DNN), gradient-boosted decision trees (XGBoost), logistic regression with Least Absolute Shrinkage and Selection (LASSO) model – to name a few. Moreover, Super Learner (SL) models have been utilised to combine predictive capacity from separate models. Park et al.’s (2022) retrospective study adds to the growing body of literature on this topic and they demonstrated how MLMs can improve overall sensitivity, specificity, positive predictive value, negative predictive value, discrimination, and calibration in predicting in-hospital mortality in patients hospitalised with sepsis. Adams et al.’s (2022) study focused on bringing an early warning system, TREWS (Targeted Real-time Early Warning System), into clinical practice, which apparently reduced the median time to first antibiotics order by 1.85 hours. Despite acknowledging some limitations, authors demonstrated TREWS utilisation improved mortality and organ failure outcomes, particularly among patients flagged as high risk. 

Challenges of AI Related Decision Support Tools

The integration of AI in sepsis detection and management has shown promising results in enhancing early detection, streamlining workflows, and improving patient outcomes. Despite significant advancements made, challenges persist when it comes to AI implementation in the prediction, diagnosis, management, and prognostication of sepsis (Herasevich et al. 2023). Although several AI models have shown satisfactory performances, obtaining optimal model performance across diverse setups has been difficult. Apart from a risk of suboptimal performance in unseen, demographically different datasets (lack of external validity), MLMs are often thought to perform better in environments where there is access to abundant, dynamic real-time clinical data (Wardi et al. 2021; Wong et al. 2021; Agnello et al. 2024). Models have been mostly trained and tested in ICU or ED setup and there remains a question whether applicability will be similar in general medical floors. False-negative cases may lead to subpar sepsis bundle compliance if there is over-reliance on the AI-triggered alert system and false-positive cases would lead to overutilisation of resources or alert fatigue in busy clinical environments (Schinkel et al. 2023). 

Obtaining regulatory clearance prior to integration into daily clinical practice is a daunting task and consists of heavy scrutinisation of the safety, efficacy, and quality aspects. Sepsis Immunoscore, the first FDA-cleared AI algorithm for sepsis prediction, underwent extensive assessment to ensure multicentre validation, proof of improved clinical outcomes, and reliable operation across diverse settings (Bhargava et al. 2024). Similar strategies are being adopted in Europe by the Medical Device Regulation (MDR) authority. A legal framework, in the form of the AI Act, has been implemented effective August 2024 by the European Union to ensure safety transparency, and efficacy. Based on the degree of importance in the clinical decision-making process, such tools are stratified into several risk categories, with sepsis tools most likely falling into the high-risk category (The EU Artificial Intelligence Act). Therefore, clearing the regulatory process in bringing AI tools from the bench to the bedside remains a key limiting factor. 

Harnessing the power of EHR has enabled us to extract historical as well as real-time data to develop, validate, and make predictions using machine learning models (MLM). However, data protection and integrity remain another key aspect of AI model development. Apart from institutional regulations, and HIPAA compliance, there is a need for a standardised approach to handling and sharing such data. Newer technologies such as blockchain and tokenisation (e.g. Non-Fungible Tokens, NFTs) offer safe data ownership and data security (Bignami et al. 2024). Therefore, the development and integration of such model require a high-level IT support at the hospital level, which is often a limiting factor as well.

Our single centre experience with developing a multidisciplinary team highlights the importance of a human element to harvest AI’s potential in enhancing sepsis care. While adoption of telemedicine component did help with verification of automated alerts and mitigated predictive model score bias, a team-based approach in chasing the timer translated into improved compliance with sepsis bundles. Quantification of patient-centred outcomes, workforce satisfaction or cost-effectiveness are beyond the scope of this review due to lack of complete data, but we did see promising results with preliminary SEP-1 compliance rates. Incorporation of similar strategies may complement ongoing efforts in improving sepsis care and usability of AI in this regard.

Conclusion

AI-driven sepsis detection and management tools hold promise for enhancing SEP-1 compliance and improving patient outcomes. However, their effectiveness relies on thoughtful implementation, multidisciplinary collaboration, and continuous monitoring. Integrating AI surveillance with a team-based approach may lead to better adherence to early sepsis management protocols.

Acknowledgement

We are grateful for enormous contributions from the Sepsis Tools Implementation Project team and the Mayo Clinic Kern Center.

Conflict of Interest

None.