The increasing number of emergency department (ED) visits often correlates with ED crowding and delays in care. This problem highlights the need for ED triage systems that accurately differentiate and prioritise critically ill from stable patients, enabling efficient allocation of finite ED resources.  


Currently, the Emergent Severity Index (ESI) is the most commonly used triage algorithm in U.S. EDs. Despite its wide adoption, ESI heavily relies on clinical judgement, leading to high inter-rater variability and suboptimal predictive ability. A new Harvard-led study used machine learning (ML) models to predict clinical outcomes, and then compared their performance with that of ESI.


The team from Harvard Medical School created the ML prediction models using routinely available ED triage data. The team found that, compared to the conventional approach, the machine learning models were superior in predicting critical care and hospitalisation outcomes. The application of modern ML models may enhance clinicians’ triage decision making, thereby achieving better clinical care and optimal resource utilisation.


The Harvard group used combined data from the ED component of the 2007–2015 National Hospital and Ambulatory Medical Care Survey (NHAMCS) to identify all adult patients (aged ≥ 18 years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), the researchers developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, the researchers constructed a logistic regression model using the five-level ESI data. 


For this study, the clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalisation (direct hospital admission or transfer). In the test set (the remaining 30% of ED data), the researchers measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.


Of 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalisation outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85–0.87] in the deep neural network vs. 0.74 [95%CI 0.72–0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalisation outcome prediction, all ML models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82–0.83] in the deep neural network vs. 0.69 [95%CI 0.68–0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). 


In the decision curve analysis, all ML models consistently achieved a greater net benefit — a larger number of appropriate triages considering a trade-off with over-triages — across the range of clinical thresholds.


While the machine learning models achieved the superior predictive ability, the performance was not perfect. This can be attributed, at least partly, to the limited set of predictors, subjectivity of data (e.g., visit reasons), various clinical factors after ED triage (e.g., quality and timeliness of ED management and patients’ clinical responses), differences in patients’ health behaviours, providers’ practice patterns, and availability of ED resources. 


Still, in the era of health information technology, ML-based prediction has a scalable advantage — e.g., updating prediction models through an automated extraction of electronic health record data and integration with digital images, natural language processing, and continuous monitoring of physiological data.


Source: Critical Care

Image Credit: Pixabay



Raita Y, Goto T et al. (2019) Emergency department triage prediction of clinical outcomes using machine learning models. Crit. Care 23:64 

Latest Articles

emergency department, machine learning, AI Harvard study: machine learning improves ED triage