The development of predictive analytics in critical care has become increasingly important, particularly with the rising volume of electronic health record (EHR) data. Traditional models and proprietary tools have fallen short in handling time-dependent variables and irregular data intervals typically found in intensive care units (ICUs). Addressing these limitations, researchers introduced a novel Transformer-based Encounter-level Clinical Outcome (TECO) model. TECO is designed to predict ICU mortality using longitudinal EHR data, showcasing its potential as a robust early warning system for critically ill patients.
Model Development and Methodology
The TECO model was developed using EHR data from 2579 adult patients with confirmed COVID-19 admissions to ICUs across Texas Health Resources. Incorporating both baseline variables (age, sex, ethnicity, race) and time-dependent variables (such as SpO2, respiration rate and mSOFA score), the data were processed into 15-minute intervals. Each segment was encoded using a transformer architecture that applied multi-head attention across six layers, with predictions generated using a Softmax classification layer. The model was trained on nine temporal configurations to estimate ICU mortality at various future time points.
Must Read: Humanising Intensive Care: Restoring Patient Identity in the ICU
Comparative models included random forest (RF), extreme gradient boosting (XGBoost) and the Epic Deterioration Index (EDI). Performance was assessed using the area under the receiver operating characteristic (AUC) curves. Internal validation utilised a split of the COVID-19 dataset, while external validation drew from the MIMIC-IV database for patients with acute respiratory distress syndrome (ARDS) and sepsis.
Performance and External Validation
Internally, TECO consistently outperformed RF, XGBoost and EDI across all prediction intervals, with AUC values ranging from 0.89 to 0.97. Notably, TECO predicted 60-hour mortality with an AUC equivalent to EDI's 12-hour prediction, underlining its early warning capabilities. Externally, the model also demonstrated superior predictive accuracy in both ARDS and sepsis cohorts. Although overall performance dropped relative to the internal cohort, TECO maintained a lead over other models, particularly in earlier ICU stages.
TECO’s predictions were also more clinically interpretable. Feature importance analysis revealed mSOFA and SF ratio as key variables, aligning with known indicators of ICU mortality. Additionally, TECO's capacity for real-time monitoring provided dynamic probability estimates throughout the ICU stay, with distinct trajectories for patients who survived compared to those who did not.
Interpretability and Clinical Implications
One of TECO's key advantages lies in its interpretability and generalisability. Unlike proprietary tools limited to specific EHR systems, TECO operates across platforms, using openly described mechanisms. Its ability to incorporate time-dependent monitoring data makes it particularly suited for real-time applications. In comparison to EDI, which lacks transparency and has limited adaptability, TECO provides consistent performance without being constrained by a fixed dataset or commercial ecosystem.
The model's interpretability was enhanced through Shapley Additive Explanations (SHAP) for TECO and permutation importance for RF, reinforcing the clinical validity of selected features. An ablation study confirmed the value of baseline variables, though TECO remained the top performer even when these were excluded, further demonstrating its robustness in modelling dynamic patient data.
The TECO model represents a significant advance in ICU mortality prediction, offering a transparent, adaptable and high-performing alternative to existing tools. Its ability to leverage continuous, irregular EHR data positions it as a promising candidate for clinical integration. Although external validation showed some performance decline, TECO consistently outperformed other models and showed strong potential for early deterioration alerts. Future development should explore broader validations across different patient populations and assess the impact of additional variables on performance and computational efficiency. As a lightweight transformer model, TECO could offer a scalable and impactful solution for critical care monitoring and decision support.
Source: JAMIA Open
Image Credit: iStock
