A recent analysis published in BMC Medical Informatics and Decision Making reports the development and external validation of a machine learning model designed to predict chronic critical illness in intensive care unit patients with acute pancreatitis. Acute pancreatitis is a common condition that can progress to severe disease requiring intensive care, with high complication rates and mortality. Chronic critical illness represents a prolonged clinical state defined by extended ICU stay and persistent organ dysfunction, yet predictive tools for this outcome remain limited. The reported approach uses clinical data collected within the first 24 hours of ICU admission and evaluates several modelling strategies across multicentre datasets, alongside an independent validation cohort, to support early risk assessment in critically ill patients.
Multicentre Data and Model Construction
The model is developed using two large ICU datasets, MIMIC-IV and the eICU Collaborative Research Database, which are combined to form the development cohort. A separate dataset from a tertiary hospital in China provides external validation, allowing performance assessment across different clinical settings. The study population includes adult ICU patients diagnosed with acute pancreatitis, with exclusions applied to repeated admissions, short ICU stays and patients under 18 years. Chronic critical illness is defined as an ICU stay of at least 14 days combined with persistent organ dysfunction measured using SOFA criteria.
Must Read: Pancreatic Tumour Boards and AI Decision Limits
A broad set of candidate variables is considered to capture early clinical status. These include demographic characteristics, comorbidities, vital signs, laboratory measurements and treatment interventions recorded within the first 24 hours of ICU admission. Data processing involves standardisation and the management of outliers by limiting extreme values, while missing data are addressed using a random forest–based imputation approach. Feature selection relies on three complementary methods, with variables retained when identified by at least two approaches. This process results in a final set of eight predictors, combining physiological measurements, laboratory indicators and clinical history, providing a structured input for model development.
Model Performance and Validation
Three modelling approaches are evaluated: logistic regression, random forest and extreme gradient boosting. Model development includes hyperparameter optimisation using cross-validation, with additional adjustments to account for class imbalance due to the relatively low incidence of chronic critical illness. Performance is assessed using multiple metrics, including AUROC, AUPRC, accuracy and calibration measures, across both internal and external validation datasets.
The random forest model demonstrates the most consistent performance across evaluations. In internal validation, it achieves an AUROC of 0.85 and an AUPRC above 0.5, alongside a low Brier score, indicating strong discrimination and calibration. External validation shows a moderate decline, with an AUROC of around 0.73 and an AUPRC above 0.4, while maintaining acceptable calibration. Logistic regression shows similar discrimination in some measures, but the random forest model provides a stronger balance between precision and recall. Comparisons with the SOFA score as a benchmark indicate that machine learning approaches offer competitive performance in identifying patients at risk, particularly when addressing class imbalance and complex data relationships.
Key Predictors and Clinical Patterns
The final model includes eight predictors: calcium level, body temperature, vasopressor use, urine output, Glasgow Coma Scale score, albumin level, haemoglobin level and a history of cerebrovascular disease. Feature importance analysis highlights calcium level and body temperature as leading contributors in the random forest model, followed by vasopressor use and urine output. Other variables, including neurological status and laboratory values, contribute to refining risk estimation.
Across the datasets, patients who develop chronic critical illness show distinct clinical patterns at ICU admission. These patients require greater life support, demonstrate more severe organ dysfunction and have higher in-hospital mortality rates compared with those who do not develop this condition. Incidence rates vary between cohorts, with values around 7% in the training set and higher levels in external validation. Differences in baseline characteristics are observed across datasets, including variations in vital signs, laboratory values and treatment use. Despite these variations, the model maintains consistent predictive performance, suggesting applicability across diverse ICU populations and healthcare settings.
The machine learning model uses a limited set of clinical variables collected within the first 24 hours of ICU admission to predict chronic critical illness in patients with acute pancreatitis. Validation across multiple datasets shows stable performance, with the random forest approach achieving the most balanced results. The model integrates key physiological and clinical factors associated with prolonged critical illness and provides a structured method for early risk assessment. Its reliance on routinely available data supports potential integration into clinical workflows focused on patient stratification and monitoring in intensive care environments.
Source: BMC Medical Informatics & Decision Making
Image Credit: iStock
References:
Xu Z, Yang Q, Su Y et al. (2026) Development and external validation of a machine learning model for predicting chronic critical illness in ICU patients with acute pancreatitis. BMC Med Inform Decis Mak: In Press.
