Missing information is a routine feature of electronic health records (EHRs), shaped by irregular sampling, clinical workflows and incomplete documentation. In intensive care unit (ICU) settings, those gaps can weaken machine learning models that aim to support risk prediction, particularly for outcomes such as mortality. Different types of missingness add further complexity, from situations where values are missing randomly to patterns linked to what is observed or unobserved in the record. A comparative evaluation explored how modelling strategies respond when increasing levels of missing data are introduced, and when entire data sources, such as numerical measurements or clinical text, become unavailable. The findings point towards an important operational consideration: approaches that learn directly from incomplete data can remain robust, while heavy reliance on imputation may alter the data distribution in ways that are not always beneficial.
Missingness Mechanisms Shape Model Reliability
Missingness in clinical data can be described through three broad mechanisms: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). These categories matter because they influence how much information is genuinely absent versus indirectly recoverable from other recorded variables. In ICU time-series data, missingness is also tied to how care is delivered, including when measurements are taken, which tests are ordered and which values are recorded consistently. As a result, missingness is not always a simple technical problem to solve, because it may reflect differences in patient pathways and documentation patterns.
Traditional strategies often treat missingness as a preprocessing issue. Complete case analysis removes incomplete observations, while imputation attempts to fill gaps with estimated values. Simple statistical imputations can be quick to implement but risk washing out important variation. More advanced approaches, including ensemble imputation methods, can capture non-linear relationships across variables, yet they may still introduce bias if the missingness structure is complex. The evaluation also highlighted that as missingness increases, the similarity between the original dataset and the altered dataset declines, and the decline can be more pronounced under MNAR conditions. This supports the view that data loss is not only a question of volume, but also of whether what is missing carries hidden clinical meaning.
Must Read:EHR Interoperability: Levels, Standards and Practical Paths
Direct Modelling Shows Greater Stability Under Data Loss
The evaluation compared modelling strategies on established ICU and clinical benchmark datasets, including MIMIC-III, MIMIC-IV and P12. Numerical features were affected by missingness introduced at several levels, from low to high, and under different missingness mechanisms. The models included two approaches designed to handle missingness inside the architecture, compared against a baseline that relied on imputation. Overall, performance tended to degrade as missingness increased, but the extent of the decline depended strongly on the modelling approach and the type of missingness.
In general terms, direct modelling approaches remained more stable across missingness conditions, particularly when missingness reached higher levels. Performance drops were more noticeable under MNAR conditions, where missing values are linked to unobserved information. In these scenarios, models that rely on internal representations of missingness maintained more consistent discrimination than methods that depended on completing the dataset before prediction. While imputation sometimes produced comparable results, improvements were not consistent across missingness types or datasets.
Beyond prediction accuracy, the analysis explored how imputation affects the data itself. Distribution-level comparisons indicated that imputation can shift the original feature patterns, and that these shifts become larger as more values are filled in. Visual analyses of learned representations also suggested that while imputation may create smoother embeddings, it can weaken the separation between outcome groups once information is aggregated. This implies that imputation does not merely patch gaps but can change the shape of the dataset in ways that may be hard to detect through headline metrics alone.
Multimodal Records Highlight the Importance of Text Data
Missingness can also occur at the level of entire data modalities. In multimodal ICU records, numerical measurements and clinical notes can each provide distinct signals, and losing one modality may have a disproportionate effect. The evaluation therefore examined situations where either clinical text or measurements were masked, at different levels of modality absence, and compared several multimodal modelling approaches. Text embeddings were derived using a clinical language representation model, allowing notes to be included alongside structured variables.
A clear pattern emerged: missing clinical text tended to have a stronger negative effect than missing measurements. Across datasets, models generally remained more resilient when numerical measurements were partially removed than when clinical notes were unavailable. Some architectures designed around attention and reconstruction maintained comparatively strong performance even as missing modality rates increased, whereas other strategies showed steeper declines when text was masked.
This difference suggests that clinical narratives may carry contextual information not easily replaced by structured values alone, including summary-level descriptions and patient course cues. It also highlights a practical concern for health systems: multimodal ML pipelines may be more vulnerable to documentation gaps in text fields than to partial loss of numerical measurements. From an implementation perspective, this strengthens the case for designing workflows that preserve the availability and quality of notes, while ensuring models are built to tolerate incomplete multimodal input.
The evaluation shows that missingness is not a single technical obstacle but a multi-layered feature of EHR data, shaped by measurement practices, documentation patterns and modality dependence. Approaches that incorporate missingness directly into model design can maintain more stable performance as data loss increases, particularly under more challenging missingness structures. Imputation-based pipelines can sometimes achieve similar results, but they may also shift the data distribution and reduce meaningful separation in learned representations. The results reinforce the need to match ML design choices to realistic clinical data conditions and to treat robustness to missing data as a core requirement for safe and reliable deployment.
Source: Journal of Medical Informatics
Image Credit: iStock
