Clinical decision instruments are widely used to support diagnosis, prognosis and treatment decisions by translating clinical data into structured guidance. Their growing role reflects a broader shift towards standardisation and data-driven practice in medicine. At the same time, concerns are emerging about whether these tools consistently serve diverse patient populations. Instruments developed from skewed data or narrow perspectives may perform unevenly across demographic groups, potentially reinforcing existing inequities in care. A quantitative systematic review of 690 clinical decision instruments listed on MDCalc highlights how development-stage choices can shape fairness and generalisability. By examining participant demographics, investigator geography and sex distribution, predictor variables that may be prone to bias and outcome definitions that rely on follow-up, the analysis shows where imbalances can be introduced before an instrument reaches the bedside.
Participant And Author Demographics
One prominent source of potential bias lies in the composition of participant cohorts used to develop clinical decision instruments. Across hundreds of instruments, self-reported demographics show substantial skew. Participant populations are predominantly white, accounting for 73% overall, with Latino groups notably under-represented. Sex distribution is also uneven, with 55% of participants identified as male and nearly two thirds of instruments enrolling a higher proportion of men than women. A small number of instruments were developed using cohorts composed exclusively of one sex, including cases not limited to sex-specific conditions. Such imbalances matter because instruments derived from narrow cohorts may yield less reliable guidance for under-represented groups.
Must Read: AI Risks in Healthcare: From Bias to Existential Threats
Beyond patient cohorts, the characteristics of investigator teams also reveal geographic and demographic concentration. More than half of authors are based in North America and almost one third in Europe, with 45% affiliated with institutions in the United States. Author sex distribution is even more skewed than that of participants, with 71% inferred as male. While author demographics do not directly determine algorithmic performance, they may influence which research questions are prioritised, which populations are recruited and which outcomes are considered relevant. Together, these patterns raise questions about how well instruments developed in specific regions and contexts translate to broader and more diverse clinical settings.
Predictor Variables and Embedded Bias
The choice of predictor variables represents another pathway through which bias can enter clinical decision instruments. Most instruments rely on commonly collected clinical measures such as age, vital signs and laboratory results. Age appears most frequently, present in nearly one third of instruments, while variables such as sex often exert limited influence on final outcomes. However, a small subset of predictors warrants closer scrutiny because of their susceptibility to subjective interpretation or structural inequities.
Race and ethnicity are included in 1.9% of instruments, as is abdominal pain, while family history appears in 1.4%. When incorporated without careful consideration, these variables may encode social and systemic disparities rather than biological risk. Race and ethnicity can reflect differences in access to care or diagnostic pathways, which may then be embedded into predictive models. Family history may disadvantage individuals with limited medical records or recent migration, reducing the apparent number of risk factors available to inform calculations. Even symptoms such as pain severity may be influenced by clinician perception rather than patient experience, introducing further variability.
Importantly, the presence of such variables does not automatically imply biased outcomes. In some contexts, sensitive predictors may help correct for disparities in data quality or access. The analysis instead highlights the need for clarity about how predictors are used and which populations may be affected, enabling clinicians to interpret results with appropriate caution.
Outcome Definitions and Follow-Up Effects
Outcome definitions, particularly those requiring follow-up, represent a less visible but significant source of potential bias. Around 26% of analysed instruments rely on follow-up data to determine outcomes, either through active methods such as telephone or in-person contact, passive surveillance using records or a combination of both. Active and hybrid follow-up approaches, together accounting for 10% of instruments, are more vulnerable to socioeconomic barriers. Access to transport, stable housing, telephone connectivity and language support can all influence whether follow-up data are captured.
When participants cannot be reached, some development studies exclude their data or make assumptions to preserve sample size. These practices may systematically disadvantage groups less able to engage in follow-up, skewing outcome measures towards those with greater resources. Passive follow-up mitigates some barriers but introduces others, including missing data, fragmented records and measurement error across unlinked databases. As a result, outcome definitions based on follow-up can disproportionately reflect the experiences of certain populations, affecting the apparent performance of an instrument.
Clinical decision instruments offer clear benefits by promoting consistency and evidence-informed care. However, analysis of hundreds of widely used tools shows that bias can arise at multiple stages of development, from who is enrolled and who authors the work to which predictors and outcomes are selected. These factors do not imply intentional bias but highlight structural patterns that may limit generalisability and equity. Awareness of these issues is essential when implementing and interpreting such tools. Greater transparency around development cohorts, predictor choices and outcome measures can support more informed use. As reliance on algorithmic guidance continues to expand, careful consideration of these dimensions remains central to delivering fair and effective healthcare.
Source: npj digital medicine
Image Credit: iStock
