Medical imaging is central to diagnosis and treatment planning across a wide range of clinical conditions. However, rising scan volumes and workforce shortages are placing pressure on radiology services. Artificial intelligence offers tools to reduce workload and improve diagnostic performance, and among these, foundation models are attracting attention. Trained on vast, often unlabelled datasets, these models can be adapted for specific radiological tasks with fewer annotations, potentially addressing some of the fairness and generalisation issues that hinder traditional AI. At the same time, they share many of the same vulnerabilities, such as biases, performance degradation over time and limited transparency. Effective integration into clinical practice requires more than technical optimisation—it demands robust validation, ongoing monitoring and a framework for responsible use.
Advantages and Capabilities of Foundation Models
Foundation models differ fundamentally from conventional supervised learning methods. Instead of being trained from scratch for a single purpose, they are first pre-trained on large datasets drawn from varied sources. In radiology, these datasets can include millions of medical images from different modalities, acquisition protocols and patient populations. The pre-training phase captures broad patterns in the data, enabling the model to adapt to specific tasks with relatively small amounts of additional training. This data efficiency can be particularly valuable in areas where annotated datasets are scarce or expensive to produce.
Applications in radiology are wide-ranging. Vision-language models can process both images and text, enabling capabilities such as generating structured radiology reports from images or interpreting imaging findings in the context of accompanying clinical notes. Generative approaches based on foundation architectures can create synthetic medical images, which are valuable for research on rare diseases or for training when access to real patient data is restricted. Image-based foundation models can perform segmentation, classification and detection tasks across multiple anatomies, often with fewer training examples than conventional AI systems require.
Pre-training strategies such as contrastive learning, masked autoencoders and report generation have been used to teach models to recognise relevant structures and pathologies. These methods make it possible to extract clinically meaningful features even from unannotated images. In practical terms, this can mean faster model development, broader applicability and the ability to handle a variety of imaging conditions encountered in day-to-day clinical practice.
Limitations and Risks in Clinical Use
Despite their strengths, foundation models face challenges that limit their immediate clinical adoption. Their size and complexity make it difficult to trace the reasoning behind specific outputs. This lack of transparency complicates efforts to explain decisions to clinicians, patients or regulators. Some models may produce outputs that appear plausible but are factually incorrect, a phenomenon sometimes referred to as confabulation. Fine-tuning on small datasets can also lead to catastrophic forgetting, where the model loses previously acquired capabilities when adapted to a new task.
Bias is another significant concern. Performance differences have been observed across demographic groups such as age, gender and race, even in models trained on large and diverse datasets. These biases can lead to underdiagnosis or overtreatment in certain populations, reinforcing existing disparities in healthcare. Technical factors such as differences in scanner manufacturer, acquisition protocols or disease prevalence between training and deployment sites can also reduce accuracy. Over time, shifts in patient populations, imaging technologies and clinical priorities can degrade performance if models are not regularly monitored and updated.
Medical imaging presents additional challenges compared with natural image analysis. Clinical datasets are often much smaller, particularly for 3D imaging modalities, and diagnostically relevant features may occupy only a small portion of the image. Annotation quality can vary between raters, and differences in interpretation can introduce inconsistencies that affect model performance. These factors underline the need for rigorous, ongoing validation before and after deployment in clinical settings.
Building Trust through Validation and Collaboration
The successful integration of foundation models into radiology depends on more than technical capability. Continuous validation across multiple institutions and patient populations is essential to ensure models remain accurate and fair. Decentralised and federated approaches offer a way forward, enabling collaborative model development and testing without requiring data to leave its originating institution. This preserves patient privacy while increasing the diversity of data used for training and evaluation.
Must Read: Foundation Models in Radiology: Potential, Challenges and Prospects
Examples from national and international initiatives show the potential of such approaches. Networks that coordinate model training at centres with strong computational resources and validation at centres with fewer resources can broaden participation and strengthen model reliability. These collaborative frameworks can also streamline the process of updating models in response to new data or changing clinical requirements.
Clinician involvement throughout the development lifecycle is critical. Feedback from radiologists and other healthcare professionals can help ensure that models address relevant clinical needs, adapt to evolving workflows and provide outputs that are understandable and actionable. Open models with clearly documented training data and methods are more likely to meet regulatory requirements for transparency under frameworks such as the European Union’s AI Act. They are also better positioned to earn the trust of both clinicians and patients.
In addition to technical validation, adopting principles from established trustworthiness frameworks can help guide responsible deployment. These include fairness, universality, traceability, usability, robustness and explainability. Incorporating uncertainty estimation into model outputs can alert users to cases where human review is essential, reducing the risk of over-reliance on automated decisions.
Foundation models hold considerable promise for advancing radiology AI by reducing annotation burdens, improving generalisation and enabling flexible adaptation to diverse clinical tasks. Yet these benefits will only be realised if development is embedded in a framework that addresses transparency, bias and long-term reliability. Collaborative, decentralised validation, active clinician involvement and open technical practices can strengthen trust and ensure these models enhance patient care. With such safeguards, foundation models could become a central element in a robust, equitable and future-ready clinical imaging ecosystem.
Source: Insights into Imaging
Image Credit: iStock
