Explainable AI in Cancer Imaging Faces Validation Gap

Explainable artificial intelligence is increasingly used in oncology imaging to make model outputs more transparent and clinically interpretable. Many systems, however, still rely on opaque model structures. A 2026 review in the Journal of Medical Internet Research maps 371 publications on explainable AI in radiologic cancer imaging. The findings show rapid growth, but validation, reproducibility and clinical integration remain uneven. Post hoc explanations dominate, while built-in interpretability remains less common.

Cancer Imaging AI Is Concentrated in Common Modalities
The mapped publications concentrate mainly on breast, lung and brain or central nervous system cancers. Breast cancer accounts for 112 of 371 publications, followed by lung cancer with 87 and brain or central nervous system tumours with 56. Prostate, thyroid, head and neck, liver, pancreatic, colorectal, renal or kidney and cervical malignancies also appear, alongside rare malignancies and multidisease contexts. These patterns place the strongest activity in cancer areas where radiologic imaging already has a major role in diagnosis, treatment planning and monitoring. The smaller categories show that explainable artificial intelligence is also reaching less frequent tumour sites and mixed cancer cohorts.

Computed tomography and magnetic resonance imaging are the leading modalities. Computed tomography appears in 139 publications and magnetic resonance imaging in 104, followed by ultrasound, mammography, x-ray, positron emission tomography-computed tomography and other or multiple modalities. Dataset access varies. Open datasets account for 149 publications, mixed or combined sources for 100 and not-open institutional or private sources for 85. Another 29 provide uncertain information about data sources and eight use synthetic or custom datasets. Deep learning is the main modelling approach, appearing in 260 publications, while classical machine learning appears in 67, hybrid approaches in 37 and emerging or other approaches in seven.

Post Hoc Explanations Dominate the Field
Explainability most often functions as an added layer after model training rather than as a built-in design feature. Post hoc methods account for 305 of 371 publications, while hybrid approaches account for 45 and intrinsically interpretable models for 21. The distribution shows a strong preference for explaining opaque models retrospectively, commonly through outputs that help users inspect where or how an artificial intelligence system reaches a result. Intrinsically interpretable approaches remain comparatively rare, despite the clinical interest in transparent decision logic.

Among post hoc approaches, visual methods are the largest category. They appear in 163 of the 305 post hoc publications and include saliency maps, attention maps and activation overlays that identify image regions influencing model output. Feature relevance methods, including approaches such as Shapley Additive Explanations and local interpretable model-agnostic explanations, appear in 111 post hoc publications and assign importance scores to image regions, radiomic features or clinical attributes. Text explanations, explanation by example and simplification or surrogate methods are uncommon. Modality patterns also differ. Mammography publications commonly use gradient-weighted class activation mapping, while computed tomography publications use Shapley Additive Explanations more often. Magnetic resonance imaging shows a more balanced use of both approaches.

Must Read: Explainable AI in Sepsis Prediction Faces Biomarker Gap

Validation and Clinical Integration Remain Limited
Validation does not keep pace with the spread of explainability methods. Among the 371 publications, 193 include at least one form of explainability validation and 178 include none. Expert or user-based validation is the most common form, appearing in 104 validated publications, followed by mixed methods in 74. Quantitative metrics appear in only 10 validated publications and domain or clinical knowledge-based validation in eight. These figures indicate that many explanations receive plausibility checks from medical users, but far fewer undergo quantitative or clinically grounded assessment.

Validation patterns differ by modality. Ultrasound and mammography publications have higher validation rates than computed tomography and magnetic resonance imaging publications. Expert-based validation is also higher in ultrasound and mammography. Even where validation exists, active clinical integration and workflow testing remain limited. Few publications include clinicians as active participants or use feedback to assess whether explanations align with decision-making processes or improve workflow integration.

Reproducibility and deployment also remain constrained. Code sharing appears in 65 of 371 publications, while 280 do not share code and 26 do not specify availability. Decision support system integration appears in 45 publications; 321 include none and five do not specify it. Reliance on private or mixed datasets further limits independent replication, and fairness or subgroup analysis appears rarely. These patterns leave many systems closer to proof-of-concept models than operational tools for oncology imaging.

Explainable artificial intelligence is now widely used in cancer imaging research, but the evidence base remains uneven. The field is dominated by post hoc visual and feature relevance methods, with relatively few intrinsically interpretable models. Validation is present in about half of the mapped publications, while quantitative assessment, code sharing and decision support system integration remain limited. Progress depends on stronger reporting standards, shared resources, clinically informed design, user-centred evaluation and collaboration among developers, clinicians and domain experts. These priorities link transparency with practical readiness for oncology imaging workflows.

Source: Journal of Medical Internet Research

Image Credit: iStock