Prostate cancer remains a major global health issue for men, and timely diagnosis is vital to ensuring effective treatment. Accurate segmentation of prostate cancer (PCa) lesions on MRI scans is a critical step in evaluating tumour extent, guiding biopsies and supporting treatment decisions. While magnetic resonance imaging (MRI) has proven effective in visualising prostate structures, manual segmentation remains time-consuming and prone to variability. Recent advances in deep learning have enabled automated approaches, though challenges persist in handling multimodal data and generalising across institutions. In response, the PCaSAM model was developed, leveraging multimodal MRI images and foundation model capabilities to improve PCa segmentation accuracy and clinical utility.
Improving Segmentation through Multimodal Integration
Multimodal MRI—incorporating T2-weighted (T2W), diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) scans—provides complementary views of the prostate. Each modality contributes different information: T2W images highlight anatomical structures, DWI reveals tissue diffusion patterns and ADC quantifies diffusion, offering insight into cellular density. Integrating these views is critical for comprehensive lesion characterisation. However, traditional models often struggle to fuse these data types effectively. PCaSAM addresses this by combining the capabilities of MedSAM, a medical foundation model built on a Transformer architecture, with custom modules for multimodal feature fusion and prompt generation.
The PCaSAM model enhances segmentation accuracy through a multimodal fusion module (MFM) that employs attention mechanisms to extract and merge relevant features from each modality. In doing so, it preserves the specificity of each image type while creating a unified representation optimised for segmentation tasks. Additionally, PCaSAM includes a prompt generation module (PGM) to replace manual bounding box inputs, enabling fully automated segmentation. By retaining the fixed parameters of MedSAM's image encoder, prompt encoder and mask decoder, PCaSAM achieves strong generalisation with fewer trainable parameters.
Performance and Generalisability across Clinical Datasets
PCaSAM was trained and evaluated on a diverse collection of multicentre datasets comprising 1,431 cases, drawn from both public and private sources. Internal datasets were used for training and testing, while external datasets assessed the model’s ability to generalise. Compared with convolutional and Transformer-based specialist models, PCaSAM consistently outperformed its counterparts. On internal datasets, it achieved the highest Dice similarity coefficients (DSCs), significantly improving segmentation accuracy. Unlike models that rely solely on convolutional layers or require manual prompts, PCaSAM demonstrated robustness across variable imaging protocols and patient populations.
Must Read: Enhancing Prostate MRI Reporting
In external validation, where performance often declines due to domain shifts, PCaSAM showed minimal loss in accuracy, maintaining a high level of consistency. This stability is crucial for clinical deployment, where data heterogeneity is common. Moreover, visual inspections confirmed that PCaSAM handled challenging cases—such as those with weak tumour boundaries or varying imaging conditions—better than comparison models. Its reduced standard deviation across datasets highlighted its reliability.
PCaSAM also demonstrated superior training efficiency. With only 12.48 million trainable parameters—around 36% of those in U-Net—it achieved faster convergence within five epochs and required less video RAM during training. This efficiency enables use in settings with limited computational resources, making it practical for real-world applications.
Enhancing Clinical Decision Support with PI-RADS Scoring
Beyond segmentation, PCaSAM contributes to improving the Prostate Imaging Reporting and Data System (PI-RADS) scoring, which is crucial in determining cancer severity and treatment pathways. By integrating predicted lesion masks into the PI-RADS scoring pipeline, PCaSAM helps focus attention on relevant tumour regions. This approach was evaluated using a ResNet50-based classifier across two external datasets containing high-suspicion PCa cases.
Three configurations were tested: one using raw multimodal MRI images, another incorporating PCaSAM-generated masks, and a third using expert-drawn masks. The addition of PCaSAM masks led to an average AUC increase of 8.3% to 8.9% across PI-RADS scores of 3 to 5, aligning closely with expert-level performance. These findings indicate that accurate segmentation can significantly enhance downstream classification tasks and provide a pathway for integrating AI tools into radiology workflows. Notably, PCaSAM’s automated nature reduces the need for manual annotation, allowing radiologists to focus on interpretation and diagnosis.
PCaSAM represents a significant advancement in prostate cancer imaging by effectively combining multimodal MRI data with the generalisation capabilities of medical foundation models. Its architecture, which integrates attention-based fusion and automated prompt generation, overcomes longstanding challenges in PCa segmentation. By outperforming specialist models and maintaining performance across diverse datasets, PCaSAM proves its readiness for clinical application. Furthermore, its contribution to improved PI-RADS scoring underscores its potential to support radiologists in making timely and accurate decisions. With its efficient training requirements and robust performance, PCaSAM is set to become a valuable tool in precision oncology, streamlining workflows and enhancing patient outcomes.
Source: npj digital medicine
Image Credit: iStock
