Radiomics has become important in precision medicine, utilizing both deep and engineered features to analyze shape, intensity, or texture. Initially, radiomics focused on a single region of interest (ROI) such as a primary tumor, but now many models aggregate data from multiple ROIs within the same image. This includes information from multiple tumors and healthy regions to better predict cancer outcomes, aiming to enhance predictive performance.
This aggregation approach raises the question of identifying which ROIs most influence model predictions. Understanding these key ROIs can improve model explainability and provide medical insights. Traditional tools for explaining predictive models often assign global importance to aggregated features without pinpointing individual ROIs' roles.
The Shapley value, a concept from cooperative game theory, has been adapted to explain machine learning model outputs. It assigns scores to each ROI based on its contribution to the overall prediction. To address the need for better interpretability in radiomics, a new tool called RadShap has been introduced. RadShap uses Shapley values to highlight each ROI's influence within a radiomic model. It has been tested on synthetic tasks with known ground truths and applied to models for histologic classification and survival prediction, demonstrating its utility.
Evaluating Immunotherapy Response in Advanced NSCLC Using Radiomics
A retrospective cohort study included 130 individuals with advanced non-small cell lung cancer (NSCLC) treated with pembrolizumab and chemotherapy. Response to immunotherapy was assessed using progression-free survival (PFS). Clinical data and baseline [18F]FDG PET scans were collected following GDPR guidelines, with approval from the Institut Curie and patient consent.
For PET scans, intensities were converted to standardized uptake values (SUVs), and tumor foci were delineated using LIFEx software. Tumors were resampled to a fixed voxel size, and a threshold of 2.5 SUV units was used for segmentation. Ten radiomic features (four shape features and six first-order features) were computed using PyRadiomics.
RadShap, a tool leveraging Shapley values, provides local explanations for predictions made by radiomic models by assigning importance scores to each ROI based on their contribution to the model’s prediction. The tool can handle scenarios where some ROIs are not optional by replacing missing values with those from a background dataset.
Validating RadShap: Synthetic Task Evaluation and Case Studies in NSCLC Prediction and Classification
The tool was validated using a synthetic task where a radiomic signature was created from a subset of features. Labels were assigned to patients based on the signature values, and a logistic regression model was used to predict these labels. The consistency of the explanations provided by RadShap was evaluated by measuring the frequency of correct predictions and the assignment of higher Shapley values to significant lesions.
Two case studies were conducted. The first compared PFS prediction using a primary tumor model versus an aggregative model, incorporating multilesion features like total metabolic tumor volume and standardized Dmax. The second study compared histologic subtype prediction (adenocarcinoma vs. other subtypes) using features from the biopsied lesion versus all segmented lesions. Both models were evaluated using cross-validation, with performance metrics including AUC, balanced accuracy, sensitivity, specificity, C-index, and time-dependent AUC.
Statistical analyses involved right-tailed tests to assess the significance of RadShap explanations and permutation tests to compare model performance. RadShap was shown to effectively identify influential ROIs, enhancing model explainability and providing insights into the prediction of PFS and histologic classification in NSCLC patients.
RadShap for Explaining Multiregion Radiomic Models in NSCLC Prediction & Classification
Radiomic studies are increasingly using explanation tools to understand their predictive models. However, there is a lack of suitable tools for models combining multiple regions within an image. Tools that can highlight the most impactful regions are valuable for deciphering model behavior, detecting biases, and providing medical insights. Current methods, like attention weights, are often tied to specific predictive models.
The novel tool RadShap, which is model- and modality-agnostic, addresses this gap by using Shapley values to highlight the impact of individual regions on model predictions. It was validated with 1,000 synthetic classification tasks involving 130 metastatic NSCLC cases. RadShap effectively identified lesions with high signature values, aligning with the ground truth.
RadShap was then applied to predict PFS and classify histologic subtypes using [18F]FDG PET scans of metastatic NSCLC patients. Models that aggregated information from all lesions performed better than those using a single lesion. Specifically, for PFS prediction, a model combining total metabolic tumor volume and standardized Dmax outperformed a primary tumor-only model. RadShap provided insights into these aggregative models, highlighting the advantages of using information from multiple lesions.
RadShap, implemented as a Python package, is user-friendly and runs quickly without needing model retraining. It uses Shapley values to provide robust explanations and requires some Python coding skills for custom aggregation functions. The tool should be used with rigorous validation to ensure the model's robustness and generalization.
The study acknowledges limitations, including a small sample size and the need for further validation across diverse real-world scenarios. Despite these, RadShap's alignment with human understanding in real-world contexts and its potential benefits underscore its relevance.
RadShap enhances the understanding of radiomic models involving multiple regions, facilitating the adoption of high-performing models and providing valuable medical insights.
Source & Image Credit: Journal of Nuclear Medicine
