Lung nodules are crucial indicators of lung cancer, particularly in individuals with emphysema, a condition linked to smoking that alters lung structure on CT scans. Although automated AI algorithms offer promise in aiding radiologists, their performance may vary based on the presence of emphysema, potentially leading to increased false positives or missed nodules. A recent study published in European Radiology Experimental investigates whether AI algorithms exhibit differences in sensitivity or false positives per scan (FPs/scan) when analysing CT scans of individuals with and without emphysema. This comparison extends to assessing the performance against human readers using a consensus expert panel review. The study aims to shed light on potential biases within AI software and its ability to generalise to diverse populations beyond its original training data. To achieve this, a commercially available software product serves as a test case, allowing researchers to scrutinise the impact of emphysema on AI performance. By scrutinising the performance of AI algorithms across varied patient cohorts, the study contributes to understanding the nuances of AI-based lung nodule detection and its applicability in real-world clinical settings.
Investigating Emphysema's Impact on Lung Nodule Detection
The study utilised data from the ImaLife cohort, a subset of the Lifelines study focusing on individuals aged 45 and above who underwent low-dose chest CT scans between 2017 and 2022. Lifelines is a comprehensive population-based cohort study in the Netherlands, aiming to understand various health factors and behaviours across generations. The CT scans were performed using a specific protocol on Siemens Healthineers equipment, with detailed demographic information available from the Lifelines database.
Participants were divided into groups based on the presence or absence of emphysema, with individuals with at least moderate emphysema selected for analysis. Lung nodules were initially detected and categorised by trained personnel using dedicated software, with nodules larger than 300 mm3 disregarded due to their relative ease of detection. Subsequently, a detailed classification of emphysema severity was conducted on a subset of scans.
Image Credit: European Radiology Experimental
Methodological Approach for Validating AI Nodule Detection in Emphysema Cases
To ensure valid comparisons between emphysema and non-emphysema cases, nodules in emphysema cases were matched in size to nodules in individuals without emphysema. This matching process involved selecting nodules of similar sizes from the non-emphysema group, ensuring that nodules from the same volume subgroup were chosen.
The study also employed AI algorithms to detect nodules on the CT scans, with scans processed by an AI tool that highlighted potential nodules with red boxes. Discrepancies between AI and human reader findings were evaluated by experienced radiologists, who determined the accuracy of the detections and provided confidence scores. Non-malignant findings, such as perifissural nodules and bronchovascular lymph nodes, were included in the analysis but considered less clinically relevant.
Nodule size was determined based on the volume of the solid component, with additional measurements conducted for findings detected only by AI to ensure consistency. Nodules smaller than 30 mm3 or larger than 300 mm3 were excluded from analysis, along with findings detected only by AI but manually measured within the software. This detailed methodology enabled the investigation of AI performance in detecting lung nodules, particularly in individuals with emphysema, providing insights into the potential impact of emphysema on nodule detection accuracy.
Nodule Analysis in Emphysema vs. Non-Emphysema Cohorts
After excluding participants with trace and mild emphysema and those with moderate emphysema but lacking nodule information, the study included 39 participants with at least moderate emphysema (12 with advanced destructive/confluent emphysema and 27 with moderate emphysema) and 82 non-emphysema participants. The characteristics of these groups were comparable in terms of age and gender distribution, although individuals with emphysema had higher smoking pack years due to a higher prevalence of smoking.
In total, 131 nodules were analysed, with 73 nodules detected in individuals with emphysema and 58 in those without emphysema. These nodules were matched in size, ensuring comparability between the two groups. The distribution of nodule sizes was similar between the emphysema and non-emphysema groups.
Analysis of nodular findings revealed discrepancies between AI and human readers, with 184 discrepant findings identified. Most of these discrepancies were classified as true lung nodules by the consensus panel, with AI detecting additional true nodules compared to the human reader, particularly in individuals with emphysema.
Comparison of AI Sensitivity and False Positives in Emphysema and Non-Emphysema Groups
The sensitivity of AI for nodule detection was similar between the emphysema and non-emphysema groups, but AI exhibited a higher number of false positives per scan in the emphysema group compared to the non-emphysema group. The sensitivity of the human reader was also comparable between the two groups, with no significant difference in false positives per scan.
Further analysis excluding benign typical and bronchovascular lymph nodes showed a tendency for higher sensitivity of AI in both groups. Additionally, separate analyses based on nodule size categories revealed differing outcomes, with AI showing higher sensitivity for larger nodules compared to smaller ones, regardless of emphysema presence.
In summary, while AI demonstrated similar sensitivity for nodule detection in individuals with and without emphysema, it exhibited a higher false positive rate in the presence of emphysema. The human reader's performance was comparable between the two groups. Further analyses suggested that AI's sensitivity varied depending on nodule size, with higher sensitivity for larger nodules.
Fibrosis or Scars Often Identified as Nodular Findings
The study found that the sensitivity of AI software for detecting lung nodules on low-dose chest CT scans was comparable between individuals with emphysema and those without. Notably, AI demonstrated higher sensitivity for larger-sized nodules (101–300 mm3), which are clinically relevant and often require follow-up. Similarly, the presence of emphysema did not significantly affect the sensitivity of human readers. However, there were differences in false positive rates (FPs/scan) between AI and human readers. AI had a higher rate of false positives per scan in individuals with emphysema compared to those without, whereas this discrepancy was not observed for human readers. Particularly, AI tended to produce more false positives for nodules in the 101–300 mm3 volume category in emphysema cases, primarily identifying fibrosis or scars as nodular findings.
Enhancing AI Detection of Lung Nodules: Clinical Implications
Despite the overall higher sensitivity of human readers compared to AI, this trend changed after excluding benign lymph nodes from the analysis. In this scenario, AI showed improved sensitivity, especially in emphysema cases. These findings suggest that AI can effectively assist radiologists in detecting relevant nodules, particularly those requiring follow-up. The study highlighted the importance of including emphysema cases in benchmark datasets for AI algorithms, as emphysema presence significantly influenced AI performance, particularly in terms of false positive rates. Additionally, the study underscored the need for further research with larger emphysema cohorts to better understand AI performance across diverse patient populations.
Although the study had some limitations, such as subjective emphysema classification and suboptimal nodule size matching, it provided valuable insights into AI's role in lung nodule detection. Moving forward, future research should focus on validating AI algorithms in larger emphysema cohorts and assessing the performance of different commercial software packages in detecting lung nodules in both emphysema and non-emphysema cases.
Source: European Radiology Experimental
Image Credit: iStock
