A deep learning system combining suspected nodule indexing with malignancy risk assessment has been evaluated for its impact on pulmonary nodule detection on chest CT. The assessment included both standalone performance testing and a multi-reader crossover study involving board-certified radiologists. The dataset was intentionally designed to be clinically challenging, with a focus on subtle early-stage cancers and small nodules, including cases derived from earlier rounds of a large lung cancer screening trial. In total, several hundred CT examinations were analysed, spanning screening and non-screening settings and including malignant, benign and normal cases. Radiologist performance was assessed using localisation-based receiver operating characteristics, alongside nodule-level sensitivity, specificity and false-positive rates. Interpretation time was also recorded to examine potential workflow implications when AI outputs were available at the outset of image review.
Dataset Composition and Reference Standard
Ground truth was established by a panel of experienced thoracic radiologists from different institutions. Panel members independently marked regions of interest, measured lesions and assigned nodule-level classifications. Nodules within a defined size range were eligible for inclusion, and a majority agreement rule was applied to confirm true findings. A lower size threshold consistent with established reporting frameworks was used to reflect routine clinical practice and to account for variability in assessing very small lesions.
Must Read: Parametrial Endometriosis Detection with Transvaginal US
The final dataset comprised several hundred chest CT scans with thin-slice reconstructions. Most examinations were low-dose scans from a major screening trial, while the remainder were non-screening studies, including some contrast-enhanced acquisitions. The case mix included a substantial number of confirmed lung cancers, a group of benign non-calcified nodules and a large set of normal examinations. Across cancer cases, more than one malignant lesion was present in some individuals, and benign nodules were also annotated within both cancer and non-cancer groups.
To increase clinical complexity, many screening cancers were selected from earlier imaging rounds in which a lesion was retrospectively visible at the site of a later confirmed malignancy. The dataset was enriched with small nodules, the majority measuring below a commonly used clinical threshold. Cancer diagnoses were confirmed by histopathology or by matching to a corresponding lesion at diagnosis, while benign nodules were classified based on radiological stability over an extended follow-up period.
AI System Design and Standalone Performance
The evaluated model was developed to analyse chest CT scans and identify potential nodule locations with an associated suspicion score. Preprocessing steps included standardisation of slice spacing, lung segmentation and calibration of image windows, followed by three-dimensional convolutional neural network–based detection and additional processing for feature analysis and false-positive reduction. The malignancy probability score was derived using a framework aligned with radiological principles, incorporating characteristics such as size, morphology and density patterns.
Before the reader evaluation, the system underwent training and validation using a publicly available lung imaging dataset. In standalone testing on the curated dataset, the AI demonstrated moderate-to-high sensitivity for qualified nodules at a predefined operating threshold, with a limited number of false positives per case. Performance was also assessed at the case level using receiver operating characteristics analysis, showing acceptable discrimination for both all nodules and malignant nodules specifically.
The AI output included highlighted nodule locations, an estimated suspicion level and quantitative descriptors such as size and mean density. These outputs were presented as indexed thumbnail previews integrated with the CT images. The suspected nodule indexing function was intended to guide radiologists towards AI-identified regions at the start of review, potentially streamlining visual search in examinations containing numerous slices.
Impact on Radiologist Performance and Workflow
The reader study involved 16 board-certified radiologists with varying years of post-certification experience. Using a crossover design with a washout interval, each participant interpreted the full dataset both with and without AI assistance. In the AI-supported arm, indexed thumbnail prompts and optional overlays were available at the beginning of case review, while radiologists retained full control to accept, dismiss or add findings.
AI assistance was associated with improved localisation-based performance for both malignant nodules and the overall set of qualified nodules. Nodule-level sensitivity increased when AI was available, while specificity remained largely stable. The overall number of false positives per case showed only minor change. Improvements were also observed in a subset of cancers that had been missed during original screening interpretations. In these cases, the AI system identified a substantial proportion of retrospectively visible lesions, and the average number of radiologists detecting them increased when AI support was provided.
At the same time, not all AI-flagged malignancies were accepted by readers, particularly when lesions displayed atypical morphology. There were also instances in which the AI failed to detect a lesion on an earlier scan but identified it on a later study when conspicuity had increased. Mean interpretation time per case was reduced in the AI-assisted arm, reflecting a measurable gain in reading efficiency attributed to the indexing preview function.
The authors noted several limitations. The study was conducted in a controlled setting that differs from routine clinical workflow. Disease prevalence in the dataset was higher than would be expected in typical screening populations, and some subgroup analyses were limited by small sample sizes.
Integration of suspected nodule indexing with malignancy risk estimation was associated with improved radiologist performance in detecting pulmonary nodules on chest CT within a challenging, enriched dataset. AI support increased localisation-based performance and nodule-level sensitivity while maintaining similar specificity and only minor changes in false positives. Interpretation time was reduced when indexed AI outputs were available at the start of review. The findings also indicated that AI prompts contributed to the detection of retrospectively visible cancers previously missed on screening examinations, although certain atypical lesions remained challenging for both the system and human readers. Controlled study conditions and dataset composition should be considered when interpreting these results.
Source: Journal of the American College of Radiology
Image Credit: iStock
