Radiology departments face increasing pressure to deliver timely and accurate reports. Compounding this challenge is a global shortage of radiologists, creating a pressing need for solutions that enhance productivity without compromising quality. Generative artificial intelligence has been recognised as a promising tool in this context, offering automated draft reporting to support radiologist workflows. A recent prospective cohort study explored the integration of a generative AI model into the clinical workflow of a tertiary care academic health system. The study aimed to assess the impact of AI-assisted draft reports on documentation efficiency, clinical accuracy and the detection of critical conditions such as pneumothorax.
Improving Workflow Efficiency with AI Drafts
The study involved the analysis of 23,960 radiographs, split evenly between those interpreted with and without model assistance. Radiologists using AI-generated drafts completed their reports significantly faster, with an average documentation time of 159.8 seconds compared to 189.2 seconds for non-assisted cases. This represents a 15.5% improvement in documentation efficiency. The model was seamlessly embedded into the hospital’s existing electronic health record and radiology reporting systems, enabling radiologists to review and edit AI-generated drafts without disrupting their usual routines.
The efficiency gains were consistent across both chest and nonchest radiographs, with no significant variation based on the type of procedure. A linear mixed-effects model confirmed the statistical significance of these improvements, and sensitivity analyses excluding individual radiologists upheld the overall findings. Notably, documentation efficiency improved even for musculoskeletal radiographs, where the model’s word error rate was relatively higher. These time savings translated into a reduction of over 63 total documentation hours during the study period, which is roughly equivalent to twelve full radiologist shifts.
To evaluate whether faster documentation came at the expense of quality, the researchers also analysed the frequency of addenda—amendments made to reports after finalisation. The rate remained virtually unchanged, at 0.13% before model implementation and 0.14% with model-assisted reports. This suggested that the introduction of AI drafts did not increase the likelihood of reporting errors that required later correction.
Ensuring Quality in Clinical and Textual Reporting
To assess the clinical accuracy and textual quality of AI-assisted reports, the study included a peer review of 800 randomly selected radiographs. Half were completed with AI assistance and half without, and all were rated blindly by experienced radiologists. Ratings were based on a Likert scale for both clinical accuracy—judging the correctness of reported findings—and text quality, which assessed grammar, organisation and appropriate language use.
The results showed no statistically significant difference in clinical accuracy between model-assisted and non-assisted reports. Both groups demonstrated high standards, and error types such as omissions or extraneous details were similar in frequency and nature. Chest radiographs were rated more highly than nonchest studies, but this was unrelated to AI use and reflected broader trends in radiological complexity.
Must Read: Transforming Radiology Reporting with Large Language Models
Likewise, the textual quality of the reports did not differ significantly between groups. The proportion of studies receiving the highest rating was comparable, and there was fair inter-rater agreement across both chest and nonchest categories. These findings reinforce the conclusion that AI-assisted reporting maintains the standards expected in clinical practice, offering radiologists a tool to enhance productivity without undermining the accuracy or clarity of their work.
Flagging Critical Conditions in Real Time
In addition to improving workflow, the AI model demonstrated potential to enhance patient safety through early identification of life-threatening conditions. The researchers deployed a shadow system to evaluate the model’s ability to flag clinically significant, unexpected pneumothorax in real time. Over 97,000 radiographs were screened during this phase, with the model issuing alerts within a median of 24 seconds after image acquisition.
Of the flagged cases, 71.8% were confirmed as true pneumothorax diagnoses, and 38.5% resulted in immediate calls to clinical teams. When cross-checked against the radiologist’s final report and inclusion criteria, the system achieved a sensitivity of 72.7% and a specificity of 99.9%. While a few small or uncertain pneumothoraces went undetected, most were not considered clinically actionable. The system also avoided over-alerting by factoring in clinical context, such as whether a chest tube was already present, reducing the risk of alarm fatigue.
Importantly, the AI system identified several cases where care was delayed due to oversight in initial interpretations. These included patients who were discharged or remained untreated until the issue was discovered during later review. By flagging such cases rapidly, the AI system offers a mechanism for earlier intervention, supporting clinicians in delivering timely care without overwhelming them with unnecessary alerts.
The study demonstrated the benefits of integrating generative AI into radiological workflows. The model significantly improved documentation efficiency, saving time without reducing clinical accuracy or textual quality. Additionally, its capability to flag urgent cases such as pneumothorax highlights its potential to enhance patient safety. These findings support a future in which generative AI tools collaborate effectively with clinicians, streamlining workflows and safeguarding care standards. Such innovations may play a critical role in ensuring both efficiency and quality in diagnostic imaging.
Source: JAMA Network Open
Image Credit: Freepik
