Researchers in the UK conducted a comparative study using a standardised assessment to evaluate the performance of a commercially available artificial intelligence (AI) algorithm with human readers of screening mammograms. The outcomes of this research have been published in the Radiology journal.
Although double reading can increase cancer detection rates by 6 to 15%, and recall rates are kept low, this strategy is labor-intensive and difficult to achieve when there is a shortage of readers.
As a solution to the problem AI is usually deployed to quickly solve these problems, but as Yan Chen, Ph.D., professor of digital screening at the University of Nottingham confirmed, “We need to get it right to protect women’s health”.
Professor Chen and her research team conducted their study using test sets sourced from the Personal Performance in Mammographic Screening, known as PERFORMS. Their objective was to assess and compare the performance of human readers with that of AI.
Each PERFORMS test comprises 60 exams from the NHSBSP, encompassing a spectrum of abnormal, benign, and normal findings. The scores assigned by human readers for each test mammogram were meticulously compared to the ground truth provided by AI results.
The research team harnessed data derived from two consecutive PERFORMS test sets, encompassing a total of 120 screening mammograms. These same two sets were also employed to assess the performance of the AI algorithm. The researchers compared the AI test scores with the scores of the 552 human readers; 315 (57%) were board-certified radiologists, while the remaining 237 consisted of non-radiologist readers, comprising 206 radiographers and 31 breast clinicians.
The study revealed that there was no difference in the performance observed between AI and human readers when it came to detecting breast cancer within the 120 examinations assessed. Human readers demonstrated an average sensitivity of 90% and specificity of 76%. AI exhibited comparable results, with a sensitivity of 91% and specificity of 77% when compared to human readers.
Prof. Chen said, “The results of this study provide strong supporting evidence that AI for breast cancer screening can perform as well as human readers”.
“The large prospective clinical trials that are ongoing will tell us more. But no matter how we use AI, the ability to provide ongoing performance monitoring will be crucial to its success”.
Source: RSNA
Image Credit: iStock
References:
Chan Y, Adnan GT, Darker IT, James JJ (2023). Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme.Radiological Society of North America. 308(3).
