Imaging Management, Volume 11 - Issue 4, 2011

Does CAD Provide an Alternative to Double Reading?

share Share
Carla Guerriero
Research Fellow in Health Economics
John Cairns
Health Service Research and Policy Department
London School of Hygiene and Topical Medicine
London, UK
Fiona j. Gilbert
Maureen GC Gilian Aberdeen, Biomedical Imaging Centre
Lilian Sutton Building,University of Aberdeen
Aberdeen, UK
Mathew G. Wallis
Cambridge Breast Unit and Biomedical Research Institute
Addenbrookes Hospital
Cambridge, UK
Can You Briefly Summarise the Aims and Conclusions of the CADET II Study for Our  Readers?

The aim of the CADET II study was to determine whether the performance (cancer detection rate and recall rate) of a single reader using CAD (SRCAD) could match that of the standard protocol in the UK NHS Breast Screening Programme (BSP), of double reading. Mammograms from over 28,000 women attending the screening mammography trial were double read and then read by another reader using CAD, and cancer detection and recall rates were compared. There was no significant difference in cancer detection rate (87.2 SRCAD vs 87.7 percent DR), although there was a small but significant increase in recall rate with SRCAD (3.9 percent vs 3.4 percent DR). This is still within the acceptable limits of the UK and other European screening programme guidelines. Overall the CADET II study demonstrated that SRCAD could be an alternative to DR.

Which Parameters Affected the Model Used to Assess Cost-Effectiveness in CADET II and How?

In our economic model we showed that SRCAD would be cost-increasing compared to DR in all sizes of screening centre because the cost of CAD equipment, staff training and the additional assessment costs associated with reading with CAD are greater than the saving in reading costs that would be made. Sensitivity analysis showed reading time and the reader type i.e. whether radiologist or radiographer, had the greatest effect on the cost-effectiveness of SRCAD compared with DR. In addition, the cost of the CAD equipment had a significant effect.

Studies Evaluating CAD have Mainly Focused on the Higher Recall Rates that Reportedly Arise With its Use - Why Did You Decide to Focus on Cost-Effectiveness?

The results of the CADET study had shown the clinical effectiveness of SRCAD but before it could be considered for widespread implementation in the UK screening programme, evaluation of its cost-effectiveness compared to DR was required. We focused on cost effectiveness because the increase in recall rate using SRCAD leads to additional healthcare costs: for example, additional assessment clinics and diagnostic tests.

What, in Your Opinion, is the General Perception Amongst Radiologists on its Effectiveness Versus Double Reading at Present?

The CADET II study was one of the first prospective studies conducted in the real-life setting of a screening programme and it demonstrated that SRCAD could be an alternative to DR. However, if you look at a couple of meta-analyses of CAD studies that have been published recently, the general consensus appears to be that DR, especially if used with some form of arbitration/consensus is superior to SRCAD. But it is difficult to generalise the results of CAD studies because of limitations and biases in the design of many of the published studies and also the fact that many of the early CAD studies were conducted in the U.S. Differences in the screening programme organisation between the U.S. and the EU make it difficult to interpret the impact that CAD may have on reader performance in the EU.

The main reason for radiologists being unconvinced about CAD is that they are concerned about the performance of CAD in terms of its relatively poor specificity i.e. there are approximately 1 - 2 false CAD marks per four-view case, which they find distracting and there is concern that if these false positive CAD marks are acted upon this could lead to additional recalls and associated costs. To address this, imaging scientists and commercial CAD vendors are constantly working to try to improve the specificity of the algorithms and make CAD marking more informative. In the UK many of the units used for screening are analogue and not digital mammography. It is difficult to implement CAD with a film/screen system.

Many EU Countries Cite a Shortage of Trained Radiologists. Could This Trend Incentivies Imaging Departments to Deploy CAD?

Yes it could. Now that screening programmes are moving over from film-based to digital mammography systems, it will be much easier to integrate CAD into workflow. There is no need for the additional step of digitising films and CAD marks can be displayed directly as an overlay on the digital images on the reviewing workstation. However, many departments are also training radiographers to be mammography readers and the use of radiographers in DR could be a more cost-effective solution to the shortage of specialist breast radiologists. This is why it is important to evaluate the cost-effectiveness of using CAD with different grades of readers.

You Conclude that Indeed, CAD is Not Cost-Effective – Can You Summarise the Reasons Why and Elucidate With Some Figures that Demonstrate This?

The higher recall rate of single reading with CAD versus double reading (3.9 percent vs 3.4 percent) and the assessment cost (average cost estimated for 153 UK pounds per patient) were the main reason why CAD is cost ineffective even in high volume screening sites where economies of scale allow the lowering of the equipment cost per woman screened. According to sensitivity analyses results, if the assessment visit for all women had consisted only in an ultrasound, SRCAD would have been cost saving compared with double reading in both high and average screening units. This is impossible because according to latest NHS Breast Screening Programme data, 32 percent of assessment visits include a needle biopsy and an additional 2.2 percent of women are referred for an open biopsy. The only possibility for SRCAD to be cost effective is to improve CAD's performance and lower the difference in recall rates between SRCAD and SR. If the difference becomes as low as three percent SRCAD would be cost saving in high and average volume sites. 

What Does the Study Tell Us About What Further Needs to be Done to Assess the Cost Performance and Utility of CAD?

More accurate estimates of the cost of CAD equipment and upgrading need to be determined. The number of CAD servers and workstations required is dependent on the size of the screening centre. Also, if CAD technology was to be used in a national screening programme then it is likely that bulk purchases could be negotiated with commercial suppliers. The estimation of healthcare costs should also take into consideration any increase in healthcare resource use arising from increased anxiety and stress associated with unnecessary recalls.

In view of the high false marker rate with CAD, the additional cost of involving an additional reader to arbitrate cases in which there is discordance between the human reader and CAD should also be evaluated. Arbitrating all discordant cases would clearly not be cost effective so appropriate reading training and QA with CAD should be introduced to minimise the need for arbitration.

Is the Current Streaming of Students into Mammography Reading Practice Adequate?

More radiologists and radiographers are being trained in mammography reading. Reading standards continue to improve. This results in a reduced need for CAD.

In screening mammography, readers need to look at large numbers of mammograms and only about six per 1,000 are cancer cases. This is a challenging task and it is recognised that there is considerable variation in performance between different readers and by the same reader on different occasions. This is partly due to individual perceptual variation but can be influenced by reader fatigue and distraction. CAD does not suffer from these human limitations so should be more consistent in its performance. However, even CAD does not have 100 percent sensitivity and specificity. Although CAD detects some cancers that the human reader has overlooked or misinterpreted there are also some cancers that the human reader(s) detect that are not marked by CAD. This was shown in CADET II study. It is likely that large-scale introduction of CAD technology should allow for CAD vendors to offer more competitive prices and therefore make it more attractive to healthcare purchasers.




IntervieweeCarla Guerriero Research Fellow in Health Economics John Cairns Health Service Research and Policy Department London School of Hygiene and Topic

No comment

Please login to leave a comment...

Highlighted Products