Is resident case volume a good indicator of clinical performance?

The total number of studies interpreted during radiology training correlates with performance, according to a retrospective cohort study of three graduating classes of radiology residents between the years 2015–2017. 

"The results of this study indicated that there is a correlation between the volume of studies interpreted during radiology residency and performance with objective assessment tools," write the authors from the University of Pittsburgh Medical Center. "However, the relationship is nonlinear, with optimum performance seen at a volume of about 16,000 cases." 

The authors found that residents who interpreted too many or too few examinations had poorer performance, suggesting that there is a “sweet spot” for optimising resident workload.

Today the volume of studies ordered by clinicians is on the rise. Radiology residents are therefore interpreting increased volume over the course of their residency training when compared to their predecessors. Increased volume potentially means less time is spent reading each study which can lead to increased error rates. Studies of “on-call” radiology residents demonstrate higher discrepancy rates with increasing case volumes. The Accreditation Council for Graduate Medical Education (ACGME) does not have a cap on the number of studies that radiology residents can interpret over the course of their residency training.

Pitt researchers were keen to determine whether there is a correlation between the volume of studies interpreted during radiology residency and performance with objective assessment tools. They examined data from three graduated classes of radiology residents from a single residency programme. The total number of studies interpreted by each resident during residency was tracked. Clinical performance was determined by tracking an individual resident's major discordance rate. A major discordance was recorded when there was a difference between the preliminary resident interpretation and final attending interpretation that could immediately impact patient care.

The ACGME milestones at the completion of residency, Diagnostic radiology in-training scores in the third year, and score from the American board of radiology core exam were also tabulated. Pearson correlation coefficients and polynomial regression analysis were used to identify correlations between the total number of interpreted films and clinical, test, and milestone performance.

Thirty-seven residents interpreted a mean of 12,709 studies (range 8,898–19,818; standard deviation [SD] 2,351.9) in residency with a mean major discordance rate of 1.1% (range 0.34%–2.54%; SD 0.49%). There was a nonlinear correlation between total number of interpreted films and performance. As the number of interpreted films increased to approximately 16,000, clinical performance (p = 0.004) and test performance (p = 0.01) improved, but volumes over 16,000 correlated with worse performance.

"The number of studies interpreted by radiology residents during their training, correlate with clinical performance and test performance. While excessive volume can be detrimental to accuracy, the value of adequate clinical exposure during residency is clear. Training programmes should focus on providing adequate resident volume and variety to maximise resident clinical education and, ultimately performance," the study concludes.

The authors cite some important limitations to this study, including: 1) data were derived from a single high volume academic residency programme and these may not be generalisable across residency programmes; 2) the sample size was relatively small; and 3) it is likely some studies with major discordances, are not officially recorded by the attending radiologist and thus excluded from the calculation of the resident's discordance rate.

Source: Academic Radiology