No Blame, No Shame - A New Quality Approach in Radiology With Peer Learning

Radiologists make errors, and this is inevitable. Peer Learning is a safe way to improve the performance of the entire group of radiologists, benefitting from each other’s errors in a spirit of Just Culture. The process is anonymous; there is no scoring, shame, or blame.

Key points

• The overall prevalence of errors in radiology has not significantly changed since the 1960s.
• Radiologists are not aware of their blind spots or of the need to create a search pattern.
• Other factors contribute to the generation of errors, such as workplace interruptions, reader fatigue, volume overload or poor image quality.
• The radiology community needs an innovative system where the whole group can learn from individual mistakes in a safe and non-punitive way.
• The Canadian Association of Radiologists has recommended the cultural shift from peer review to peer learning in its Peer Learning Guide.

The overall prevalence of errors in radiology has not significantly changed since the 1960s. If we consider that the error rate is close to 2% and that one billion studies are performed worldwide annually, it means twenty million errors per year, which is considerable.

Errors in Radiology

So, why do radiologists make mistakes? In a famous research study at Harvard University called The Invisible Gorilla, participants were presented with a short video in which six people passed basketballs around and were asked to count the number of passes by the people dressed in white. At some point, a gorilla appeared in the middle of the action. Half of the people watching the video missed the gorilla. The same experience was applied to radiology, and a gorilla was superimposed on the right superior quadrant of a chest CT image. Amazingly, 83% of participants missed the gorilla (Drew et al. 2013).

There are multiple reasons why we can miss a finding in radiology. The radiology interpretative process is a combination of two decision mechanisms, as described by Daniel Kahneman, the famous psychologist and Nobel Prize of Economics 2002 (Kahneman 2011):

• Fast, using heuristics or intuitive thought processes.
• Slow, analytical with a deliberate and rational approach to decision-making.

Some of the most common causes of errors are (Busby et al. 2018):

• Inattentional blindness (42%): a finding is present on the image but is missed, maybe due to lack of context information, unexpected location, or nature of the finding.
• Satisfaction of search (22%): additional abnormalities are not identified after the first abnormality has been seen.
• Satisfaction of report (6%): perpetuating an impression from a previous report.
• Anchoring bias: radiologists won’t change their opinion even if additional contrary information is provided.
• Lack of knowledge (3%): a finding is seen but attributed to the wrong cause.

In many cases, radiologists are not aware of their blind spots or of the need to create a search pattern. Other factors contribute to the generation of errors, such as workplace interruptions, reader fatigue, volume overload or poor image quality. Radiologists need help to overcome the limitations they face in their daily practice.

The Airline Industry Example

Would you take a plane if you knew that there was a 1 or 2% risk that it may crash? I guess that you would be less tempted to travel and would consider other ways of transportation if possible. As we all know, the airline industry has achieved incredible results in improving air transportation safety to the point where it is one of the safest ways to travel.

But it has not been an easy task. According to a famous paper by David Larson et al. (2011) published in Radiology, a trigger event was the disaster of TWA flight 514, which crashed en route to Washington Dulles on December 1st, 1974, after a misunderstanding between the pilot and the air controller. The pilot thought he was clear for landing and could start his descent when the controller only gave him the authorisation to land on runway 12 according to the flight plan, which stated that the descent should start a few miles away, after the Round Hill intersection. The pilot started the descent too early and crashed the plane on a hill 29 miles northwest of Dulles International. Soon after, other pilots reported the same misleading communication happened to them, but they landed safely as they made the appropriate correction.

This was when TWA decided to implement a safe registry collecting errors and near misses, with pilots and controllers reporting voluntarily and safely their experiences without fear of blame, humiliation, or retribution. The registry is completely anonymous, and the participants cannot be identified. This is now the Aviation Safety Reporting System, where incidents are shared with the flying community, and everybody benefits from their experience.

Quality Improvement in Radiology

How does it relate to radiology? Learning from the airline industry experience, the radiology community needed an innovative system where the whole group could learn from individual mistakes in a safe and non-punitive way.

The need to improve the quality of radiology reports was evidenced by the Cochrane report in 2011, after a large-scale review of two radiologists in British Columbia, Canada. (Cochrane 2011). Similar, highly publicised reviews in other provinces confirmed the need for improvement.

The retrospective peer review system created by the American College of Radiology to answer a requirement from the Joint Commission in the United States proved inefficient and, at some point, dangerous as it was perceived as punitive and detrimental to radiologists and patients. The system was not anonymous and would point to the poor performers. The grading system would easily lead to unfair targeting and punishment. The unintended result was radiologists trying to turn around the system to avoid the consequences, which defeated the purpose of peer review.

It was opposed to the recommendation of Edwards Deming, the famous American mathematician who helped build the Japanese automotive industry after the Second World War. He revolutionised the concept of quality assurance, based at the time on measuring defects, identifying individuals producing more defects than others, punish or firing them. Instead, he stated that we must identify and correct the systemic barriers to a quality product and improve everybody’s performance. We need to focus on “what, when and how” and not only “who” (Walton 1986).

Peer Learning in a Spirit of Just Culture

Health Quality Ontario and, more recently, the Canadian Association of Radiologists, in its Peer Learning Guide, have recommended the cultural shift from peer review to peer learning (Torres et al. 2022).

So, what is peer learning? It is a continuous quality improvement (CQI) initiative focused on life-long learning based on the spirit of Just Culture, which states that “a fair and just culture improves patient safety by empowering employees to proactively monitor the workplace and participate in safety efforts in the work environment” (David Larson, RSNA Newsletter 18/02/2019, part 1 series on Just Culture). “Just Culture is steeped in the importance of patient safety and the acknowledgment that even experienced professionals make mistakes” (Just Culture: Balancing Accountability with Quality and Safety. Using errors to create opportunities to learn instead of tracking errors - Jennifer Allyn, RSNA newsletter, 25/06/2019, part 3 series on Just Culture).

Peer learning promotes collaborative group learning, removes scoring and identifies errors by type and contributing factors, discussing why and how rather than who. The review is anonymous, with no fear of punishment or medicolegal consequences. The process is prospective, with errors identified and corrected before the report is distributed. Alternatively, it can be near time retrospective, with amendments issued in a short time window to prevent impact on patient management.

The system is designed to develop a collaborative approach, increasing radiologist participation and engagement, with regular online rounds where discrepancies and great catches are presented, supported by literature, to promote a learning culture where the group as a whole benefits from the acquired knowledge.

At the annual meeting of the Society for Imaging Informatics in Medicine in 2022, we presented the anonymous, prospective, and timed retrospective, multi-institutional cloud-based peer learning solution that we recently implemented at Hamilton Health Sciences and St Joseph Healthcare, deployed for a group of close to 80 radiologists and nuclear medicine physicians. Implemented with the support of senior administration, PACS/IT management and Privacy and Legal, the radiology quality leadership developed a robust governance structure and rigorous and unbiased processes to ensure a successful deployment.

The commercial solution we adopted has been developed based on years of experience with large-scale reviews of radiologists. The process is fully anonymised: not only are patients de-identified, but the radiologists’ names are removed, and the reviewers do not know the name of the radiologists they are reviewing and vice versa. This ensures full confidentiality and non-discoverability. Cases are automatically attributed to radiologists functioning in the same subspecialty to ensure that the review is operated by peers. Sampling volumes are decided by the leadership, and we agreed on 2% of all cases. There is no scoring system; the radiologists are not ranked or evaluated. Discrepancies are categorised as major if there is a potential impact on patient care and minor if there is no impact; there is an option to recognise great catches. In case of discrepancy, the reviewer notifies the reporting radiologist, who can amend the report before it is communicated to the referring physician, as the process is prospective. If disagreement persists, cases are brought to arbitration. All major discrepancies are reviewed anyway by the Quality Assurance lead.

The solution is cloud-based with a zero-footprint viewer, which allows us to add to the roster of radiologists working in small practices or remote community hospitals where they don’t have the manpower required to implement peer learning. The adoption has been excellent, and radiologists have understood the value of peer learning and Just Culture. The attendance at the monthly peer learning rounds has been far beyond expectations.

Conclusion

Errors will happen, and this is inevitable. We can decrease the rate of errors and their impact by implementing a quality culture where errors are communicated anonymously, without fear of blame or punishment in the best interest of all. The whole group benefits from each individual experience, and the overall quality improves. This is the goal of peer learning in a spirit of just culture.

Conflict of Interest

None.