Prof. Utku Senol MD, PhD
Consultant Professor of Department of Radiology
Head of Division of Neuroradiology
Akdeniz University School of Medicine, Department of Radiology
- Radiologists are constantly dealing with probability calculations during their daily practices, whether consciously or not. To be aware of underlying mathematical processes may help to avoid errors due to subjective probability estimations.
- It is logical to expect a correct post-test probability estimate from the radiologists only when the pre-test probability has been completely provided. This argument can be mathematically proved.
- The likelihood ratio is an excellent tool to define the strength of the test and to estimate the post-test probability.
Assume that a physician has asked for your consultation regarding a low quality CAT scan. It looks like there is sulcal hyperdensity in left Sylvian fissure that is suggestive of a subarachnoid haemorrhage (SAH), but you cannot be entirely sure. Wouldn’t you like to learn something that was helpful in the diagnosis of SAH? Every radiologist would like to learn more about the complaints of the patient in similar situations. If the finding is somewhat incidental and the patient has no complaint, then the probability of a subarachnoid haemorrhage would be very low. However, if the CAT image belongs to a young female who is experiencing the worst headache of her life on that particular day, the probability of SAH would definitely be increased. What would be the probability of SAH in the first case? Although you will not be able to provide an exact numerical answer to this question, you might estimate it to be ‘very low’. On the contrary, the answer would be ‘very high’ in the latter case. Although you might not think that numbers are a way of expressing your thoughts, are you aware that there are certain probability calculations taking place in your mind? So, what type of mathematical processes lead to different probability evaluations in these two cases?
Probability in Daily Practice
This famous old quote from Sir William Osler underlines the importance of mathematics in medical decision-making: “Medicine is a science of uncertainty and an art of probability” (Bean and Bean 1950). Indeed, physicians are constantly dealing with probability calculations during their daily practices, whether consciously or not. In this article, I will try to explain the underlying mathematical calculations that physicians in general, and radiologists in particular, come up against during such decision-making processes.
In reality, every human being has a certain probability of contracting each disease. This probability varies between 0 and 100 percent. For instance, the probability of any particular 45-year-old Caucasian female having breast cancer is theoretically equal to the breast cancer prevalence of 45-year-old female Caucasians. Each finding or symptom the individual shows either increases or decreases this probability, adjusting it to a new value. In some cases, definite diagnosis might not be possible, so treatment begins when the probability reaches a certain threshold. What is important for clinicians is to decide on whether ‘to treat’ or ‘not to treat’. This decision directly depends on the probability of the disease. Radiologic diagnostic tests are therefore essential instruments in evaluating the probabilities that are crucial for the decision to start a treatment. Nevertheless, radiologists often face decision-making problems following the diagnostic test findings, and must revise the probabilities.
More often than not physicians do not use these probabilities as ‘point estimate’ values. Rather, they perceive them in terms of being ’low’, ‘high’, ‘very high’ and so on. Casford proposed 101 different descriptors, which may take place in radiology reports matching every integer percent values from 0 to 100 in his interesting letter (Casford 2000). Every physician has an approximate probability estimation related to his or her patient, and this probability changes to a new level depending on new test results or findings. The clinical information of the patient covered in the radiological orders is also important for pre-diagnosis probabilities. Moreover, radiological reports contain valuable clues about the posttest probabilities. In some cases, this information may be quite clear, such as in the exact ruling in or out of a disease. Radiologists and physicians always revise probabilities. Although the underlying mathematical steps in such instances are generally not consciously known, they are nonetheless the bases of all probability revisions.
It is clear that continuous calculations of probabilities using mathematical operations in a medical environment are neither easy to calculate, nor easily applicable. For complex problems such as those encountered in medicine, ‘mental shortcuts’ are often preferred over mathematical calculations. Such shortcuts are known as ‘heuristics’. Heuristics are used in decision-making, and they facilitate the process (Marewski and Gigerenzer 2012). However, ultimately they are only subjective probability estimates. In radiology, there are two well-known major error sources resulting from subjective probability estimates: pseudodiagnosticity and premature diagnosis (Wood 1999). Additionally, a number of heuristics might also cause errors due to being subjective.
There are several different types of heuristics. An example of ‘availability heuristics’ is when a physician has recently read or learned about a certain disease, he or she might quickly recall this during the differential diagnosis phase, even though it is actually quite unlikely. Another example of such heuristics is the ‘value-induced bias’. In this case, a certain disease is wrongly assigned a higher degree of probability due to its perceived importance. ‘Anchoring and adjustment’, ‘representativeness’ and ‘affect heuristics’ are the other examples of common heuristics that may cause errors in similar ways (Levy and Hershey 2008; Senay and Kaphingst 2009). As with the other subjective probability estimations, heuristics are prone to errors. Therefore, it is very important to be aware of all these processes and know the mathematical calculations that are the basis of probability evaluations.
Diagnostic tests are crucial for calculating disease probability. Sensitivity and specificity scores of the tests are mostly used for excluding and confirming certain diseases and are generally known, thus leading to easier probability calculations when compared to other medical processes. Therefore, radiologists are at something of an advantage for probability revisions in comparison to other physicians.
What should physicians do if they would like to mathematically calculate the probabilities and revisions?
There are several different methods for conducting such calculations, although almost all are based on Bayes’ Conditional Probability Theory. Nomograms, conditional probability graphs, 2 x 2 tables and decision tree revisions are some of the methods used for this purpose (Straus et al. 2005). In this article, I will try to explain the method, using likelihood ratios in detail. Likelihood ratio can be described as “the likelihood that a given test result would be expected in a patient with the target disorder compared with the likelihood that the same result would be expected in a patient without target disorder” (Straus et al. 2005). The likelihood ratio defines the strength of the test, and enables e stimation of the post-test probability, with the help of the post-test odds. As shown in the formula below, multiplication of the pre-test disease odds with the likelihood ratio gives us the post-test odds.
Post-test odds: Pre-test odds X the likelihood ratio:
Let’s see how to calculate odds and the likelihood ratio:
a-Pre-test odds: Pre-test probability / (1-pre-test probability)
b-Likelihood ratios: The likelihood ratio is a single number that is calculated by using sensitivity and specificity values. It is called ‘positive likelihood ratio’, when the test is positive and ’negative likelihood ratio’ when the test is negative.
Positive Likelihood Ratio: Sensitivity / (1-Specificity) Negative Likelihood Ratio : (1-Sensitivity) / Specificity
As shown in the above formula, two elements are essential for calculating the probability following a radiological test result.
1. Pre-test probability: It is possible to obtain correct probability estimations before doing any tests, with the help of accurate and complete clinical information. This explains why accurate clinical information included in radiological orders is both necessary and important. It is logical to expect a correct posttest probability estimate from the radiologists only when the pre-test probability has been completely provided. The above formula is the mathematical demonstration for this argument.
2. Likelihood ratio of the test: This is calculated by using the sensitivity specificity of the diagnostic test. Such information regarding the radiological tests is usually available in the literature. It is important to ensure that the testing method is identical in order to be able to apply this knowledge from the literature to our own circumstances. At this point, another question arises: Is it possible to use the predictive values for post-test probability calculations? Generally in the literature, such predictive values are provided along with the diagnostic characteristics. However, it is vital to remember that those predictive values regarding the diagnostic tests are valid only in the specific conditions of the original research. It cannot simply be a case of direct transfer to other study environments. An exception, however, is where the prevalence is exactly equal to the original research, since the predictive values are dependent on the prevalence. However, the likelihood ratio can be transferred to any study or situation, as long as the test method is preserved.
Let’s try to explain the calculation of post-test probabilities with two examples:
Question 1 - A CT scan reports a lymph node of 11 mm in short axis suggesting lymphadenopathy in the mediastinum. Assuming that the sensitivity is 90% and specificity is 80% for this method, what is the real lymphadenopathy probability? In this case, post-test probability cannot be calculated, because the pre-test probability is not known. Remember that the pre-test probability would be different for a young, healthy individual and a 60-year-old patient with lung cancer.
Question 2 - A breast cancerscreening programme is carried out in a population where the disease prevalence is 0.1%. Let’s assume that a patient is diagnosed with mammography and the mammography sensitivity and specificity are both 95%. What is the real probability of having a correct diagnosis for this patient?
In this situation, clinicians tend to assign a high probability based on high sensitivity and specificity scores. However, since the pre-test probability is quite low, the probability is also very low: about 2%. With a pre-test probability of 1/1000, pre-test odds for the test will be 1/999. In this case, the positive likelihood ratio is calculated as 0.95/(1-0.95)=19. Post-test odds can be calculated as 19/999, and the posttest probability is exactly 1.87% (since post-test probability = post-test odds/(post-test odds + 1)) for this specific example. This is, perhaps surprisingly, quite low. Let’s confirm it; a specificity of 95% would mean that the falsepositive rate is actually 5%. This in turn means that out of 1000 cases, only one real positive case is expected, while around 50 falsepositives would be diagnosed.
Sometimes younger colleagues might think that pre-test probability is not important if a very specific radiologic finding has been obtained. How can we explain such a statement? Consider this example: a radiograph of a ER patient apparently shows that the tibia is fractured into two. Although we do not have any patient history, we do not need pretest probability to diagnose the fracture. In this situation, the finding has a specificity of 100%. In such cases, the positive likelihood ratio would approach infinity, if calculated. No matter what the pre-test probability is, the multiplication with the pre-test odds will provide a post-test probability of 100%. In other words, the finding is ‘SPECIFIC’ or ‘SpIN’ (stands for a highly specific test, positive test result, rule the disease in)’ as named in the evidence-based medicine terminology (Straus et al. 2005).
In conclusion, like all physicians, radiologists have to deal with mathematical processes during their everyday practices. Deciding on the post-test disease probability after radiological tests requires certain probability calculations. To conduct these calculations, pre-test disease probability, test sensitivity and test specificity must be known. This notwithstanding, radiologists often make such calculations subconsciously, with the help of mental shortcuts they have developed through their experiences. However, heuristics may easily cause errors. This is why learning mathematical processes is crucial for physicians as well as radiologists. At the same time, it is somewhat intriguing to observe such similar medical practices in everyday routines.