ICU Volume 5 - Issue 4 - Winter 2005 - Management

How to Read a Paper: Critical Appraisal of Research


Thomas Owen MBBCh, FRCA.

Specialist Registrar in Anaesthesia & Intensive Care,

North West Deanery, UK


Andrew Smith FRCA, MRCP (UK),

Royal Lancaster Infirmary, UK


[email protected]


Evidence-based medicine (EBM) has been defined as the integration of individual clinical expertise and best external evidence from research (Sackett et al. 1996). The process involves: (1) asking a clinical question (2) searching for evidence to answer it (3) appraising the evidence and (4) integrating it into practice. However, each step needs skill and practice to make EBM ‘work’. Julie Benbenishty (opposite on page 38) has provided a checklist for nursing research, which relates to step 1. Here we focus on how to make sense of published research (step 3).


Types of Evidence

Primary research may take the form of randomised controlled trials (RCTs), observational studies, case series/ reports or laboratory investigations. Some types of research design are thought to be less prone to bias. Journals publish review articles too, and there is a move towards the preparation of systematic reviews, to try to avoid bias here also. It is now common to grade the strength of evidence according to how robust, (that is, free from bias) it is likely to be (see table 1). Systematic reviews and RCTs give the highest quality evidence, and clinical practice should be based on these whenever possible (Meeran & Grocott 2005), although this may not be so straightforward in ICU practice (Vincent 2004).


Making Sense of a Randomised Controlled Trial

Overview- Here we illustrate the principles for the RCT. The three main principles are logic, validity and applicability. The first is evident from reading the paper as a whole, but the other two are apparent from the ‘methods’ section, which is often missed by readers in their rush to read the results!


Logic - Does the study seem to ‘flow’ from the starting idea or hypothesis? Has the right question been asked? Is the method appropriate for the question, and do the results follow from the methods? Lastly, does the discussion deal with the good and bad points of the study accurately and does the conclusion actually follow from the results? No amount of statistical manipulation can save a poorly thought-out study!


Validity - Is the study any good? If there are flaws in the methodology of the trial then the conclusions cannot be relied upon. For a RCT the most important determinants of quality are:

• Randomisation is important to reduce bias; randomised trials give stronger results. Method of randomisation should be explained (some methods may not actually be random!), and there should be ‘allocation concealment’. Clinicians should be unaware of how patients were allocated.

• Control Groups: depending on what is being studied the control group(s) should consist of a placebo group and/or another recognised intervention that is known to be effective.

• Blinding: ideally, both patients and clinicians should be blinded to the intervention (termed a double-blind study), but this is not always possible (use of tracheostomy for example). It is usually possible however to ensure that those assessing clinical outcomes are blinded to the treatment received.

• Analysis: the more patients are lost in follow-up, the weaker the study. The correct analytical method to adjust for this is called ‘intention to treat’ analysis, where patients should be analysed in the group into which they were randomised, even if their treatment changes during the study period.


Any baseline differences between study groups, and the contaminating effect of other treatments should also be noted.

Applicablity - Are the patients like mine? Enough detail should be given for readers to decide if the study patients are similar enough for the results to be more widely applicable. As suggested above, researchers are more likely to end up with clinically irrelevant results, if they ask a clinically irrelevant question at the beginning of the study!


Further Information

Many printed and electronic sources of guidance on how to read research reports are available (see for instance or Greenhalgh 2000). Some are more complex than others, but we believe that keeping the above 3 simple principles in mind will serve as an excellent starting-point. Future articles in this series will cover more specific topics in more detail.

Print as PDF
AuthorsThomas Owen MBBCh, FRCA. Specialist Registrar in Anaesthesia & Intensive Care, North West Deanery, UK Andrew Smith FRCA, MRCP (UK), Royal L

No comment

Please login to leave a comment...

Highlighted Products