Clinical Audit: How Simple can we Make it?

Clinical audit is a continuous improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change.A cyclical approach to comparison of practice against the standard with implementation of changes has led to the audit spiral concept, leading onwards and upwards to the eventual attainment of the standard, and presumably nirvana.

Clinical audit is defined in the UK government white paper Working for Patients (1989) as: “the systematic critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient”. This inclusive approach to providing evidence of quality of care was begun in 1993 when the concept of ‘clinical audit’ superseded ‘medical audit’. The Health Act of 1999 set up the framework of ‘Clinical Governance’ through which ‘NHS organisations were to be accountable for continually improving the quality of their services and safeguarding high standards of care’. Clinical audit forms an integral part of governance.

During the years of the NHS internal market, clinical audit was of paramount importance to providers to show evidence of quality to purchasers. In recent years the emphasis has shifted to support the use of evidence- based practice, a trend which has proved useful in patient care and popular amongst clinicians. Sadly, the evidence base for many standards in imaging is lacking but the worm is starting to turn and the need for auditable standards now drives research.

Types of Audit

• Audit of structure: How is the service set up and is this optimal?

• Process audit: How do we provide a service and is this up to standard?

• Outcome audit: How do we influence patients’ health and is this in line with best practice?

Although structure and process audits are simplest to carry out and are powerful tools for securing scarce resources, outcome audit is essential to show improvement in patient health and is favoured by proponents of evidence-based practice. This type of audit, although feasible and frequently performed for interventional radiology, is difficult in diagnostic imaging. Accuracy of diagnoses and staging or biopsy success are often regarded as outcome audit.

Sources of Audit Standards

Guidelines for good practice have proliferated in recent years. Many are evidence-based but some also take cost-effectiveness into consideration. UK national bodies such as Scottish Intercollegiate Guidelines Network (SIGN) and the National Institute for Health and Clinical Excellence (NICE) in England and Wales, the Healthcare Commission, and the Department of Health have developed, commissioned and adopted guidelines which are freely available through their websites.

Radiological referral guidelines are available through learned bodies such as the Royal College of Radiologists, the American College of Radiology and the Royal Australian and New Zealand College of Radiologists. Frequently, appropriate standards do not exist globally, nationally or in world literature, in which case a local standard may be derived.

Audit Methodology

Once the standard has been identified, an indicator (measurable quantity related to the target), such as the percentage of cancer patients staged within two weeks, is measured for a sample of patients. The sample size should be related to the target percentage and statistical advice may be needed here for national audits or larger departmental or hospital audits.

On the whole, the selection of “the last sixty patients” is better for analysis than “last month’s patients”. Results may be as simple as either “standard achieved” or “standard not achieved”, the former resulting in reassurance and proof for doubters while the latter will lead correctly to the recommendation of changes. Reaudit marks the closure of the audit loop and is particularly relevant following implementation of change but is also important for reassurance, albeit at a longer interval from the first audit. This cycle or spiral leading to proof of ideal practice underpins the principle of good audit.

Scheduling Audits

Around two half-days are required for most departmental audits, including data gathering, analysis and preparing the presentation. Medical staff should have time identified in their job plans, often 2 - 4 hours per week. A half-day per month is often allocated by many Trusts for audit meetings. Consultants are expected to attend at least 70% of meetings. Management should be represented at all meetings. Audit facilitators are employed by many UK Trusts to advise and coordinate larger audits.

Presenting an Audit

Departmental audit meetings are the appropriate forum for presentation and when audits are interdepartmental, the other departments should be invited to participate at an early stage. This has the double advantage of helping with data gathering and facilitating changes which may cross boundaries. Ownership of audits remains with the lead auditor, usually a radiologist or other health professional. A departmental slide template is useful for presentations. Such a template would include:

• Title slide with the lead auditor and co-workers as well as the department (of use when audits are collated and shared with others within the organisation).

• Background. Why is this audit worth doing?

• Standards: National or local? Evidence-based?

• Results

• Discussion and conclusions. Do we meet the target? If not, why not?

• Recommendations. This is best kept brief and feasible and should include recommendation for re-audit, if appropriate. When tabulated as a last slide (see Fig.1, page 16), these recommendations may be coupled with a column for agreement by management and a third column for date of implementation (or reason for not implementing).

Statistical Considerations

The need for sample size estimates and for statistical comparisons when re-auditing will depend on the size and scope of the project. This is often necessary for national audits. Of more practical use is the display of results. Although while for most departmental and personal audits, simply achieving or not achieving the target is sufficient, it is helpful in national audits to benchmark departments giving median and quartiles (see Fig. 2, page 16). Anonymising departments is essential both for confidentiality and to ensure participation.

How Simple can we make Clinical Audit?

Whether you are designing an audit to support your appraisal folder, producing an audit programme for your department of radiology or advising on a national audit, the same principles apply:

• Choose a relevant topic which is either badly done and could be improved or well done but not recognised;

• Adopt a standard which is widely agreed and preferably evidence-based;

• Keep it simple. A complicated methodology will put off many otherwise enthusiastic participants; and,

• Be brutally brief with statistical analyses. The questions to be answered are simple and usually binary. Are patients given the best care and are they better?

The departmental annual audit programme should aim to:

• Be inclusive of all sections and professional groups;

• Mix presentations of projects with medical interest, often outcome audits, with those with wider interest, often process audits;

• Ensure all projects are interesting, relevant and preferably topical;

• Be produced three months before the first audit presentation;

• Be distributed to all presenters at least one month in advance; and

• Ideas for audit projects can be from members of staff, national guidelines (eg NICE, SIGN), patient safety agencies (e.g., National Patient Safety Agency NPSA) and patient liaison groups as well as publications.