The Pitfalls and Use of Outcome Indicators, Taking Post-Surgical Infections as an Example

Various partners in the health system are increasingly turning to indicators to reveal the quality of the care given. The path of 'outcome measurement' is however, strewn with pitfalls, and it is essential to be aware of these in order to interpret and use such data. Prof. Troillet illustrated this through the example of a Swiss multi-centric programme for monitoring surgical site infections.

Prof. Troillet stressed that the programme concerns outcome indicators and not process indicators. The focus is on figures for mortality, patient satisfaction, infection rates, SSIs. Surgical site infections (SSIs) are infections occurring within 30 days (12 months in case of foreign bodies) at the level of the surgical incision or involving the organs or spaces that were opened or manipulated during the intervention. SSIs are the most frequent nosocomial infection in Switzerland. 

SSIs are too common a complication, the rate in Switzerland is 5.6% equating to 33,500 patients per year. The surveillance of SSI provides data that can both inform and influence practice to minimise the risk of SSI. Data collection results should be followed by analysis and interpretation and then a feedback and discussion session. This in turn should result in corrective measures. We must strive to bridge the gap between what we know and what we do. 

Data must be broken down in order to make comparisons. Prof. Troillet cited several rules for surveillance and comparison of results. The first rule to remember is that there are rates of infection by type of intervention; there are different risks categories for SSIs among the same interventions, not linked to quality of care (case-mix). Secondly, data should be broken down into rates by intervention and by risk category and thirdly not all surgical site infections are identical. A lot of SSIs occur after the patients have left the hospital and this should be taken into account. The fourth and fifth rules ask the questions, is there post-discharge follow-up? And is there a validation system in place? It is hard to decipher whether rates within hospitals and countries are so different or whether some countries are doing a better job of finding them. 

According to Troillet, one major pitfall lies in the comparison and interpretation of such data. There are many reasons for observed differences and these differences can be artificial and true differences. Artificial differences are case definitions, case detection (during hospital stay, post discharge) and the aggregation of small samples. True differences are quality of care and risk factors (procedures and patients case-mix). 

Prof. Troillet concluded his informative presentation by discussing the challenging questions that lie ahead. The first question is what to do with the results? They should be given to surgeons and management should make sure they are used. The second question is whether inter-hospital comparisons are meaningful or not? Troillet believes that under several conditions and with caution they can be useful. The third and final question concerns public reporting- is it a good idea? Although it might induce gaming and be counterproductive it is nonetheless important to get prepared!