ICU Management & Practice, ICU Volume 5 - Issue 3 - Autumn 2005
Authors
On behalf of the NICE foundation
Nicolette F. de Keizer
Dept. of Medical Informatics,
Academic Medical Centre, Amsterdam,
The Netherlands
Evert de Jonge
Dept. of Intensive Care,
Academic Medical Centre, Amsterdam,
The Netherlands
The National Intensive Care Evaluation (NICE) is a quality system for Dutch intensive care, with registries including
effectiveness and efficiency measures, analyses, feedback and reports, and an infrastructure for evaluating the
implications of the registered data to improve intensive care treatment and organization.
Introduction
Intensive care medicine has evolved rapidly over the last decades. Survival chances of severely ill patients have been improved due to technological innovation, including new possibilities in surgical procedures, and the use of highly specialized medical and nursing personnel. The drawbacks are high costs and the psychological burden of this kind of intensive treatment for patients and their families. It is important to treat critically ill patients efficiently and to demonstrate the effectiveness of treatment.
Both nationally and internationally, several quality systems have been developed which include registries enabling quality assessment of intensive care. Examples can be found in Australia/New Zealand (ANZICS www.anzics.com.au), the United Kingdom (ICNARC www.icnarc.org) and the Netherlands.
Quality assessment can be defined as the critical appraisal of the measured results of a healthcare program as compared with the formulated objectives (Donabedian1988). Two aspects can be distinguished: efficiency and effectiveness. Commonly used outcome measures for effectiveness of Intensive Care are mortality, morbidity and quality of life. Commonly used outcome measures for efficiency are costs and length of stay.
A quality system not only contains one or more registers including effectiveness and efficiency measures, but also the associated analyses, feedback and reports, and a structure in which the participating intensivists discover and discuss the implications of the registered data to improve intensive care treatment and organization. In 1996 in the Netherlands, the National Intensive Care Evaluation (NICE) foundation was established by the professional group of intensivists to enable such a quality system for Dutch ICUs (www.stichting-nice.org/introductie. jsp?lang=en). Since 1996 the number of participating ICUs has grown continuously. Currently, more than 120,000 ICU admissions in 28 Dutch ICUs are included in the NICE database (see figure 1).
Quality Assurance and Improvement with
a National Database
The NICE national database is an important tool to monitor the quality of Dutch ICUs in terms of effectiveness and efficiency. Mortality constitutes one of the most important outcome measures of effectiveness in intensive care and is the starting point of NICE. Observed in-hospital mortality within an ICU population can be compared with the severity of illness adjusted expected mortality. Specially developed prognostic scoring models such as APACHE II (Knaus 1985) and SAPS II (Le Gall et al. 1993) can be used to calculate severity of illness adjusted expected mortality. The Standardized Mortality Ratio (SMR) is the ratio between the observed and the expected mortality and thereby provides a measure of effectiveness corrected for severity of illness. The national database provides participating ICUs the opportunity for benchmarking. This means that an ICU may compare the characteristics of its population and the outcome, e.g. in terms of the SMR, to the average of the other participating ICUs, a best performing ICU or with their own performance during an earlier time period. Benchmarking plays an important role in a quality system because observed differences between SMRs provide the opportunity for discussion and explanation of these differences.
National Database
Each participating ICU collects the minimal dataset for each admitted patient (www.stichting-nice.org/introductie. jsp?lang=en). It includes demographic data (e.g. age, gender), admission and discharge data (e.g. referring specialty, ICU and hospital admission/discharge date), diagnoses (e.g. reason for admission, comorbidities) and physiological data (e.g. blood pressure, body temperature). The minimal dataset enables calculation of five prognostic scoring systems: APACHE II (Knaus et al. 1985), SAPS II, MPM0II, MPM24II (Lemeshow et al. 1993) and LODS (Le Gall et al. 1996). These models quantify the severity of illness, and predict the chance of in-hospital mortality. Most items of the minimal dataset can be collected in the first 24 hours. Some variables must be collected on ICU or hospital discharge (see figure 2).
In addition to the minimal dataset, ICUs may voluntarily collect the Sequential Organ Failure Assessment (SOFA) score on a daily basis. SOFA quantifies failure or dysfunction of six organ systems based on a limited set of variables (Vincent et al. 1996). The SOFA score cannot be used to predict mortality, but may give insight into the course of the severity of illness.
Only a few ICUs have a Patient Data Management System from which the data for the national database can be automatically extracted. Most ICUs collect the data manually and enter them into a specific computer application. Intensivists, ICU nurses or both are responsible for data collection. Dependent on the local situation it takes 10 to 20 minutes to collect the minimal dataset per admission. Collection of the SOFA score takes 3 to 5 minutes per patient per day.
To protect the privacy of patients and the confidentiality of the participating hospitals, only encrypted patient names and numbers are imported into the database and each hospital is encoded by a hospital number known only to the secretary of NICE. In all NICE documents, descriptions of case mix and outcome of individual hospitals are presented anonymously.
From Data to Information
The large amount of data in the NICE database is processed and represented in such a way that it provides ICU physicians and managers with information that can be used to assess and improve the quality of care. Benchmarking is the most important method used by NICE to contextualise data. Each quarter, the participating ICUs receive a report on patient characteristics and outcome compared to the national average. A yearly report includes trend information on patient population and outcome for the years the ICU has participated in the NICE registry. Based on this report, changes in population and/or performance can be detected.
In addition to these static reports periodically provided by NICE, a web-based application, called NICE online (see the article in ICU Management Winter 2005), enables the authorized participants to do online analysis on their own data and compare these with several benchmarks, e.g. university vs. non-university hospitals, small (<600 admissions a year) vs. large (>=600 admissions a year) hospitals, or the same hospital for different time periods.
Once a year NICE organizes a meeting for all participating ICUs. During this meeting new initiatives are presented but most importantly, analyzed data are discussed. Prior to the meeting the participants receive a report with detailed analyses of some specific patient categories e.g. patients admitted with pneumonia or with heart valve surgery. Participants of the yearly meeting split up into subgroups and discuss the results of one specific patient group to explore possible explanations for differences in outcome. At the end of the meeting each subgroup summarizes their findings and presents ideas for new research. Figure 3 shows an example of a graph used during the annual meeting to discuss differences in outcome between ICUs.
Data Quality
Information from the NICE database is only meaningful if the quality of the data on which it is based is adequate. Adequate quality of data implies correct and complete data. Several procedures optimize the quality of the data (Arts et al. 2002). First extensive definitions and practical examples of each variable in the dataset are described in the NICE data dictionary (www.stichting- nice.org/introductie.jsp?lang=en) Secondly, at least two persons per participating ICU follow the NICE training program to learn the definitions of each variable and the potential pitfalls in collecting data. Training data definitions have been shown to reduce inter observer variability of the severity of illness scores (Arts et al.2003). Thirdly, each dataset is checked for missing and incorrect values before it is imported into the database. A report of observed data quality problems is sent to the ICU within a week after receiving the dataset. Fourthly, a biannual site visit is organized in which a random sample of patient records from the database is compared with the data found in the local medical records (de Jonge et al. 2003). Additionally, the organization which processes and maintains the NICE database has recently been NEN-ENISO9001: 2000 certified for their quality management system, assuring the quality of the registry.
Discussion, Conclusion and Future Plans
NICE has set up a successful voluntary registry of anonymous ICU data. This registry supports the improvement of quality of care by benchmarking performance measures of different ICUs. Prognostic scoring systems are used to adjust the outcome measure mortality for severity of illness. These prognostic scoring systems were developed on large USA/European databases some decade(s) ago and may therefore not be perfectly adjusted for the Dutch situation. Recalibration of these models to the Dutch situation is planned for the future. Furthermore NICE feels the need to further adjust for differences in case mix. Therefore a terminological system to describe the reason(s) for admission is under development and will be implemented as a pilot in two hospitals next year (Arts et al. 2002).
Current variables in the database are more relevant for medical management than nursing personnel. To pay
attention to outcome measures relevant to ICU nurses, we plan to extend the database with a TISS score module representing nursing workload. Contrary to the minimal data set which is collected once per admission, this TISS module will include a daily data collection such as the SOFA module.
Although the primary aim of NICE is quality improvement of Dutch intensive care medicine, the database is also valuable for epidemiological research (Bosman et al. 2003; de Jonge et al. 2003) and for planning clinical trials. The database can be used as a sampling frame to investigate the number of potentially eligible patients (Peelen et al. 2005 submitted).
NICE currently assesses the quality of intensive care, although quality assurance is not yet fully realized. Quality assurance can be defined as the critical appraisal of the measured results of intensive care to identify whether the formulated objectives are achieved and in case of discrepancies quality assurance implies a response to reduce the deviations from the objective (Donabedian 1988). In our annual meeting we are beginning to address what the causes of discrepancies between individual ICUs and the best performing ICU are, and to re-evaluate the performance of ICUs after implementation of improvements. Only when these steps are implemented will NICE become a complete quality assurance system.
