Data from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) are more accurate than administrative data for driving surgical quality improvement in hospitals, according to two new studies presented at the 2015 ACS NSQIP National Conference in Chicago.
The first study, conducted by Inova Health System in Virginia, sought to determine the validity of two sources of information for quality metrics — ACS NSQIP clinical data and administrative claims data used for billing purposes. ACS NSQIP is the leading nationally validated, risk-adjusted, outcomes-based programme to measure and improve the quality of surgical care in hospitals.
The study examined data on 157 hospital readmissions in the Inova Fairfax Hospital ACS NSQIP database. The patients had undergone general, endovascular and colorectal surgical procedures between January and December 2013 and were readmitted within 30 days of their operations.
Researchers reviewed medical records of all readmitted patients to determine the causes for readmission. The results from that review became the gold standard for hospital readmissions, with the top two reasons being surgical site infections (SSIs) and intestinal obstruction.
The researchers then assessed the accuracy of both the ACS NSQIP clinical data and the administrative claims data by comparing them both to the gold standard. Their analysis revealed the following:
"There are various reasons for the discrepancies, but one main cause is the lack of specificity within the diagnosis codes of administrative claims data," said lead author Amber Trickey, MSc, PhD, a surgery epidemiologist and biostatistician at Inova Fairfax Hospital. In contrast, ACS NSQIP codes are standardised, very specific and can identify the primary causes for readmissions.
In another study, Anna Weiss, MD, a surgical resident at the University of California San Diego and her colleagues examined if ACS NSQIP is superior to National Inpatient Sample (NIS), the largest U.S. administrative database of inpatient hospital stays. The NIS contains information on patients discharged from about 1,000 U.S. community hospitals.
The researchers pooled data from ACS NSQIP and NIS databases for the year 2010. Their analysis involved approximately 300,000 patients who had undergone 11 major surgical procedures, including coronary artery bypass, aortic valve replacement and appendectomy.
Dr. Weiss et al. compared the "c-statistic" for complications and mortality models built from the two datasets. The c-statistic is a measure of model discrimination where a value of 1 indicates that the model is perfect in discrimination between cases that experienced and did not experience the adverse event, and a value of 0.5 indicates that discrimination is at the level of chance. The team found that the values were consistently higher (closer to one) for ACS NSQIP data, compared with the NIS data.
The study results showed that unadjusted complication rates were higher in hospitals in the NIS for 7 out of the 11 operations, but unadjusted mortality rates in every procedure were lower in ACS NSQIP hospitals, in comparison with NIS.
Source and image credit: American College of Surgeons
The first study, conducted by Inova Health System in Virginia, sought to determine the validity of two sources of information for quality metrics — ACS NSQIP clinical data and administrative claims data used for billing purposes. ACS NSQIP is the leading nationally validated, risk-adjusted, outcomes-based programme to measure and improve the quality of surgical care in hospitals.
The study examined data on 157 hospital readmissions in the Inova Fairfax Hospital ACS NSQIP database. The patients had undergone general, endovascular and colorectal surgical procedures between January and December 2013 and were readmitted within 30 days of their operations.
Researchers reviewed medical records of all readmitted patients to determine the causes for readmission. The results from that review became the gold standard for hospital readmissions, with the top two reasons being surgical site infections (SSIs) and intestinal obstruction.
The researchers then assessed the accuracy of both the ACS NSQIP clinical data and the administrative claims data by comparing them both to the gold standard. Their analysis revealed the following:
- ACS NSQIP readmission reasons had a 71 percent agreement rate with the gold standard, compared with 61 percent for administrative claims data
- Notably, the NSQIP analysis showed that approximately 60 percent of the hospital readmissions are potentially preventable.
"There are various reasons for the discrepancies, but one main cause is the lack of specificity within the diagnosis codes of administrative claims data," said lead author Amber Trickey, MSc, PhD, a surgery epidemiologist and biostatistician at Inova Fairfax Hospital. In contrast, ACS NSQIP codes are standardised, very specific and can identify the primary causes for readmissions.
In another study, Anna Weiss, MD, a surgical resident at the University of California San Diego and her colleagues examined if ACS NSQIP is superior to National Inpatient Sample (NIS), the largest U.S. administrative database of inpatient hospital stays. The NIS contains information on patients discharged from about 1,000 U.S. community hospitals.
The researchers pooled data from ACS NSQIP and NIS databases for the year 2010. Their analysis involved approximately 300,000 patients who had undergone 11 major surgical procedures, including coronary artery bypass, aortic valve replacement and appendectomy.
Dr. Weiss et al. compared the "c-statistic" for complications and mortality models built from the two datasets. The c-statistic is a measure of model discrimination where a value of 1 indicates that the model is perfect in discrimination between cases that experienced and did not experience the adverse event, and a value of 0.5 indicates that discrimination is at the level of chance. The team found that the values were consistently higher (closer to one) for ACS NSQIP data, compared with the NIS data.
The study results showed that unadjusted complication rates were higher in hospitals in the NIS for 7 out of the 11 operations, but unadjusted mortality rates in every procedure were lower in ACS NSQIP hospitals, in comparison with NIS.
Source and image credit: American College of Surgeons
Latest Articles
healthmanagement, mortality rates, c-statistic, quality improvement, healthcare databases, readmissions
Data from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) are more accurate than administrative data for driving surgical quality improvement in hospitals, according to two new studies.
