Cost of a Sustainable Annual Validation Process to Ensure Credibility of State HAI Reporting

BACKGROUND:  Few states conduct annual validation to ensure reliable quality in healthcare-associated infection (HAI) reporting, partly due to perceived resource burden.  In 2009, Washington State established an annual validation protocol for central line associated bloodstream infection (CLABSI) reporting.  This ensures hospital processes consistently meet our pre-defined data quality standards.  There are established international standards for process validation methods.  Our annual validation and related data cleaning activities have proven sustainable within existing staffing and ensures our publicly reported HAI rates remain trustworthy.

METHODS:  From 2008-August 2013, all 65 state-licensed acute-care hospitals with intensive care units caring for central-line patients reported; after 2013 our network grew to 93.  They report through CDC’s National Healthcare Safety Network (NHSN).  Sensitivity and specificity standards defined by our department are enforced using a validation protocol consistent with the International Standards Organization (ISO) 2859 double-sampling method.  Our validation includes two components: “internal” (hospitals assess their own sensitivity and specificity under our direction) and “external” (department staff makes on-site visits, independently reviewing primary records).  Beyond validation, we perform data cleaning when downloading NHSN data biweekly.  This resolves data gaps, inconsistencies or denominator anomalies, and also helps prioritize hospitals for “external” visits.  From 2009-2014, calendared activity logs, expense records, our 2009 pilot study, and feedback from hospital infection control professionals, were reviewed for annual validation cost.  Results are expressed in full time equivalents (FTE).

RESULTS:  All hospitals in our network participate in annual validation; each received ≥1 “external” visit in 2010-2013.  Hospital staff time for “internal” validation takes <6 hours/year; department time to review results takes approximately one hour.  Hospital staff time for “external” validation includes record gathering and 30-minute briefing and debriefing.  Department staff time for visits averaged ~0.16-0.19 FTE/year (0.11-0.13 FTE/year for record review).  89% of visits took one day; only the most complex hospitals required two days.  In addition to FTE, travel expenses were approximately $3,400/year.  Data cleaning (examining biweekly data, phoning hospitals to resolve issues, contacting high-rate outliers to understand context prior to posting public reports) requires ~0.18-0.21 FTE.

CONCLUSIONS:  Washington State’s annual validation protocol (fully described in a publicly available reference manual), has proven practical, sustainable and scalable since 2009.  The validation approach is consistent with ISO methods.  Its credibility was confirmed through independent review by quality professionals of national standing.  Our experience and reference manual have been requested for use by other countries.