![[ Visit Client Website ]](images/banner.png)
BACKGROUND: In September 2006 the Massachusetts Department of Public Health Bureau of Infectious Disease (BID) implemented its electronic disease surveillance system (MAVEN). Following five years of development and use, the system required a comprehensive, systematic evaluation framework and protocol to ensure data quality and assess performance. A team composed of staff from research and evaluation, epidemiology and surveillance and informatics units was tasked with its construction and implementation. Project objectives included a) development and application of a framework for routine evaluation and b) dissemination of results to local public health partners to promote data quality, improve transparency and enhance communication of surveillance requirements. The project’s initial phase focused on the development of performance measures for case investigation completeness at the state and local levels.
METHODS: For a subset of reportable conditions (n = 39) a set of "key indicators" essential to investigation were selected from an extract of 75,000 case records reported between 2007 and 2010. Date values associated with initial case investigation and completion facilitated the calculation of measures for investigation timeliness. Tabular summary reports accompanied by individual case line-lists were constructed and distributed to local public health (i.e. those responsible for case investigation in their jurisdiction) for their review. A uniform, weighted performance index which accounts for disease volume and population variations among jurisdictions was applied.
RESULTS: Completeness of key variables (as a measure of appropriate case investigation and follow-up) varied widely by jurisdiction (city/town) and by disease. Early feedback signaled the need for clear communication of key variables between state and local public health. Data quality issues were uncovered through inspection of the results and corrected through a combination of system modifications, training and improved communication.
CONCLUSIONS: The construction of the surveillance system evaluation framework required extensive and detailed analysis from an array of committed and thoughtful staff. Significant development time was spent clarifying misunderstood or conflicting procedures and requirements with respect to disease investigation. The initial phase of this project provides an invaluable baseline for subsequent performance evaluation. Key challenges to this project were disagreements regarding definitions for completeness and the inconsistent communication of these definitions with local public health. Overall, the response from local public health partners was very positive regarding their town/city-specific reports, with some concerns and suggestions.