Converting Historical Syndromic Surveillance Data for Use in the Biosense 2.0 Cloud

BACKGROUND: In 2013 the North Dakota Department of Health (NDDoH) Syndromic Surveillance program transitioned an automated electronic syndromic surveillance system called RedBat^TM to BioSense 2.0. To maintain continuity of data, ten years of historical RedBat^TM data was uploaded to the BioSense 2.0 cloud environment. To determine if syndromes between the two systems were similar, Influenza-like illness (ILI) data were compared between RedBat^TMand BioSense 2.0.

METHODS: For the historical uploads, all syndromic data in the RedBat^TM system from January 1, 2003 forward were converted to comma separated value (CSV) file format using Microsoft Excel 2010. One file was created for each facility. The Excel data files were reviewed and de-identified by removing or modifying fields containing identifiable facility codes or HIPAA-protected information. We then matched corresponding HL7 fields for each variable that could be contained within the HL7 message structure. This information was used to create a mapper in Orion Health’s Rhapsody© Integration Engine to create HL7 message files. Initially, a small batch of messages from one historical site was sent to the BioSense 2.0 testing environment, of which 80% were accepted. However, when the full set of messages was sent, only 40% uploaded.  An error due to a transposition of month and date for the visit date variable in the Rhapsody mapper was identified and corrected. The large batch was sent again, with 100% acceptance.  To ensure data completeness, BioSense 2.0 was used to calculate ILI by visit date and compared with the flu-like illness reported in RedBat^TM.

RESULTS: Uploading historical data to BioSense 2.0 allowed us to develop a baseline using ten years of data that we could compare using both surveillance interfaces. For ILI, analysis using the BioSense 2.0 definition for ILI was combined in an Excel data set along with the data on flu-like illness computed by RedBat^TMfor comparison. The two data sets were found to be identical.

CONCLUSIONS: Transferring data from RedBat^TM to BioSense 2.0 allowed the NDDoH to put previous work to current use in the BioSense 2.0 environment. We established that rates for ILI calculated in BioSense 2.0 would be comparable to output from our previous system, allowing us continue to use ILI data coming into BioSense 2.0 as part of our syndromic and ILI  surveillance programs.  Continued monitoring of ILI and other syndromes will be needed as more data is uploaded into BioSense 2.0.