BACKGROUND: State health departments rely on the Behavioral Risk Factor Surveillance System (BRFSS) for monitoring population health. This population-based telephone survey includes required core questions and each state can add questions to meet their needs. In Colorado, demand for state-added questions exceeds the limits set by the Colorado Department of Public Health and Environment (CDPHE) to keep the survey at about 20 minutes – a limit imposed to reduce respondent burden and maximize response rates. CDPHE conducted and evaluated two online survey pilot projects to assess whether alternative data collection systems could help meet data demands.
METHODS: In April 2016, CDPHE purchased a sample representative of Colorado’s population from an online panel of a marketing research company. An online survey of 50 BRFSS questions was administered. In 2017, CDPHE is conducting a BRFSS online follow-up survey of new behavioral health questions. All BRFSS cell phone respondents are invited to participate and offered an incentive. Those who agree are sent a survey link via text message. Sampling weights for both surveys were calculated using raking ratio estimation based on known population characteristics. The evaluation compared the panel survey and BRFSS prevalence estimates and compared both online surveys and BRFSS on system attributes, including simplicity, cost-effectiveness, flexibility, data quality, acceptability, representativeness, and timeliness.
RESULTS: The panel survey sample was younger (median age 45 vs. 57 years) and more likely to have higher household income (51% vs. 36% earning $75,000+) and to be non-Hispanic White (84% vs. 78%) compared with the BRFSS survey sample. The panel survey had lower per respondent costs ($15 vs. $50), higher question-specific response rates, and higher weighted prevalence estimates for sensitive questions (e.g., binge drinking, 24% vs. 18%) compared with the BRFSS. The panel survey data were quickly collected, immediately available, and, in general, comparable with BRFSS estimates. Both online surveys were flexible, acceptable (e.g., faster to complete, 5 vs. 22 minutes), and efficient in that information on several demographic factors already existed and did not have to be asked. The BRFSS follow-up survey is preferred over the panel survey for ongoing collection of surveillance data because of sampling methodology.
CONCLUSIONS: Both survey methodologies show promise as alternative data collection methods to meet the demands for population-based data in Colorado. The preferred method depends on the intended use of the data. Recommendations for future online surveys (e.g., sampling, incentives, and options to compel response) have been drafted.