Validation of Using Multiple Imputation for Missing Transmission Risk Data in HIV/AIDS Surveillance

BACKGROUND: In 2010, 25% of newly diagnosed HIV cases reported to the New York City Department of Health and Mental Hygiene (NYC DOHMH) had unknown transmission risk, as defined by the DOHMH transmission risk hierarchy.  Complete risk ascertainment is necessary to accurately distribute resources, monitor trends, and target prevention interventions.  Multiple statistical methods are available for handling missing data.  Multiple imputation (MI) is one method which, unlike traditional methods, should preserve the inherent relationships between HIV risk and demographic and other characteristics.  The Centers for Disease Control and Prevention (CDC) encourage local jurisdictions to apply MI to HIV surveillance data.  The MI method has not been validated with HIV surveillance data.  We therefore aimed to assess the use of MI for transmission risk classification using the NYC HIV Surveillance Registry.           

METHODS: The CDC MI method, which replaces each missing risk value with a set of plausible values, was used for adult (≥13) HIV diagnoses in 2008 that were reported to the DOHMH through 09/30/2009.  Imputed transmission risk values were compared with values found through re-investigations for a subset of cases.  Re-abstracted values were assumed to represent “true” values and were collected through: 1) a search of STD clinic electronic medical records or 2) special case investigations, routine data quality assurance activities, or subsequent provider reporting.  Percentages and Cohen’s kappa statistics were used to measure agreement between the imputed and re-abstracted risk values.

RESULTS: Out of 3,801 new adult HIV diagnoses in NYC in 2008, a total of 1,102 (29%) had no transmission risk information reported as of 09/30/2009.  Re-investigation identified classifiable risk information for 105 cases, 17 out of 98 STD records searched (17.3%) and 88 from additional sources.  Of the 105 cases, imputed and re-abstracted risk values were in agreement for 71.6% of those classified as men who have sex with men, 45.0% of those with injection drug use history, and 75.9% of those categorized as heterosexual.  The kappa coefficient for agreement was 0.52 (95% CI = 0.38 to 0.65).    

CONCLUSIONS: Multiply imputed transmission risk values had moderate agreement with re-abstracted values in this analysis.  This suggests that MI could be a useful method for improving risk ascertainment, but that model inputs may need further refinement for the NYC epidemic.  Risk has been re-abstracted for a small number of cases to date, and agreement will be further assessed as additional information is collected.