Quantifying the potential role of unmeasured confounders: The example of influenza vaccination

Background: The validity of non-randomized studies using healthcare databases is often challenged because they lack information on potentially important confounders, such as functional health status and socioeconomic status. In a study quantifying the effects of influenza vaccination among community-dwelling elderly we assessed whether additional information on not routinely available covariates was indeed associated with exposure to influenza vaccination and could, therefore, have led to residual confounding in healthcare databases. Methods: We randomly selected 500 persons aged 65 years and older from the computerized Utrecht General Practitioner database. Information on exposure status and on demographics, co-morbidity status, prior healthcare use and medication use was extracted from the database. A questionnaire was used to obtain additional information on not routinely available risk factors [e.g. functional health status (SF-20), smoking status and alcohol consumption]. Missing data from the questionnaire was imputed and multivariable logistic regression analysis was applied to quantify the influence of covariates on the prediction of exposure to influenza vaccination. Within an existing dataset the potential impact of functional health status on the relation between influenza vaccination and mortality was simulated. Results: We obtained questionnaire data from 365 of 500 (73%) subjects. The model including routinely available data from the database appeared accurate in predicting exposure to influenza vaccination (c-statistic 0.86, 95% CI: 0.82-0.89). Functional health status was the only additional characteristic measured with the questionnaire that was not similar in vaccinated and unvaccinated subjects. However, extending the multivariable regression model with functional health status did not significantly improve the prediction of exposure to influenza vaccination, nor did it affect the relation between influenza vaccination and mortality. Conclusion: The potential for unmeasured confounding on the association between influenza vaccination and health outcomes as quantified in healthcare databases seems small for non-randomized intervention studies within extensive and reliable databases.