Validating the use of veterans affairs tobacco health factors for assessing change in smoking status: Accuracy, availability, and approach

Anne C. Melzer, Erika A Pinsker, Barbara A Clothier, Siamak Noorbaloochi, Diana J Burgess, Elizabeth R Danan, Steven S. Fu

Research output: Contribution to journalArticlepeer-review

19 Scopus citations


Background: Accurate smoking status is key for research purposes, but can be costly and difficult to measure. Within the Veteran's Health Administration (VA), smoking status is recorded as part of routine care as "health factors" (HF) - fields that researchers can query through the electronic health record (EHR). Many researchers are interested in using these fields to track changes in smoking status over time, however the validity of this measure for assessing change is unknown. The primary goal of this project was to examine whether HFs can be used to accurately measure change in tobacco status over time, with secondary goals of assessing the optimum timeframe for assessment and variation in accuracy by site. Methods: Secondary analysis of the Veterans VICTORY study, a pragmatic smoking cessation randomized controlled trial conducted from 2009 to 2011. Eligible subjects were identified via the EHR using a past 90-day HF indicating current tobacco use (for example: "CURRENT SMOKER", "CURRENTLY USES TOBACCO"). Participants were surveyed at 1 year to determine prolonged smoking abstinence. We identified HFs for tobacco status within +/- 120 days of the follow-up survey mailing date and recorded the temporally closest HF. Among subjects with both measures, we compared the two for agreement using kappa statistics and concordance. Results: 1713 subjects (33%) had both follow-up survey and HF data, 1594 (31%) had only a survey response, 790 (15%) had only HF and 1026 (20%) had neither. For subjects with both measures, there was 90% concordance and moderate agreement (Kappa 0.48, 95%CI 0.41-0.55, Sensitivity 54.4, 95%CI 41.1-67.7, Specificity 94.3, 95%CI 87.5-100.0). Conclusions: We found high concordance but only moderate agreement by kappa statistics between HFs and survey data. The difference is likely accounted for by the natural history of quit attempts, in which patients cycle in and out of quit attempts. HFs appear to provide an accurate measure of population level quit behavior utilizing data collected in the course of clinical care.

Original languageEnglish (US)
Article number39
JournalBMC Medical Research Methodology
Issue number1
StatePublished - May 11 2018

Bibliographical note

Funding Information:
This work was funded by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, and Health Services Research and Development (IAB-05-303) and registered in (NCT00608426). EAP is supported by the Veterans Affairs Advanced Fellowship Program in Health Service Research. The funding organizations had no role in the design or execution of the study, the interpretation of the data, or in the writing of the manuscript.

Publisher Copyright:
© 2018 The Author(s).


  • Behavior change
  • Electronic health record
  • Health services
  • Methods
  • Smoking
  • Tobacco cessation


Dive into the research topics of 'Validating the use of veterans affairs tobacco health factors for assessing change in smoking status: Accuracy, availability, and approach'. Together they form a unique fingerprint.

Cite this