Validating the use of veterans affairs tobacco health factors for assessing change in smoking status: Accuracy, availability, and approach

Anne C. Melzer, Erika A. Pinsker, Barbara Clothier, Siamak Noorbaloochi, Diana J. Burgess, Elisheva R. Danan, Steven S. Fu

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Accurate smoking status is key for research purposes, but can be costly and difficult to measure. Within the Veteran's Health Administration (VA), smoking status is recorded as part of routine care as "health factors" (HF) - fields that researchers can query through the electronic health record (EHR). Many researchers are interested in using these fields to track changes in smoking status over time, however the validity of this measure for assessing change is unknown. The primary goal of this project was to examine whether HFs can be used to accurately measure change in tobacco status over time, with secondary goals of assessing the optimum timeframe for assessment and variation in accuracy by site. Methods: Secondary analysis of the Veterans VICTORY study, a pragmatic smoking cessation randomized controlled trial conducted from 2009 to 2011. Eligible subjects were identified via the EHR using a past 90-day HF indicating current tobacco use (for example: "CURRENT SMOKER", "CURRENTLY USES TOBACCO"). Participants were surveyed at 1 year to determine prolonged smoking abstinence. We identified HFs for tobacco status within +/- 120 days of the follow-up survey mailing date and recorded the temporally closest HF. Among subjects with both measures, we compared the two for agreement using kappa statistics and concordance. Results: 1713 subjects (33%) had both follow-up survey and HF data, 1594 (31%) had only a survey response, 790 (15%) had only HF and 1026 (20%) had neither. For subjects with both measures, there was 90% concordance and moderate agreement (Kappa 0.48, 95%CI 0.41-0.55, Sensitivity 54.4, 95%CI 41.1-67.7, Specificity 94.3, 95%CI 87.5-100.0). Conclusions: We found high concordance but only moderate agreement by kappa statistics between HFs and survey data. The difference is likely accounted for by the natural history of quit attempts, in which patients cycle in and out of quit attempts. HFs appear to provide an accurate measure of population level quit behavior utilizing data collected in the course of clinical care.

Original languageEnglish (US)
Article number39
JournalBMC Medical Research Methodology
Volume18
Issue number1
DOIs
StatePublished - May 11 2018

Fingerprint

Veterans
Tobacco
Smoking
Electronic Health Records
Health
Tobacco Use
Track and Field
Research Personnel
Veterans Health
United States Department of Veterans Affairs
Smoking Cessation
Health Surveys
Natural History
Randomized Controlled Trials
Delivery of Health Care
Research
Population
Surveys and Questionnaires

Keywords

  • Behavior change
  • Electronic health record
  • Health services
  • Methods
  • Smoking
  • Tobacco cessation

PubMed: MeSH publication types

  • Journal Article
  • Research Support, U.S. Gov't, Non-P.H.S.
  • Validation Study

Cite this

@article{5a4f20bb73e6415a9130012bf86fb0f5,
title = "Validating the use of veterans affairs tobacco health factors for assessing change in smoking status: Accuracy, availability, and approach",
abstract = "Background: Accurate smoking status is key for research purposes, but can be costly and difficult to measure. Within the Veteran's Health Administration (VA), smoking status is recorded as part of routine care as {"}health factors{"} (HF) - fields that researchers can query through the electronic health record (EHR). Many researchers are interested in using these fields to track changes in smoking status over time, however the validity of this measure for assessing change is unknown. The primary goal of this project was to examine whether HFs can be used to accurately measure change in tobacco status over time, with secondary goals of assessing the optimum timeframe for assessment and variation in accuracy by site. Methods: Secondary analysis of the Veterans VICTORY study, a pragmatic smoking cessation randomized controlled trial conducted from 2009 to 2011. Eligible subjects were identified via the EHR using a past 90-day HF indicating current tobacco use (for example: {"}CURRENT SMOKER{"}, {"}CURRENTLY USES TOBACCO{"}). Participants were surveyed at 1 year to determine prolonged smoking abstinence. We identified HFs for tobacco status within +/- 120 days of the follow-up survey mailing date and recorded the temporally closest HF. Among subjects with both measures, we compared the two for agreement using kappa statistics and concordance. Results: 1713 subjects (33{\%}) had both follow-up survey and HF data, 1594 (31{\%}) had only a survey response, 790 (15{\%}) had only HF and 1026 (20{\%}) had neither. For subjects with both measures, there was 90{\%} concordance and moderate agreement (Kappa 0.48, 95{\%}CI 0.41-0.55, Sensitivity 54.4, 95{\%}CI 41.1-67.7, Specificity 94.3, 95{\%}CI 87.5-100.0). Conclusions: We found high concordance but only moderate agreement by kappa statistics between HFs and survey data. The difference is likely accounted for by the natural history of quit attempts, in which patients cycle in and out of quit attempts. HFs appear to provide an accurate measure of population level quit behavior utilizing data collected in the course of clinical care.",
keywords = "Behavior change, Electronic health record, Health services, Methods, Smoking, Tobacco cessation",
author = "Melzer, {Anne C.} and Pinsker, {Erika A.} and Barbara Clothier and Siamak Noorbaloochi and Burgess, {Diana J.} and Danan, {Elisheva R.} and Fu, {Steven S.}",
year = "2018",
month = "5",
day = "11",
doi = "10.1186/s12874-018-0501-2",
language = "English (US)",
volume = "18",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Validating the use of veterans affairs tobacco health factors for assessing change in smoking status

T2 - Accuracy, availability, and approach

AU - Melzer, Anne C.

AU - Pinsker, Erika A.

AU - Clothier, Barbara

AU - Noorbaloochi, Siamak

AU - Burgess, Diana J.

AU - Danan, Elisheva R.

AU - Fu, Steven S.

PY - 2018/5/11

Y1 - 2018/5/11

N2 - Background: Accurate smoking status is key for research purposes, but can be costly and difficult to measure. Within the Veteran's Health Administration (VA), smoking status is recorded as part of routine care as "health factors" (HF) - fields that researchers can query through the electronic health record (EHR). Many researchers are interested in using these fields to track changes in smoking status over time, however the validity of this measure for assessing change is unknown. The primary goal of this project was to examine whether HFs can be used to accurately measure change in tobacco status over time, with secondary goals of assessing the optimum timeframe for assessment and variation in accuracy by site. Methods: Secondary analysis of the Veterans VICTORY study, a pragmatic smoking cessation randomized controlled trial conducted from 2009 to 2011. Eligible subjects were identified via the EHR using a past 90-day HF indicating current tobacco use (for example: "CURRENT SMOKER", "CURRENTLY USES TOBACCO"). Participants were surveyed at 1 year to determine prolonged smoking abstinence. We identified HFs for tobacco status within +/- 120 days of the follow-up survey mailing date and recorded the temporally closest HF. Among subjects with both measures, we compared the two for agreement using kappa statistics and concordance. Results: 1713 subjects (33%) had both follow-up survey and HF data, 1594 (31%) had only a survey response, 790 (15%) had only HF and 1026 (20%) had neither. For subjects with both measures, there was 90% concordance and moderate agreement (Kappa 0.48, 95%CI 0.41-0.55, Sensitivity 54.4, 95%CI 41.1-67.7, Specificity 94.3, 95%CI 87.5-100.0). Conclusions: We found high concordance but only moderate agreement by kappa statistics between HFs and survey data. The difference is likely accounted for by the natural history of quit attempts, in which patients cycle in and out of quit attempts. HFs appear to provide an accurate measure of population level quit behavior utilizing data collected in the course of clinical care.

AB - Background: Accurate smoking status is key for research purposes, but can be costly and difficult to measure. Within the Veteran's Health Administration (VA), smoking status is recorded as part of routine care as "health factors" (HF) - fields that researchers can query through the electronic health record (EHR). Many researchers are interested in using these fields to track changes in smoking status over time, however the validity of this measure for assessing change is unknown. The primary goal of this project was to examine whether HFs can be used to accurately measure change in tobacco status over time, with secondary goals of assessing the optimum timeframe for assessment and variation in accuracy by site. Methods: Secondary analysis of the Veterans VICTORY study, a pragmatic smoking cessation randomized controlled trial conducted from 2009 to 2011. Eligible subjects were identified via the EHR using a past 90-day HF indicating current tobacco use (for example: "CURRENT SMOKER", "CURRENTLY USES TOBACCO"). Participants were surveyed at 1 year to determine prolonged smoking abstinence. We identified HFs for tobacco status within +/- 120 days of the follow-up survey mailing date and recorded the temporally closest HF. Among subjects with both measures, we compared the two for agreement using kappa statistics and concordance. Results: 1713 subjects (33%) had both follow-up survey and HF data, 1594 (31%) had only a survey response, 790 (15%) had only HF and 1026 (20%) had neither. For subjects with both measures, there was 90% concordance and moderate agreement (Kappa 0.48, 95%CI 0.41-0.55, Sensitivity 54.4, 95%CI 41.1-67.7, Specificity 94.3, 95%CI 87.5-100.0). Conclusions: We found high concordance but only moderate agreement by kappa statistics between HFs and survey data. The difference is likely accounted for by the natural history of quit attempts, in which patients cycle in and out of quit attempts. HFs appear to provide an accurate measure of population level quit behavior utilizing data collected in the course of clinical care.

KW - Behavior change

KW - Electronic health record

KW - Health services

KW - Methods

KW - Smoking

KW - Tobacco cessation

UR - http://www.scopus.com/inward/record.url?scp=85046893288&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046893288&partnerID=8YFLogxK

U2 - 10.1186/s12874-018-0501-2

DO - 10.1186/s12874-018-0501-2

M3 - Article

C2 - 29751746

AN - SCOPUS:85046893288

VL - 18

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

M1 - 39

ER -