Modified interactive Q-learning for attenuating the impact of model misspecification with treatment effect heterogeneity

Yuan Zhang, David M. Vock, Megan E. Patrick, Thomas A. Murray

Research output: Contribution to journalArticlepeer-review


A sequential multiple assignment randomized trial, which incorporates multiple stages of randomization, is a popular approach for collecting data to inform personalized and adaptive treatments. There is an extensive literature on statistical methods to analyze data collected in sequential multiple assignment randomized trials and estimate the optimal dynamic treatment regime. Q-learning with linear regression is widely used for this purpose due to its ease of implementation. However, model misspecification is a common problem with this approach, and little attention has been given to the impact of model misspecification when treatment effects are heterogeneous across subjects. This article describes the integrative impact of two possible types of model misspecification related to treatment effect heterogeneity: omitted early-stage treatment effects in late-stage main effect model, and violated linearity assumption between pseudo-outcomes and predictors despite non-linearity arising from the optimization operation. The proposed method, aiming to deal with both types of misspecification concomitantly, builds interactive models into modified parametric Q-learning with Murphy’s regret function. Simulations show that the proposed method is robust to both sources of model misspecification. The proposed method is applied to a two-stage sequential multiple assignment randomized trial with embedded tailoring aimed at reducing binge drinking in first-year college students.

Original languageEnglish (US)
Pages (from-to)2240-2253
Number of pages14
JournalStatistical methods in medical research
Issue number11
StatePublished - Nov 2023

Bibliographical note

Publisher Copyright:
© The Author(s) 2023.


  • Dynamic treatment regime
  • heterogeneous treatment effect
  • main effect model misspecification
  • omitted variable bias
  • sequential multiple assignment randomized trial

PubMed: MeSH publication types

  • Journal Article


Dive into the research topics of 'Modified interactive Q-learning for attenuating the impact of model misspecification with treatment effect heterogeneity'. Together they form a unique fingerprint.

Cite this