Assessment of modifications to evidence-based psychotherapies using administrative and chart note data from the US department of veterans affairs health care system

Background: The US Department of Veterans Affairs (VA) has over 15 years of experience in delivery of evidence-based psychotherapies (EBPs). This paper describes strategies for using clinical documentation and administrative data to understand adherence and modifications to EBPs for Posttraumatic Stress Disorder (PTSD). Methods: This study focused on two EBPs for PTSD, Cognitive Processing Therapy (CPT) and Prolonged Exposure (PE). The sample included VA therapists from across the US who provided CPT and PE and the patients they treated over a 1-year period. The data sources for this study were templated EBP chart notes and VA administrative data. We used a manual review of note content and administrative data rules to code therapy adherence and modifications in 7,297 EBP sessions for 1,257 patients seen by 182 therapists. Two trained coders rated each therapy note and resolved discrepancies through consensus. To contextualize and explain variation in adherence and modifications, we conducted brief 30–45-min semi-structured interviews with a purposive subsample of these therapists (n = 32). Findings: Combining manual chart review and administrative data allowed for identification of 11 types of modifications. Raters disagreed on adherence for 30% of notes. The disagreement stemmed from the presence of therapy modifications that were not clearly documented, necessitating the development of decision rules and strategies for modification coding. Both therapists and patients contributed to the variance in the extent to which different modifications occurred. Therapist interviews demonstrated therapist awareness of modifying the protocols in the ways identified through chart review. Conclusion: Healthcare systems can use data collected as part of routine care to understand how and when EBPs are modified but need to develop scalable strategies to document adaptations and modifications to EBPs in routine care.