Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation

Margrét Sigmarsdóttir, Melanie M.Domenech Rodríguez, Abigail Gewirtz, Laura Rains, Jolle Tjaden, Marion S. Forgatch

    Research output: Contribution to journalArticlepeer-review


    Background: Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners’ competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners’ fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period. Methods: Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor’s two fidelity teams and the fidelity teams in seven implementation sites. Results: Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams. Conclusions: This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.

    Original languageEnglish (US)
    Article number576
    JournalBMC Health Services Research
    Issue number1
    StatePublished - Dec 2023

    Bibliographical note

    Funding Information:
    The authors want to acknowledge and thank the participating GenerationPMTO sites for their contribution and development of this work. Thank you for giving us permission to use these data, which made the study possible. We also thank the PMTO therapists around the world for providing the services to parents and children in need and leaders for making this possible. Finally, we thank the FIMP leaders and coders in all the fidelity teams for their quality work.

    Publisher Copyright:
    © 2023, The Author(s).


    • Evidence-based parenting program
    • Fidelity
    • GenerationPMTO
    • Implementation


    Dive into the research topics of 'Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation'. Together they form a unique fingerprint.

    Cite this