Methods matter: A multi-trait multi-method analysis of student behavior

Faith G. Miller, Austin H. Johnson, Huihui Yu, Sandra M. Chafouleas, D. Betsy McCoach, T. Chris Riley-Tillman, Gregory A. Fabiano, Megan E. Welsh

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Reliable and valid data form the foundation for evidence-based practices, yet surprisingly few studies on school-based behavioral assessments have been conducted which implemented one of the most fundamental approaches to construct validation, the multitrait-multimethod matrix (MTMM). To this end, the current study examined the reliability and validity of data derived from three commonly utilized school-based behavioral assessment methods: Direct Behavior Rating – Single Item Scales, systematic direct observations, and behavior rating scales on three common constructs of interest: academically engaged, disruptive, and respectful behavior. Further, this study included data from different sources including student self-report, teacher report, and external observers. A total of 831 students in grades 3–8 and 129 teachers served as participants. Data were analyzed using bivariate correlations of the MTMM, as well as single and multi-level structural equation modeling. Results suggested the presence of strong methods effects for all the assessment methods utilized, as well as significant relations between constructs of interest. Implications for practice and future research are discussed.

Original languageEnglish (US)
Pages (from-to)53-72
Number of pages20
JournalJournal of school psychology
StatePublished - Jun 2018

Bibliographical note

Funding Information:
Preparation of this article was supported by funding provided by the Institute of Education Sciences, U.S. Department of Education (R324A110017: PI, Chafouleas). Opinions expressed herein do not necessarily reflect the position of the U.S. Department of Education, and such endorsements should not be inferred.


  • Assessment
  • Behavior
  • Direct Behavior Rating
  • MTMM
  • Rating scales
  • Systematic direct observation
  • Validity


Dive into the research topics of 'Methods matter: A multi-trait multi-method analysis of student behavior'. Together they form a unique fingerprint.

Cite this