Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening: A multisite investigation

Austin H. Johnson, Faith G. Miller, Sandra M. Chafouleas, Megan E. Welsh, T. Chris Riley-Tillman, Gregory Fabiano

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as "at-risk" or "not-at-risk." The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating-Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points.

Original languageEnglish (US)
Pages (from-to)39-57
Number of pages19
JournalJournal of School Psychology
Volume54
DOIs
StatePublished - Feb 1 2016

Fingerprint

Students
rating
student
ROC Curve
Area Under Curve
recipient
school grade
Sensitivity and Specificity
teacher
school
performance
Group
time
Problem Behavior

Keywords

  • Behavioral risk
  • Direct behavior rating
  • Screening

Cite this

Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening : A multisite investigation. / Johnson, Austin H.; Miller, Faith G.; Chafouleas, Sandra M.; Welsh, Megan E.; Chris Riley-Tillman, T.; Fabiano, Gregory.

In: Journal of School Psychology, Vol. 54, 01.02.2016, p. 39-57.

Research output: Contribution to journalArticle

Johnson, Austin H. ; Miller, Faith G. ; Chafouleas, Sandra M. ; Welsh, Megan E. ; Chris Riley-Tillman, T. ; Fabiano, Gregory. / Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening : A multisite investigation. In: Journal of School Psychology. 2016 ; Vol. 54. pp. 39-57.
@article{37d79ee7304245a5b05468409b6cf707,
title = "Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening: A multisite investigation",
abstract = "The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as {"}at-risk{"} or {"}not-at-risk.{"} The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating-Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points.",
keywords = "Behavioral risk, Direct behavior rating, Screening",
author = "Johnson, {Austin H.} and Miller, {Faith G.} and Chafouleas, {Sandra M.} and Welsh, {Megan E.} and {Chris Riley-Tillman}, T. and Gregory Fabiano",
year = "2016",
month = "2",
day = "1",
doi = "10.1016/j.jsp.2015.10.001",
language = "English (US)",
volume = "54",
pages = "39--57",
journal = "Journal of School Psychology",
issn = "0022-4405",
publisher = "Elsevier BV",

}

TY - JOUR

T1 - Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening

T2 - A multisite investigation

AU - Johnson, Austin H.

AU - Miller, Faith G.

AU - Chafouleas, Sandra M.

AU - Welsh, Megan E.

AU - Chris Riley-Tillman, T.

AU - Fabiano, Gregory

PY - 2016/2/1

Y1 - 2016/2/1

N2 - The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as "at-risk" or "not-at-risk." The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating-Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points.

AB - The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as "at-risk" or "not-at-risk." The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating-Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points.

KW - Behavioral risk

KW - Direct behavior rating

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=84958550801&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958550801&partnerID=8YFLogxK

U2 - 10.1016/j.jsp.2015.10.001

DO - 10.1016/j.jsp.2015.10.001

M3 - Article

C2 - 26790702

AN - SCOPUS:84958550801

VL - 54

SP - 39

EP - 57

JO - Journal of School Psychology

JF - Journal of School Psychology

SN - 0022-4405

ER -