Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge

Teresa Ober; Madelynn Denner; Yikai Lu; Cheng Liu; Ying Cheng

Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge

Teresa Ober
, Madelynn Denner
, Yikai Lu
, Cheng Liu
, Ying Cheng

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We examine associations between subjective (self-reported) and objective (derived from behaviors or performance) indicators of effort and ability and whether the format of a computerized assessment (adaptive v. non-adaptive) moderates these associations. Participants (N=219; Mage=19.83) were randomly assigned to one of two conditions in which they completed either an adaptive or a non-adaptive version of a 20-item computerized assessment. They then self-reported their effort and the number of items they believe they answered correctly on the assessment. Self-reported effort was not significantly correlated with the behavioral indicator. However, self-reported ability was positively correlated with participants' test score, even after controlling for test condition. Test condition appeared to moderate the association between subjective and objective estimates of ability with participants with higher ability in the non-adaptive condition estimating a greater number of items answered correctly. Further work should examine whether effects of test condition differ for certain student subgroups.

Original language	English (US)
Title of host publication	International Collaboration toward Educational Innovation for All
Subtitle of host publication	Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022
Editors	Clark Chinn, Edna Tan, Carol Chan, Yael Kali
Publisher	International Society of the Learning Sciences (ISLS)
Pages	1513-1516
Number of pages	4
ISBN (Electronic)	9781737330653
State	Published - 2022
Externally published	Yes
Event	16th International Conference of the Learning Sciences, ICLS 2022 - Virtual, Online, Japan Duration: Jun 6 2022 → Jun 10 2022

Publication series

Name	Proceedings of International Conference of the Learning Sciences, ICLS
ISSN (Print)	1814-9316

Conference

Conference	16th International Conference of the Learning Sciences, ICLS 2022
Country/Territory	Japan
City	Virtual, Online
Period	6/6/22 → 6/10/22

Bibliographical note

Publisher Copyright:
© ISLS.

Keywords

adaptive test
computerized assessment
effort
self-evaluation
subjective v. objective estimates
test mode

Find It

Find It at U of M Libraries

Cite this

Ober, T., Denner, M., Lu, Y., Liu, C., & Cheng, Y. (2022). Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge. In C. Chinn, E. Tan, C. Chan, & Y. Kali (Eds.), International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022 (pp. 1513-1516). (Proceedings of International Conference of the Learning Sciences, ICLS). International Society of the Learning Sciences (ISLS).

Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge. / Ober, Teresa; Denner, Madelynn; Lu, Yikai et al.
International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022. ed. / Clark Chinn; Edna Tan; Carol Chan; Yael Kali. International Society of the Learning Sciences (ISLS), 2022. p. 1513-1516 (Proceedings of International Conference of the Learning Sciences, ICLS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ober, T, Denner, M, Lu, Y, Liu, C & Cheng, Y 2022, Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge. in C Chinn, E Tan, C Chan & Y Kali (eds), International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022. Proceedings of International Conference of the Learning Sciences, ICLS, International Society of the Learning Sciences (ISLS), pp. 1513-1516, 16th International Conference of the Learning Sciences, ICLS 2022, Virtual, Online, Japan, 6/6/22.

Ober T, Denner M, Lu Y, Liu C, Cheng Y. Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge. In Chinn C, Tan E, Chan C, Kali Y, editors, International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022. International Society of the Learning Sciences (ISLS). 2022. p. 1513-1516. (Proceedings of International Conference of the Learning Sciences, ICLS).

Ober, Teresa ; Denner, Madelynn ; Lu, Yikai et al. / Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge. International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022. editor / Clark Chinn ; Edna Tan ; Carol Chan ; Yael Kali. International Society of the Learning Sciences (ISLS), 2022. pp. 1513-1516 (Proceedings of International Conference of the Learning Sciences, ICLS).

@inproceedings{6342520664ac4fc3bbf49cbb2b43f763,

title = "Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge",

abstract = "We examine associations between subjective (self-reported) and objective (derived from behaviors or performance) indicators of effort and ability and whether the format of a computerized assessment (adaptive v. non-adaptive) moderates these associations. Participants (N=219; Mage=19.83) were randomly assigned to one of two conditions in which they completed either an adaptive or a non-adaptive version of a 20-item computerized assessment. They then self-reported their effort and the number of items they believe they answered correctly on the assessment. Self-reported effort was not significantly correlated with the behavioral indicator. However, self-reported ability was positively correlated with participants' test score, even after controlling for test condition. Test condition appeared to moderate the association between subjective and objective estimates of ability with participants with higher ability in the non-adaptive condition estimating a greater number of items answered correctly. Further work should examine whether effects of test condition differ for certain student subgroups.",

keywords = "adaptive test, computerized assessment, effort, self-evaluation, subjective v. objective estimates, test mode",

author = "Teresa Ober and Madelynn Denner and Yikai Lu and Cheng Liu and Ying Cheng",

note = "Publisher Copyright: {\textcopyright} ISLS.; 16th International Conference of the Learning Sciences, ICLS 2022 ; Conference date: 06-06-2022 Through 10-06-2022",

year = "2022",

language = "English (US)",

series = "Proceedings of International Conference of the Learning Sciences, ICLS",

publisher = "International Society of the Learning Sciences (ISLS)",

pages = "1513--1516",

editor = "Clark Chinn and Edna Tan and Carol Chan and Yael Kali",

booktitle = "International Collaboration toward Educational Innovation for All",

}

TY - GEN

T1 - Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge

AU - Ober, Teresa

AU - Denner, Madelynn

AU - Lu, Yikai

AU - Liu, Cheng

AU - Cheng, Ying

N1 - Publisher Copyright: © ISLS.

PY - 2022

Y1 - 2022

N2 - We examine associations between subjective (self-reported) and objective (derived from behaviors or performance) indicators of effort and ability and whether the format of a computerized assessment (adaptive v. non-adaptive) moderates these associations. Participants (N=219; Mage=19.83) were randomly assigned to one of two conditions in which they completed either an adaptive or a non-adaptive version of a 20-item computerized assessment. They then self-reported their effort and the number of items they believe they answered correctly on the assessment. Self-reported effort was not significantly correlated with the behavioral indicator. However, self-reported ability was positively correlated with participants' test score, even after controlling for test condition. Test condition appeared to moderate the association between subjective and objective estimates of ability with participants with higher ability in the non-adaptive condition estimating a greater number of items answered correctly. Further work should examine whether effects of test condition differ for certain student subgroups.

AB - We examine associations between subjective (self-reported) and objective (derived from behaviors or performance) indicators of effort and ability and whether the format of a computerized assessment (adaptive v. non-adaptive) moderates these associations. Participants (N=219; Mage=19.83) were randomly assigned to one of two conditions in which they completed either an adaptive or a non-adaptive version of a 20-item computerized assessment. They then self-reported their effort and the number of items they believe they answered correctly on the assessment. Self-reported effort was not significantly correlated with the behavioral indicator. However, self-reported ability was positively correlated with participants' test score, even after controlling for test condition. Test condition appeared to moderate the association between subjective and objective estimates of ability with participants with higher ability in the non-adaptive condition estimating a greater number of items answered correctly. Further work should examine whether effects of test condition differ for certain student subgroups.

KW - adaptive test

KW - computerized assessment

KW - effort

KW - self-evaluation

KW - subjective v. objective estimates

KW - test mode

UR - https://www.scopus.com/pages/publications/85145774790

UR - https://www.scopus.com/pages/publications/85145774790#tab=citedBy

M3 - Conference contribution

AN - SCOPUS:85145774790

T3 - Proceedings of International Conference of the Learning Sciences, ICLS

SP - 1513

EP - 1516

BT - International Collaboration toward Educational Innovation for All

A2 - Chinn, Clark

A2 - Tan, Edna

A2 - Chan, Carol

A2 - Kali, Yael

PB - International Society of the Learning Sciences (ISLS)

T2 - 16th International Conference of the Learning Sciences, ICLS 2022

Y2 - 6 June 2022 through 10 June 2022

ER -

Comparing Subjective and Objective Estimates of Effort and Performance on Adaptive v. non-Adaptive Computerized Tests of Statistics Knowledge

Abstract

Publication series

Conference

Bibliographical note

Keywords

Find It

Other files and links

Fingerprint

Cite this