TY - JOUR
T1 - Effects of calibration sample size and item bank size on ability estimation in computerized adaptive testing
AU - Şahin, Alper
AU - Weiss, David J.
N1 - Publisher Copyright:
© 2015 EDAM.
PY - 2015/12
Y1 - 2015/12
N2 - This study aimed to investigate the effects of calibration sample size and item bank size on examinee ability estimation in computerized adaptive testing (CAT). For this purpose, a 500-item bank pre-calibrated using the three-parameter logistic model with 10,000 examinees was simulated. Calibration samples of varying sizes (150, 250, 350, 500, 750, 1,000, 2,000, 3,000, and 5,000) were selected from the parent sample, and item banks that represented small (100) and medium size (200 and 300) banks were drawn from the 500-item bank. Items in these banks were recalibrated using the drawn samples, and their estimated parameters were used in post-hoc simulations to re-estimate ability parameters for the simulated 10,000 examinees. The findings showed that ability estimates in CAT are robust against fluctuations in item parameter estimation and that accurate ability parameter estimates can be obtained with a calibration sample of 150 examinees. Moreover, a 200-item bank pre-calibrated with as few as 150 examinees can be used for some purposes in CAT as long as it has sufficient information at targeted ability levels.
AB - This study aimed to investigate the effects of calibration sample size and item bank size on examinee ability estimation in computerized adaptive testing (CAT). For this purpose, a 500-item bank pre-calibrated using the three-parameter logistic model with 10,000 examinees was simulated. Calibration samples of varying sizes (150, 250, 350, 500, 750, 1,000, 2,000, 3,000, and 5,000) were selected from the parent sample, and item banks that represented small (100) and medium size (200 and 300) banks were drawn from the 500-item bank. Items in these banks were recalibrated using the drawn samples, and their estimated parameters were used in post-hoc simulations to re-estimate ability parameters for the simulated 10,000 examinees. The findings showed that ability estimates in CAT are robust against fluctuations in item parameter estimation and that accurate ability parameter estimates can be obtained with a calibration sample of 150 examinees. Moreover, a 200-item bank pre-calibrated with as few as 150 examinees can be used for some purposes in CAT as long as it has sufficient information at targeted ability levels.
KW - Ability estimation accuracy
KW - Calibration sample size
KW - Computerized adaptive testing
KW - Item response theory
KW - Pretest item calibration
UR - http://www.scopus.com/inward/record.url?scp=84953375121&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84953375121&partnerID=8YFLogxK
U2 - 10.12738/estp.2015.6.0102
DO - 10.12738/estp.2015.6.0102
M3 - Article
AN - SCOPUS:84953375121
SN - 1303-0485
VL - 15
SP - 1585
EP - 1595
JO - Kuram ve Uygulamada Egitim Bilimleri
JF - Kuram ve Uygulamada Egitim Bilimleri
IS - 6
ER -