Initial Validation of the Cybersecurity Concept Inventory: Pilot Testing and Expert Review

Spencer Offenberger, Geoffrey L. Herman, Peter Peterson, Alan T. Sherman, Enis Golaszewski, Travis Scheponik, Linda Oliva

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


We analyze expert review and student performance data to evaluate the validity of the Cybersecurity Concept Inventory (CCI) for assessing student knowledge of core cybersecurity concepts after a first course on the topic. A panel of 12 experts in cybersecurity reviewed the CCI, and 142 students from six different institutions took the CCI as a pilot test. The panel reviewed each item of the CCI and the overwhelming majority rated every item as measuring appropriate cybersecurity knowledge. We administered the CCI to students taking a first cybersecurity course either online or proctored by the course instructor. We applied classical test theory to evaluate the quality of the CCI. This evaluation showed that the CCI is sufficiently reliable for measuring student knowledge of cybersecurity and that the CCI may be too difficult as a whole. We describe the results of the expert review and the pilot test and provide recommendations for the continued improvement of the CCI.

Original languageEnglish (US)
Title of host publication2019 IEEE Frontiers in Education Conference, FIE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728117461
StatePublished - Oct 2019
Event49th IEEE Frontiers in Education Conference, FIE 2019 - Covington, United States
Duration: Oct 16 2019Oct 19 2019

Publication series

NameProceedings - Frontiers in Education Conference, FIE
ISSN (Print)1539-4565


Conference49th IEEE Frontiers in Education Conference, FIE 2019
Country/TerritoryUnited States

Bibliographical note

Funding Information:
VII. ACKNOWLEDGMENTS We thank all of the experts and pilot trial participants. This work was supported in part by the U.S. Department of Defense under CAE-R grants H98230-15-1-0294, H98230-15-1-0273, H98230-17-1-0349, and H98230-17-1-0347; and by the National Science Foundation under SFS grant 1241576 and DGE grant 1820531.

Publisher Copyright:
© 2019 IEEE.


  • Cybersecurity Assessment Tools (CATS) Project
  • Cybersecurity Concept Inventory (CCI)
  • Cybersecurity education
  • assessment validation


Dive into the research topics of 'Initial Validation of the Cybersecurity Concept Inventory: Pilot Testing and Expert Review'. Together they form a unique fingerprint.

Cite this