Two peers are better than one

Aggregating peer reviews for computing assignments is surprisingly accurate

Ken Reily, Pam Ludford Finnerty, Loren G Terveen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

43 Citations (Scopus)

Abstract

Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes.

Original languageEnglish (US)
Title of host publicationGROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work
Pages115-124
Number of pages10
DOIs
StatePublished - Dec 1 2009
Event2009 ACM SIGCHI International Conference on Supporting Group Work, GROUP'09 - Sanibel Island, FL, United States
Duration: May 10 2009May 13 2009

Other

Other2009 ACM SIGCHI International Conference on Supporting Group Work, GROUP'09
CountryUnited States
CitySanibel Island, FL
Period5/10/095/13/09

Fingerprint

Students
Feedback
Computer programming
Software engineering
Open source software

Keywords

  • Collaboration
  • Education
  • Peer review

Cite this

Reily, K., Finnerty, P. L., & Terveen, L. G. (2009). Two peers are better than one: Aggregating peer reviews for computing assignments is surprisingly accurate. In GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work (pp. 115-124) https://doi.org/10.1145/1531674.1531692

Two peers are better than one : Aggregating peer reviews for computing assignments is surprisingly accurate. / Reily, Ken; Finnerty, Pam Ludford; Terveen, Loren G.

GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work. 2009. p. 115-124.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Reily, K, Finnerty, PL & Terveen, LG 2009, Two peers are better than one: Aggregating peer reviews for computing assignments is surprisingly accurate. in GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work. pp. 115-124, 2009 ACM SIGCHI International Conference on Supporting Group Work, GROUP'09, Sanibel Island, FL, United States, 5/10/09. https://doi.org/10.1145/1531674.1531692
Reily K, Finnerty PL, Terveen LG. Two peers are better than one: Aggregating peer reviews for computing assignments is surprisingly accurate. In GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work. 2009. p. 115-124 https://doi.org/10.1145/1531674.1531692
Reily, Ken ; Finnerty, Pam Ludford ; Terveen, Loren G. / Two peers are better than one : Aggregating peer reviews for computing assignments is surprisingly accurate. GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work. 2009. pp. 115-124
@inproceedings{6afca752d918466ab795163121119cd0,
title = "Two peers are better than one: Aggregating peer reviews for computing assignments is surprisingly accurate",
abstract = "Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes.",
keywords = "Collaboration, Education, Peer review",
author = "Ken Reily and Finnerty, {Pam Ludford} and Terveen, {Loren G}",
year = "2009",
month = "12",
day = "1",
doi = "10.1145/1531674.1531692",
language = "English (US)",
isbn = "9781605585000",
pages = "115--124",
booktitle = "GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work",

}

TY - GEN

T1 - Two peers are better than one

T2 - Aggregating peer reviews for computing assignments is surprisingly accurate

AU - Reily, Ken

AU - Finnerty, Pam Ludford

AU - Terveen, Loren G

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes.

AB - Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes.

KW - Collaboration

KW - Education

KW - Peer review

UR - http://www.scopus.com/inward/record.url?scp=77950479734&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950479734&partnerID=8YFLogxK

U2 - 10.1145/1531674.1531692

DO - 10.1145/1531674.1531692

M3 - Conference contribution

SN - 9781605585000

SP - 115

EP - 124

BT - GROUP'09 - Proceedings of the 2009 ACM SIGCHI International Conference on Supporting Group Work

ER -