Automated assembly of optimally spaced and balanced paired comparisons: Controlling order effects

Mark L. Davison, Andrew Zieffler, Julio Cabrera, Stacy R. Karl, Harvey S. Cohen

Research output: Contribution to journalArticlepeer-review

Abstract

To control order effects in questionnaires containing paired comparisons, Ross (1934) described an optimal ordering of the pairings. The pairs can also be balanced so that every stimulus appears equal numbers of times as the first and the second member of a pair. First, we describe and illustrate the optimally spaced, balanced ordering of pairings. Then we show how the optimally spaced, balanced order can be used to implement a matrix-sampling design or a fully incomplete design when the number of stimuli n is so large that respondents cannot reasonably be expected to judge all n(n - 1)/2 pairs. The algorithm for balancing and optimally spacing the list of pairs is described.

Original languageEnglish (US)
Pages (from-to)753-764
Number of pages12
JournalBehavior Research Methods
Volume44
Issue number3
DOIs
StatePublished - Sep 2012

Keywords

  • Forced choice tasks
  • Matrix sampling
  • Multidimensional scaling
  • Order effects
  • Paired comparisons
  • Planned missingness
  • Preference judgments
  • Questionnaire design
  • Similarity judgments

Fingerprint Dive into the research topics of 'Automated assembly of optimally spaced and balanced paired comparisons: Controlling order effects'. Together they form a unique fingerprint.

Cite this