User perception of differences in recommender algorithms

Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, Joseph A. Konstan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

78 Scopus citations

Abstract

Recent developments in user evaluation of recommender systems have brought forth powerful new tools for understanding what makes recommendations effective and useful. We apply these methods to understand how users evaluate recommendation lists for the purpose of selecting an algorithm for finding movies. This paper reports on an experiment in which we asked users to compare lists produced by three common collaborative filtering algorithms on the dimensions of novelty, diversity, accuracy, satisfaction, and degree of personalization, and to select a recommender that they would like to use in the future. We find that satisfaction is negatively dependent on novelty and positively dependent on diversity in this setting, and that satisfaction predicts the user's final selection. We also compare users' subjective perceptions of recommendation properties with objective measures of those same characteristics. To our knowledge, this is the first study that applies modern survey design and analysis techniques to a within-subjects, direct comparison study of recommender algorithms.

Original languageEnglish (US)
Title of host publicationRecSys 2014 - Proceedings of the 8th ACM Conference on Recommender Systems
PublisherAssociation for Computing Machinery, Inc
Pages161-168
Number of pages8
ISBN (Electronic)9781450326681
DOIs
StatePublished - Oct 6 2014
Event8th ACM Conference on Recommender Systems, RecSys 2014 - Foster City, United States
Duration: Oct 6 2014Oct 10 2014

Publication series

NameRecSys 2014 - Proceedings of the 8th ACM Conference on Recommender Systems

Other

Other8th ACM Conference on Recommender Systems, RecSys 2014
CountryUnited States
CityFoster City
Period10/6/1410/10/14

Keywords

  • Recommender systems
  • User study

Fingerprint Dive into the research topics of 'User perception of differences in recommender algorithms'. Together they form a unique fingerprint.

Cite this