Classification, ranking, and top-K stability of recommendation algorithms

Gediminas Adomavicius, Jingjing Zhang

Research output: Contribution to journalArticlepeer-review

23 Scopus citations


Recommendation stability measures the extent to which a recommendation algorithm provides predictions that are consistent with each other. Several approaches have been proposed in prior work to defining, measuring, and improving the stability of recommendation algorithms. Previous studies have focused primarily on understanding and evaluating recommendation stability in prediction-oriented settings, i.e., recommendation settings where it is crucial to provide the precise prediction of a user's preference rating for an item. However, the research literature has been largely silent on the topic of recommendation stability in other important types of settings, such as classification-oriented (i.e., where it is important to accurately classify the item as relevant versus irrelevant, without having to quantify the user's preference precisely), ranking-oriented (i.e., where it is important to provide accurate relative ranking of items to users), or top-K oriented (i.e., where it is important to suggest K items that are most appealing to the user). Therefore, this paper builds on prior work by generalizing the notion of stability to a broader set of recommendation settings and developing corresponding stability metrics. The paper also provides a comprehensive empirical analysis of classification, ranking, and top-K stability performance of popular recommender algorithms on real-world rating data sets under a variety of settings.

Original languageEnglish (US)
Pages (from-to)129-147
Number of pages19
JournalINFORMS Journal on Computing
Issue number1
StatePublished - Dec 1 2016


  • Classification stability
  • Evaluation of recommender systems
  • Ranking stability
  • Recommender systems
  • Top-K stability


Dive into the research topics of 'Classification, ranking, and top-K stability of recommendation algorithms'. Together they form a unique fingerprint.

Cite this