Evaluation of hierarchical clustering algorithms for document datasets

Ying Zhao, George Karypis

Research output: Contribution to conferencePaper

346 Scopus citations

Abstract

Fast and high-quality document clustering algorithms play an important role in providing intuitive navigation and browsing mechanisms by organizing large amounts of information into a small number of meaningful clusters. In particular, hierarchical clustering solutions provide a view of the data at different levels of granularity, making them ideal for people to visualize and interactively explore large document collections. In this paper we evaluate different partitional and agglomerative approaches for hierarchical clustering. Our experimental evaluation showed that partitional algorithms always lead to better clustering solutions than agglomerative algorithms, which suggests that partitional clustering algorithms are well-suited for clustering large document datasets due to not only their relatively low computational requirements, but also comparable or even better clustering performance. We present a new class of clustering algorithms called constrained agglomerative algorithms that combine the features of both partitional and agglomerative algorithms. Our experimental results showed that they consistently lead to better hierarchical solutions than agglomerative or partitional algorithms alone.

Original languageEnglish (US)
Pages515-524
Number of pages10
DOIs
StatePublished - 2002
EventProceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM 2002) - McLean, VA, United States
Duration: Nov 4 2002Nov 9 2002

Other

OtherProceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM 2002)
CountryUnited States
CityMcLean, VA
Period11/4/0211/9/02

Keywords

  • Agglomerative clustering
  • Hierarchical clustering
  • Partitional clustering

Fingerprint Dive into the research topics of 'Evaluation of hierarchical clustering algorithms for document datasets'. Together they form a unique fingerprint.

  • Cite this

    Zhao, Y., & Karypis, G. (2002). Evaluation of hierarchical clustering algorithms for document datasets. 515-524. Paper presented at Proceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM 2002), McLean, VA, United States. https://doi.org/10.1145/584792.584877