Partitioning-based clustering for Web document categorization

Daniel Boley, Maria Gini, Robert Gross, Eui Hong Han, Kyle Hastings, George Karypis, Vipin Kumar, Bamshad Mobasher, Jerome Moore

Research output: Contribution to journalArticlepeer-review

168 Scopus citations

Abstract

Clustering techniques have been used by many intelligent software agents in order to retrieve, filter, and categorize documents available on the World Wide Web. Clustering is also useful in extracting salient features of related Web documents to automatically formulate queries and search for other similar documents on the Web. Traditional clustering algorithms either use a priori knowledge of document structures to define a distance or similarity among these documents, or use probabilistic techniques such as Bayesian classification. Many of these traditional algorithms, however, falter when the dimensionality of the feature space becomes high relative to the size of the document space. In this paper, we introduce two new clustering algorithms that can effectively cluster documents, even in the presence of a very high dimensional feature space. These clustering techniques, which are based on generalizations of graph partitioning, do not require pre-specified ad hoc distance functions, and are capable of automatically discovering document similarities or associations. We conduct several experiments on real Web data using various feature selection heuristics, and compare our clustering schemes to standard distance-based techniques, such as hierarchical agglomeration clustering, and Bayesian classification methods, such as AutoClass.

Original languageEnglish (US)
Pages (from-to)329-341
Number of pages13
JournalDecision Support Systems
Volume27
Issue number3
DOIs
StatePublished - Dec 1999

Bibliographical note

Funding Information:
This work was supported in part by Army Research Office contract DA/DAAG55-98-1-0441, by NSF Grant 115-98 11229, by Army High Performance Computing Research Center cooperative agreement number DAAH04-95-2-0003/contract number DAAH04-95-C-0008, the content of which does not necessarily reflect the position or the policy of the government, and no official endorsement should be inferred. Additional support was provided by the IBM Partnership Award, and by the IBM SUR equipment grant. Access to computing facilities was provided by Minnesota Supercomputer Institute.

Fingerprint Dive into the research topics of 'Partitioning-based clustering for Web document categorization'. Together they form a unique fingerprint.

Cite this