Mining low-support discriminative patterns from dense and high-dimensional data

Gang Fang, Gaurav Pandey, Wen Wang, Manish Gupta, Michael Steinbach, Vipin Kumar

Research output: Contribution to journalArticlepeer-review

32 Scopus citations

Abstract

Discriminative patterns can provide valuable insights into data sets with class labels, that may not be available from the individual features or the predictive models built using them. Most existing approaches work efficiently for sparse or low-dimensional data sets. However, for dense and high-dimensional data sets, they have to use high thresholds to produce the complete results within limited time, and thus, may miss interesting low-support patterns. In this paper, we address the necessity of trading off the completeness of discriminative pattern discovery with the efficient discovery of low-support discriminative patterns from such data sets. We propose a family of antimonotonic measures named SupMaxK that organize the set of discriminative patterns into nested layers of subsets, which are progressively more complete in their coverage, but require increasingly more computation. In particular, the member of SupMaxK with K = 2, named SupMaxPair, is suitable for dense and high-dimensional data sets. Experiments on both synthetic data sets and a cancer gene expression data set demonstrate that there are low-support patterns that can be discovered using SupMaxPair but not by existing approaches. Furthermore, we show that the low-support discriminative patterns that are only discovered using SupMaxPair from the cancer gene expression data set are statistically significant and biologically relevant. This illustrates the complementarity of SupMaxPair to existing approaches for discriminative pattern discovery. The codes and data set for this paper are available at http://vk.cs.umn.edu/SMP/..

Original languageEnglish (US)
Article number5645630
Pages (from-to)279-294
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume24
Issue number2
DOIs
StatePublished - 2012

Keywords

  • Association analysis
  • biomarker discovery
  • discriminative pattern mining
  • permutation test

Fingerprint Dive into the research topics of 'Mining low-support discriminative patterns from dense and high-dimensional data'. Together they form a unique fingerprint.

Cite this