High-dimensional structured feature screening using binary markov random fields

Jie Liu, Chunming Zhang, Catherine McCarty, Peggy Peissig, Elizabeth Burnside, David Page

Research output: Contribution to journalConference article

3 Scopus citations

Abstract

Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudolikelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on realworld data and synthetic data.

Original languageEnglish (US)
Pages (from-to)712-721
Number of pages10
JournalJournal of Machine Learning Research
Volume22
StatePublished - Jan 1 2012
Event15th International Conference on Artificial Intelligence and Statistics, AISTATS 2012 - La Palma, Spain
Duration: Apr 21 2012Apr 23 2012

    Fingerprint

Cite this

Liu, J., Zhang, C., McCarty, C., Peissig, P., Burnside, E., & Page, D. (2012). High-dimensional structured feature screening using binary markov random fields. Journal of Machine Learning Research, 22, 712-721.