Bayesian sparse covariance decomposition with a graphical structure

Lin Zhang, Abhra Sarkar, Bani K. Mallick

Research output: Contribution to journalArticle

Abstract

We consider the problem of estimating covariance matrices of a particular structure that is a summation of a low-rank component and a sparse component. This is a general covariance structure encountered in multiple statistical models including factor analysis and random effects models, where the low-rank component relates to the correlations among variables coming from the latent factors or random effects and the sparse component displays the correlations of the remaining residuals. We propose a Bayesian method for estimating the covariance matrices of such structures by representing the covariance model in the form of a factor model with an unknown number of latent factors. We introduce binary indicators for factor selection and rank estimation for the low-rank component, combined with a Bayesian lasso method for the estimation of the sparse component. Simulation studies show that our method can recover the rank as well as the sparsity of the two respective components. We further extend our method to a latent-factor Markov graphical model, with a focus on the sparse conditional graphical model of the residuals as well as selecting the number of factors. We show through simulations that our Bayesian model can successfully recover both the number of latent factors and the Markov graphical model of the residuals.

Original languageEnglish (US)
Pages (from-to)493-510
Number of pages18
JournalStatistics and Computing
Volume26
Issue number1-2
DOIs
StatePublished - Jan 1 2016

Fingerprint

Decomposition
Decompose
Graphical Models
Covariance matrix
Markov Model
Rank Estimation
Conditional Model
Lasso
Random Effects Model
Factor Models
Covariance Structure
Multiple Models
Bayesian Methods
Factor analysis
Bayesian Model
Factor Analysis
Model Analysis
Graphics
Random Effects
Sparsity

Keywords

  • Bayesian graphical lasso
  • Covariance estimation
  • Factor analysis
  • Factor graphical model
  • Low-rank-plus-sparse decomposition

Cite this

Bayesian sparse covariance decomposition with a graphical structure. / Zhang, Lin; Sarkar, Abhra; Mallick, Bani K.

In: Statistics and Computing, Vol. 26, No. 1-2, 01.01.2016, p. 493-510.

Research output: Contribution to journalArticle

Zhang, Lin ; Sarkar, Abhra ; Mallick, Bani K. / Bayesian sparse covariance decomposition with a graphical structure. In: Statistics and Computing. 2016 ; Vol. 26, No. 1-2. pp. 493-510.
@article{c8e962fab4b54df69e509dfe3dd47dab,
title = "Bayesian sparse covariance decomposition with a graphical structure",
abstract = "We consider the problem of estimating covariance matrices of a particular structure that is a summation of a low-rank component and a sparse component. This is a general covariance structure encountered in multiple statistical models including factor analysis and random effects models, where the low-rank component relates to the correlations among variables coming from the latent factors or random effects and the sparse component displays the correlations of the remaining residuals. We propose a Bayesian method for estimating the covariance matrices of such structures by representing the covariance model in the form of a factor model with an unknown number of latent factors. We introduce binary indicators for factor selection and rank estimation for the low-rank component, combined with a Bayesian lasso method for the estimation of the sparse component. Simulation studies show that our method can recover the rank as well as the sparsity of the two respective components. We further extend our method to a latent-factor Markov graphical model, with a focus on the sparse conditional graphical model of the residuals as well as selecting the number of factors. We show through simulations that our Bayesian model can successfully recover both the number of latent factors and the Markov graphical model of the residuals.",
keywords = "Bayesian graphical lasso, Covariance estimation, Factor analysis, Factor graphical model, Low-rank-plus-sparse decomposition",
author = "Lin Zhang and Abhra Sarkar and Mallick, {Bani K.}",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/s11222-014-9540-7",
language = "English (US)",
volume = "26",
pages = "493--510",
journal = "Statistics and Computing",
issn = "0960-3174",
publisher = "Springer Netherlands",
number = "1-2",

}

TY - JOUR

T1 - Bayesian sparse covariance decomposition with a graphical structure

AU - Zhang, Lin

AU - Sarkar, Abhra

AU - Mallick, Bani K.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - We consider the problem of estimating covariance matrices of a particular structure that is a summation of a low-rank component and a sparse component. This is a general covariance structure encountered in multiple statistical models including factor analysis and random effects models, where the low-rank component relates to the correlations among variables coming from the latent factors or random effects and the sparse component displays the correlations of the remaining residuals. We propose a Bayesian method for estimating the covariance matrices of such structures by representing the covariance model in the form of a factor model with an unknown number of latent factors. We introduce binary indicators for factor selection and rank estimation for the low-rank component, combined with a Bayesian lasso method for the estimation of the sparse component. Simulation studies show that our method can recover the rank as well as the sparsity of the two respective components. We further extend our method to a latent-factor Markov graphical model, with a focus on the sparse conditional graphical model of the residuals as well as selecting the number of factors. We show through simulations that our Bayesian model can successfully recover both the number of latent factors and the Markov graphical model of the residuals.

AB - We consider the problem of estimating covariance matrices of a particular structure that is a summation of a low-rank component and a sparse component. This is a general covariance structure encountered in multiple statistical models including factor analysis and random effects models, where the low-rank component relates to the correlations among variables coming from the latent factors or random effects and the sparse component displays the correlations of the remaining residuals. We propose a Bayesian method for estimating the covariance matrices of such structures by representing the covariance model in the form of a factor model with an unknown number of latent factors. We introduce binary indicators for factor selection and rank estimation for the low-rank component, combined with a Bayesian lasso method for the estimation of the sparse component. Simulation studies show that our method can recover the rank as well as the sparsity of the two respective components. We further extend our method to a latent-factor Markov graphical model, with a focus on the sparse conditional graphical model of the residuals as well as selecting the number of factors. We show through simulations that our Bayesian model can successfully recover both the number of latent factors and the Markov graphical model of the residuals.

KW - Bayesian graphical lasso

KW - Covariance estimation

KW - Factor analysis

KW - Factor graphical model

KW - Low-rank-plus-sparse decomposition

UR - http://www.scopus.com/inward/record.url?scp=84953363806&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84953363806&partnerID=8YFLogxK

U2 - 10.1007/s11222-014-9540-7

DO - 10.1007/s11222-014-9540-7

M3 - Article

VL - 26

SP - 493

EP - 510

JO - Statistics and Computing

JF - Statistics and Computing

SN - 0960-3174

IS - 1-2

ER -