A Penalized Likelihood Method for Classification With Matrix-Valued Predictors

Aaron J. Molstad, Adam J. Rothman

Research output: Contribution to journalArticle

Abstract

We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker product decomposition. Our penalties encourage pairs of response category mean matrix estimators to have equal entries and also encourage zeros in the precision matrix estimator. To compute our estimators, we use a blockwise coordinate descent algorithm. To update the optimization variables corresponding to response category mean matrices, we use an alternating minimization algorithm that takes advantage of the Kronecker structure of the precision matrix. We show that our method can outperform relevant competitors in classification, even when our modeling assumptions are violated. We analyze three real datasets to demonstrate our method’s applicability. Supplementary materials, including an R package implementing our method, are available online.

Original languageEnglish (US)
Pages (from-to)11-22
Number of pages12
JournalJournal of Computational and Graphical Statistics
Volume28
Issue number1
DOIs
StatePublished - Jan 2 2019

Fingerprint

Penalized Likelihood
Likelihood Methods
Predictors
Estimator
Coordinate Descent
Kronecker Product
Descent Algorithm
Discriminant Analysis
Penalty
Update
Decompose
Optimization
Zero
Modeling
Estimate
Demonstrate

Keywords

  • Alternating minimization algorithm
  • Classification
  • Penalized likelihood

Cite this

A Penalized Likelihood Method for Classification With Matrix-Valued Predictors. / Molstad, Aaron J.; Rothman, Adam J.

In: Journal of Computational and Graphical Statistics, Vol. 28, No. 1, 02.01.2019, p. 11-22.

Research output: Contribution to journalArticle

@article{a72a988ca55742e8821f91dfe95e331f,
title = "A Penalized Likelihood Method for Classification With Matrix-Valued Predictors",
abstract = "We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker product decomposition. Our penalties encourage pairs of response category mean matrix estimators to have equal entries and also encourage zeros in the precision matrix estimator. To compute our estimators, we use a blockwise coordinate descent algorithm. To update the optimization variables corresponding to response category mean matrices, we use an alternating minimization algorithm that takes advantage of the Kronecker structure of the precision matrix. We show that our method can outperform relevant competitors in classification, even when our modeling assumptions are violated. We analyze three real datasets to demonstrate our method’s applicability. Supplementary materials, including an R package implementing our method, are available online.",
keywords = "Alternating minimization algorithm, Classification, Penalized likelihood",
author = "Molstad, {Aaron J.} and Rothman, {Adam J.}",
year = "2019",
month = "1",
day = "2",
doi = "10.1080/10618600.2018.1476249",
language = "English (US)",
volume = "28",
pages = "11--22",
journal = "Journal of Computational and Graphical Statistics",
issn = "1061-8600",
publisher = "American Statistical Association",
number = "1",

}

TY - JOUR

T1 - A Penalized Likelihood Method for Classification With Matrix-Valued Predictors

AU - Molstad, Aaron J.

AU - Rothman, Adam J.

PY - 2019/1/2

Y1 - 2019/1/2

N2 - We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker product decomposition. Our penalties encourage pairs of response category mean matrix estimators to have equal entries and also encourage zeros in the precision matrix estimator. To compute our estimators, we use a blockwise coordinate descent algorithm. To update the optimization variables corresponding to response category mean matrices, we use an alternating minimization algorithm that takes advantage of the Kronecker structure of the precision matrix. We show that our method can outperform relevant competitors in classification, even when our modeling assumptions are violated. We analyze three real datasets to demonstrate our method’s applicability. Supplementary materials, including an R package implementing our method, are available online.

AB - We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker product decomposition. Our penalties encourage pairs of response category mean matrix estimators to have equal entries and also encourage zeros in the precision matrix estimator. To compute our estimators, we use a blockwise coordinate descent algorithm. To update the optimization variables corresponding to response category mean matrices, we use an alternating minimization algorithm that takes advantage of the Kronecker structure of the precision matrix. We show that our method can outperform relevant competitors in classification, even when our modeling assumptions are violated. We analyze three real datasets to demonstrate our method’s applicability. Supplementary materials, including an R package implementing our method, are available online.

KW - Alternating minimization algorithm

KW - Classification

KW - Penalized likelihood

UR - http://www.scopus.com/inward/record.url?scp=85052133610&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052133610&partnerID=8YFLogxK

U2 - 10.1080/10618600.2018.1476249

DO - 10.1080/10618600.2018.1476249

M3 - Article

AN - SCOPUS:85052133610

VL - 28

SP - 11

EP - 22

JO - Journal of Computational and Graphical Statistics

JF - Journal of Computational and Graphical Statistics

SN - 1061-8600

IS - 1

ER -