Multiclass sparse discriminant analysis

Qing Mai, Yi Yang, Hui Zou

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

In recent years several sparse linear discriminant analysis methods have been proposed for high-dimensional classification and variable selection. Most of these proposals focus on binary classification and are not directly applicable to multiclass classification problems. Some sparse discriminant analysis methods can handle multiclass classification problems, but their theoretical justifications remain unknown. In this paper, we propose a new multiclass sparse discriminant analysis method that estimates all discriminant directions simultaneously. We show that when applied to the binary case our proposal yields a classification direction that is equivalent to those attained by two successful binary sparse linear discriminant analysis methods, providing a unification of these seemingly unrelated proposals. Our method can be solved by an efficient algorithm that is implemented in an open R package msda available from CRAN. We offer theoretical justification of our method by establishing a variable selection consistency result and finding rates of convergence under the ultrahigh dimensionality setting. We further demonstrate the empirical performance of our method with simulations and data.

Original languageEnglish (US)
Pages (from-to)97-111
Number of pages15
JournalStatistica Sinica
Volume29
Issue number1
DOIs
StatePublished - Jan 1 2019

Fingerprint

Multi-class
Discriminant Analysis
Multi-class Classification
Variable Selection
Justification
Classification Problems
Binary
Binary Classification
Discriminant analysis
Unification
Discriminant
Dimensionality
Rate of Convergence
High-dimensional
Efficient Algorithms
Unknown
Estimate
Demonstrate
Simulation

Keywords

  • Discriminant analysis
  • High dimensional data
  • Multiclass classification
  • Rates of convergence
  • Variable selection

Cite this

Multiclass sparse discriminant analysis. / Mai, Qing; Yang, Yi; Zou, Hui.

In: Statistica Sinica, Vol. 29, No. 1, 01.01.2019, p. 97-111.

Research output: Contribution to journalArticle

Mai, Qing ; Yang, Yi ; Zou, Hui. / Multiclass sparse discriminant analysis. In: Statistica Sinica. 2019 ; Vol. 29, No. 1. pp. 97-111.
@article{f09c42cbfd5f40e2ba13557401430ec5,
title = "Multiclass sparse discriminant analysis",
abstract = "In recent years several sparse linear discriminant analysis methods have been proposed for high-dimensional classification and variable selection. Most of these proposals focus on binary classification and are not directly applicable to multiclass classification problems. Some sparse discriminant analysis methods can handle multiclass classification problems, but their theoretical justifications remain unknown. In this paper, we propose a new multiclass sparse discriminant analysis method that estimates all discriminant directions simultaneously. We show that when applied to the binary case our proposal yields a classification direction that is equivalent to those attained by two successful binary sparse linear discriminant analysis methods, providing a unification of these seemingly unrelated proposals. Our method can be solved by an efficient algorithm that is implemented in an open R package msda available from CRAN. We offer theoretical justification of our method by establishing a variable selection consistency result and finding rates of convergence under the ultrahigh dimensionality setting. We further demonstrate the empirical performance of our method with simulations and data.",
keywords = "Discriminant analysis, High dimensional data, Multiclass classification, Rates of convergence, Variable selection",
author = "Qing Mai and Yi Yang and Hui Zou",
year = "2019",
month = "1",
day = "1",
doi = "10.5705/ss.202016.0117",
language = "English (US)",
volume = "29",
pages = "97--111",
journal = "Statistica Sinica",
issn = "1017-0405",
publisher = "Institute of Statistical Science",
number = "1",

}

TY - JOUR

T1 - Multiclass sparse discriminant analysis

AU - Mai, Qing

AU - Yang, Yi

AU - Zou, Hui

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In recent years several sparse linear discriminant analysis methods have been proposed for high-dimensional classification and variable selection. Most of these proposals focus on binary classification and are not directly applicable to multiclass classification problems. Some sparse discriminant analysis methods can handle multiclass classification problems, but their theoretical justifications remain unknown. In this paper, we propose a new multiclass sparse discriminant analysis method that estimates all discriminant directions simultaneously. We show that when applied to the binary case our proposal yields a classification direction that is equivalent to those attained by two successful binary sparse linear discriminant analysis methods, providing a unification of these seemingly unrelated proposals. Our method can be solved by an efficient algorithm that is implemented in an open R package msda available from CRAN. We offer theoretical justification of our method by establishing a variable selection consistency result and finding rates of convergence under the ultrahigh dimensionality setting. We further demonstrate the empirical performance of our method with simulations and data.

AB - In recent years several sparse linear discriminant analysis methods have been proposed for high-dimensional classification and variable selection. Most of these proposals focus on binary classification and are not directly applicable to multiclass classification problems. Some sparse discriminant analysis methods can handle multiclass classification problems, but their theoretical justifications remain unknown. In this paper, we propose a new multiclass sparse discriminant analysis method that estimates all discriminant directions simultaneously. We show that when applied to the binary case our proposal yields a classification direction that is equivalent to those attained by two successful binary sparse linear discriminant analysis methods, providing a unification of these seemingly unrelated proposals. Our method can be solved by an efficient algorithm that is implemented in an open R package msda available from CRAN. We offer theoretical justification of our method by establishing a variable selection consistency result and finding rates of convergence under the ultrahigh dimensionality setting. We further demonstrate the empirical performance of our method with simulations and data.

KW - Discriminant analysis

KW - High dimensional data

KW - Multiclass classification

KW - Rates of convergence

KW - Variable selection

UR - http://www.scopus.com/inward/record.url?scp=85054981583&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054981583&partnerID=8YFLogxK

U2 - 10.5705/ss.202016.0117

DO - 10.5705/ss.202016.0117

M3 - Article

AN - SCOPUS:85054981583

VL - 29

SP - 97

EP - 111

JO - Statistica Sinica

JF - Statistica Sinica

SN - 1017-0405

IS - 1

ER -