A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification

Boxiang Wang, Hui Zou

Research output: Contribution to journalArticle

Abstract

Distance weighted discrimination (DWD) is an interesting large margin classifier that has been shown to enjoy nice properties and empirical successes. The original DWD only handles binary classification with a linear classification boundary. Multiclass classification problems naturally appear in various fields, such as speech recognition, satellite imagery classification, and self-driving vehicles, to name a few. For such complex classification problems, it is desirable to have a flexible multicategory kernel extension of the binary DWD when the optimal decision boundary is highly nonlinear. To this end, we propose a new multicategory kernel DWD, that is, defined as a margin-vector optimization problem in a reproducing kernel Hilbert space. This formulation is shown to enjoy Fisher consistency. We develop an accelerated projected gradient descent algorithm to fit the multicategory kernel DWD. Simulations and benchmark data applications are used to demonstrate the highly competitive performance of our method, as compared with some popular state-of-the-art multiclass classifiers.

Original languageEnglish (US)
Pages (from-to)396-408
Number of pages13
JournalTechnometrics
Volume61
Issue number3
DOIs
StatePublished - Jul 3 2019

Fingerprint

Multi-class Classification
Discrimination
kernel
Classification Problems
Margin
Classifiers
Classifier
Projected Gradient
Satellite Imagery
Vector Optimization Problem
Binary Classification
Satellite imagery
Reproducing Kernel Hilbert Space
Descent Algorithm
Gradient Algorithm
Gradient Descent
Hilbert spaces
Multi-class
Speech Recognition
Speech recognition

Keywords

  • Distance weighted discrimination
  • Fisher consistency
  • Multicategory classification
  • Nesterov’s acceleration
  • Projected gradient descent
  • Reproducing kernel Hilbert space

Cite this

A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification. / Wang, Boxiang; Zou, Hui.

In: Technometrics, Vol. 61, No. 3, 03.07.2019, p. 396-408.

Research output: Contribution to journalArticle

@article{ec9ee2b34391417d9c5c2cb75c802a7f,
title = "A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification",
abstract = "Distance weighted discrimination (DWD) is an interesting large margin classifier that has been shown to enjoy nice properties and empirical successes. The original DWD only handles binary classification with a linear classification boundary. Multiclass classification problems naturally appear in various fields, such as speech recognition, satellite imagery classification, and self-driving vehicles, to name a few. For such complex classification problems, it is desirable to have a flexible multicategory kernel extension of the binary DWD when the optimal decision boundary is highly nonlinear. To this end, we propose a new multicategory kernel DWD, that is, defined as a margin-vector optimization problem in a reproducing kernel Hilbert space. This formulation is shown to enjoy Fisher consistency. We develop an accelerated projected gradient descent algorithm to fit the multicategory kernel DWD. Simulations and benchmark data applications are used to demonstrate the highly competitive performance of our method, as compared with some popular state-of-the-art multiclass classifiers.",
keywords = "Distance weighted discrimination, Fisher consistency, Multicategory classification, Nesterov’s acceleration, Projected gradient descent, Reproducing kernel Hilbert space",
author = "Boxiang Wang and Hui Zou",
year = "2019",
month = "7",
day = "3",
doi = "10.1080/00401706.2018.1529629",
language = "English (US)",
volume = "61",
pages = "396--408",
journal = "Technometrics",
issn = "0040-1706",
publisher = "American Statistical Association",
number = "3",

}

TY - JOUR

T1 - A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification

AU - Wang, Boxiang

AU - Zou, Hui

PY - 2019/7/3

Y1 - 2019/7/3

N2 - Distance weighted discrimination (DWD) is an interesting large margin classifier that has been shown to enjoy nice properties and empirical successes. The original DWD only handles binary classification with a linear classification boundary. Multiclass classification problems naturally appear in various fields, such as speech recognition, satellite imagery classification, and self-driving vehicles, to name a few. For such complex classification problems, it is desirable to have a flexible multicategory kernel extension of the binary DWD when the optimal decision boundary is highly nonlinear. To this end, we propose a new multicategory kernel DWD, that is, defined as a margin-vector optimization problem in a reproducing kernel Hilbert space. This formulation is shown to enjoy Fisher consistency. We develop an accelerated projected gradient descent algorithm to fit the multicategory kernel DWD. Simulations and benchmark data applications are used to demonstrate the highly competitive performance of our method, as compared with some popular state-of-the-art multiclass classifiers.

AB - Distance weighted discrimination (DWD) is an interesting large margin classifier that has been shown to enjoy nice properties and empirical successes. The original DWD only handles binary classification with a linear classification boundary. Multiclass classification problems naturally appear in various fields, such as speech recognition, satellite imagery classification, and self-driving vehicles, to name a few. For such complex classification problems, it is desirable to have a flexible multicategory kernel extension of the binary DWD when the optimal decision boundary is highly nonlinear. To this end, we propose a new multicategory kernel DWD, that is, defined as a margin-vector optimization problem in a reproducing kernel Hilbert space. This formulation is shown to enjoy Fisher consistency. We develop an accelerated projected gradient descent algorithm to fit the multicategory kernel DWD. Simulations and benchmark data applications are used to demonstrate the highly competitive performance of our method, as compared with some popular state-of-the-art multiclass classifiers.

KW - Distance weighted discrimination

KW - Fisher consistency

KW - Multicategory classification

KW - Nesterov’s acceleration

KW - Projected gradient descent

KW - Reproducing kernel Hilbert space

UR - http://www.scopus.com/inward/record.url?scp=85063265265&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063265265&partnerID=8YFLogxK

U2 - 10.1080/00401706.2018.1529629

DO - 10.1080/00401706.2018.1529629

M3 - Article

AN - SCOPUS:85063265265

VL - 61

SP - 396

EP - 408

JO - Technometrics

JF - Technometrics

SN - 0040-1706

IS - 3

ER -