Another look at distance-weighted discrimination

Boxiang Wang, Hui Zou

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.

Original languageEnglish (US)
Pages (from-to)177-198
Number of pages22
JournalJournal of the Royal Statistical Society. Series B: Statistical Methodology
Volume80
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

Discrimination
Support Vector Machine
kernel
Second-order Cone Programming
Bayes Risk
Gaussian Kernel
Learning Theory
Reproducing Kernel Hilbert Space
Data-driven
Margin
Classifier
Kernel
Benchmark
Methodology

Keywords

  • Bayes risk consistency
  • Classification
  • Distance-weighted discrimination
  • Kernel learning
  • Majorization–minimization principle
  • Second-order cone programming

Cite this

Another look at distance-weighted discrimination. / Wang, Boxiang; Zou, Hui.

In: Journal of the Royal Statistical Society. Series B: Statistical Methodology, Vol. 80, No. 1, 01.01.2018, p. 177-198.

Research output: Contribution to journalArticle

@article{42431c1c50444e7098c14d11bd99f341,
title = "Another look at distance-weighted discrimination",
abstract = "Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.",
keywords = "Bayes risk consistency, Classification, Distance-weighted discrimination, Kernel learning, Majorization–minimization principle, Second-order cone programming",
author = "Boxiang Wang and Hui Zou",
year = "2018",
month = "1",
day = "1",
doi = "10.1111/rssb.12244",
language = "English (US)",
volume = "80",
pages = "177--198",
journal = "Journal of the Royal Statistical Society. Series B: Statistical Methodology",
issn = "1369-7412",
publisher = "Wiley-Blackwell",
number = "1",

}

TY - JOUR

T1 - Another look at distance-weighted discrimination

AU - Wang, Boxiang

AU - Zou, Hui

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.

AB - Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.

KW - Bayes risk consistency

KW - Classification

KW - Distance-weighted discrimination

KW - Kernel learning

KW - Majorization–minimization principle

KW - Second-order cone programming

UR - http://www.scopus.com/inward/record.url?scp=85038358161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038358161&partnerID=8YFLogxK

U2 - 10.1111/rssb.12244

DO - 10.1111/rssb.12244

M3 - Article

AN - SCOPUS:85038358161

VL - 80

SP - 177

EP - 198

JO - Journal of the Royal Statistical Society. Series B: Statistical Methodology

JF - Journal of the Royal Statistical Society. Series B: Statistical Methodology

SN - 1369-7412

IS - 1

ER -