Non-convex penalized multitask regression using data depth-based penalties

Subhabrata Majumdar, Snigdhansu B Chatterjee

Research output: Contribution to journalArticle

Abstract

We propose a new class of non-convex penalties based on data depth functions for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centred at the origin. We derive the theoretical properties of an approximate one-step sparse estimator of the coefficient matrix using local linear approximation of the penalty function and provide an algorithm for its computation. For the orthogonal design and independent responses, the resulting thresholding rule enjoys near-minimax optimal risk performance, similar to the adaptive lasso (Zou, H (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association, 101, 1418–1429). A simulation study and real data analysis demonstrate its effectiveness compared with some of the present methods that provide sparse solutions in multitask regression.

Original languageEnglish (US)
Article numbere174
JournalStat
Volume7
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

Regression Depth
Data Depth
Adaptive Lasso
Penalty
Penalized Regression
Oracle Property
Orthogonal Design
Local Approximation
Linear Approximation
Penalty Function
Coefficient
Thresholding
Minimax
Data analysis
Quantify
Regression
Simulation Study
Estimator
Demonstrate
Coefficients

Keywords

  • data depth
  • multitask regression
  • non-convex penalty
  • sparsity

Cite this

Non-convex penalized multitask regression using data depth-based penalties. / Majumdar, Subhabrata; Chatterjee, Snigdhansu B.

In: Stat, Vol. 7, No. 1, e174, 01.01.2018.

Research output: Contribution to journalArticle

@article{8876797e46e841ae815ec2b01f709242,
title = "Non-convex penalized multitask regression using data depth-based penalties",
abstract = "We propose a new class of non-convex penalties based on data depth functions for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centred at the origin. We derive the theoretical properties of an approximate one-step sparse estimator of the coefficient matrix using local linear approximation of the penalty function and provide an algorithm for its computation. For the orthogonal design and independent responses, the resulting thresholding rule enjoys near-minimax optimal risk performance, similar to the adaptive lasso (Zou, H (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association, 101, 1418–1429). A simulation study and real data analysis demonstrate its effectiveness compared with some of the present methods that provide sparse solutions in multitask regression.",
keywords = "data depth, multitask regression, non-convex penalty, sparsity",
author = "Subhabrata Majumdar and Chatterjee, {Snigdhansu B}",
year = "2018",
month = "1",
day = "1",
doi = "10.1002/sta4.174",
language = "English (US)",
volume = "7",
journal = "Stat",
issn = "2049-1573",
publisher = "Wiley-Blackwell Publishing Ltd",
number = "1",

}

TY - JOUR

T1 - Non-convex penalized multitask regression using data depth-based penalties

AU - Majumdar, Subhabrata

AU - Chatterjee, Snigdhansu B

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We propose a new class of non-convex penalties based on data depth functions for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centred at the origin. We derive the theoretical properties of an approximate one-step sparse estimator of the coefficient matrix using local linear approximation of the penalty function and provide an algorithm for its computation. For the orthogonal design and independent responses, the resulting thresholding rule enjoys near-minimax optimal risk performance, similar to the adaptive lasso (Zou, H (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association, 101, 1418–1429). A simulation study and real data analysis demonstrate its effectiveness compared with some of the present methods that provide sparse solutions in multitask regression.

AB - We propose a new class of non-convex penalties based on data depth functions for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centred at the origin. We derive the theoretical properties of an approximate one-step sparse estimator of the coefficient matrix using local linear approximation of the penalty function and provide an algorithm for its computation. For the orthogonal design and independent responses, the resulting thresholding rule enjoys near-minimax optimal risk performance, similar to the adaptive lasso (Zou, H (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association, 101, 1418–1429). A simulation study and real data analysis demonstrate its effectiveness compared with some of the present methods that provide sparse solutions in multitask regression.

KW - data depth

KW - multitask regression

KW - non-convex penalty

KW - sparsity

UR - http://www.scopus.com/inward/record.url?scp=85042183158&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042183158&partnerID=8YFLogxK

U2 - 10.1002/sta4.174

DO - 10.1002/sta4.174

M3 - Article

AN - SCOPUS:85042183158

VL - 7

JO - Stat

JF - Stat

SN - 2049-1573

IS - 1

M1 - e174

ER -