Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds

Ujjal Kumar Mukherjee, Snigdhansu Chatterjee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In this paper we present a new algorithm based on a weighted projection quantiles for fast and frugal real time quantile estimation of large sized high dimensional data clouds. We present a projection quantile regression algorithm for high dimensional data. Second, we present a fast algorithm for computing the depth of a point or a new observation in relation to any high-dimensional data cloud, and propose a ranking system for multivariate data. Third, we briefly describe a real time rapid monitoring scheme similar to statistical process monitoring, for actionable analytics with big data. We believe these algorithms would be very useful for real time analysis of high dimensional 'big data' sets including streaming data sets. The proposed algorithms would be of immense use in several application areas such as real time financial market analysis, real time remote health monitoring of patients using body area networked devices and real time pricing and inventory decisions in retail and manufacturing sector.

Original languageEnglish (US)
Title of host publicationProceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014
EditorsWo Chang, Jun Huan, Nick Cercone, Saumyadipta Pyne, Vasant Honavar, Jimmy Lin, Xiaohua Tony Hu, Charu Aggarwal, Bamshad Mobasher, Jian Pei, Raghunath Nambiar
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages64-71
Number of pages8
ISBN (Electronic)9781479956654
DOIs
StatePublished - Jan 1 2014
Event2nd IEEE International Conference on Big Data, IEEE Big Data 2014 - Washington, United States
Duration: Oct 27 2014Oct 30 2014

Publication series

NameProceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

Other

Other2nd IEEE International Conference on Big Data, IEEE Big Data 2014
CountryUnited States
CityWashington
Period10/27/1410/30/14

Fingerprint

Monitoring
Process monitoring
Health
Costs
Big data
Financial markets

Keywords

  • 'Big Data'
  • body area network
  • data depth estimation
  • quantile regression
  • real time analysis
  • real time health monitoring
  • weighted projection quantiles

Cite this

Mukherjee, U. K., & Chatterjee, S. (2014). Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds. In W. Chang, J. Huan, N. Cercone, S. Pyne, V. Honavar, J. Lin, X. T. Hu, C. Aggarwal, B. Mobasher, J. Pei, ... R. Nambiar (Eds.), Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014 (pp. 64-71). [7004358] (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2014.7004358

Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds. / Mukherjee, Ujjal Kumar; Chatterjee, Snigdhansu.

Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. ed. / Wo Chang; Jun Huan; Nick Cercone; Saumyadipta Pyne; Vasant Honavar; Jimmy Lin; Xiaohua Tony Hu; Charu Aggarwal; Bamshad Mobasher; Jian Pei; Raghunath Nambiar. Institute of Electrical and Electronics Engineers Inc., 2014. p. 64-71 7004358 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mukherjee, UK & Chatterjee, S 2014, Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds. in W Chang, J Huan, N Cercone, S Pyne, V Honavar, J Lin, XT Hu, C Aggarwal, B Mobasher, J Pei & R Nambiar (eds), Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014., 7004358, Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, Institute of Electrical and Electronics Engineers Inc., pp. 64-71, 2nd IEEE International Conference on Big Data, IEEE Big Data 2014, Washington, United States, 10/27/14. https://doi.org/10.1109/BigData.2014.7004358
Mukherjee UK, Chatterjee S. Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds. In Chang W, Huan J, Cercone N, Pyne S, Honavar V, Lin J, Hu XT, Aggarwal C, Mobasher B, Pei J, Nambiar R, editors, Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 64-71. 7004358. (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). https://doi.org/10.1109/BigData.2014.7004358
Mukherjee, Ujjal Kumar ; Chatterjee, Snigdhansu. / Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds. Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. editor / Wo Chang ; Jun Huan ; Nick Cercone ; Saumyadipta Pyne ; Vasant Honavar ; Jimmy Lin ; Xiaohua Tony Hu ; Charu Aggarwal ; Bamshad Mobasher ; Jian Pei ; Raghunath Nambiar. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 64-71 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).
@inproceedings{861a2f50c27a4da587c55d68575c2dd0,
title = "Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds",
abstract = "In this paper we present a new algorithm based on a weighted projection quantiles for fast and frugal real time quantile estimation of large sized high dimensional data clouds. We present a projection quantile regression algorithm for high dimensional data. Second, we present a fast algorithm for computing the depth of a point or a new observation in relation to any high-dimensional data cloud, and propose a ranking system for multivariate data. Third, we briefly describe a real time rapid monitoring scheme similar to statistical process monitoring, for actionable analytics with big data. We believe these algorithms would be very useful for real time analysis of high dimensional 'big data' sets including streaming data sets. The proposed algorithms would be of immense use in several application areas such as real time financial market analysis, real time remote health monitoring of patients using body area networked devices and real time pricing and inventory decisions in retail and manufacturing sector.",
keywords = "'Big Data', body area network, data depth estimation, quantile regression, real time analysis, real time health monitoring, weighted projection quantiles",
author = "Mukherjee, {Ujjal Kumar} and Snigdhansu Chatterjee",
year = "2014",
month = "1",
day = "1",
doi = "10.1109/BigData.2014.7004358",
language = "English (US)",
series = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "64--71",
editor = "Wo Chang and Jun Huan and Nick Cercone and Saumyadipta Pyne and Vasant Honavar and Jimmy Lin and Hu, {Xiaohua Tony} and Charu Aggarwal and Bamshad Mobasher and Jian Pei and Raghunath Nambiar",
booktitle = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",

}

TY - GEN

T1 - Fast algorithm for computing weighted projection quantiles and data depth for high-dimensional large data clouds

AU - Mukherjee, Ujjal Kumar

AU - Chatterjee, Snigdhansu

PY - 2014/1/1

Y1 - 2014/1/1

N2 - In this paper we present a new algorithm based on a weighted projection quantiles for fast and frugal real time quantile estimation of large sized high dimensional data clouds. We present a projection quantile regression algorithm for high dimensional data. Second, we present a fast algorithm for computing the depth of a point or a new observation in relation to any high-dimensional data cloud, and propose a ranking system for multivariate data. Third, we briefly describe a real time rapid monitoring scheme similar to statistical process monitoring, for actionable analytics with big data. We believe these algorithms would be very useful for real time analysis of high dimensional 'big data' sets including streaming data sets. The proposed algorithms would be of immense use in several application areas such as real time financial market analysis, real time remote health monitoring of patients using body area networked devices and real time pricing and inventory decisions in retail and manufacturing sector.

AB - In this paper we present a new algorithm based on a weighted projection quantiles for fast and frugal real time quantile estimation of large sized high dimensional data clouds. We present a projection quantile regression algorithm for high dimensional data. Second, we present a fast algorithm for computing the depth of a point or a new observation in relation to any high-dimensional data cloud, and propose a ranking system for multivariate data. Third, we briefly describe a real time rapid monitoring scheme similar to statistical process monitoring, for actionable analytics with big data. We believe these algorithms would be very useful for real time analysis of high dimensional 'big data' sets including streaming data sets. The proposed algorithms would be of immense use in several application areas such as real time financial market analysis, real time remote health monitoring of patients using body area networked devices and real time pricing and inventory decisions in retail and manufacturing sector.

KW - 'Big Data'

KW - body area network

KW - data depth estimation

KW - quantile regression

KW - real time analysis

KW - real time health monitoring

KW - weighted projection quantiles

UR - http://www.scopus.com/inward/record.url?scp=84988273584&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988273584&partnerID=8YFLogxK

U2 - 10.1109/BigData.2014.7004358

DO - 10.1109/BigData.2014.7004358

M3 - Conference contribution

AN - SCOPUS:84988273584

T3 - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

SP - 64

EP - 71

BT - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

A2 - Chang, Wo

A2 - Huan, Jun

A2 - Cercone, Nick

A2 - Pyne, Saumyadipta

A2 - Honavar, Vasant

A2 - Lin, Jimmy

A2 - Hu, Xiaohua Tony

A2 - Aggarwal, Charu

A2 - Mobasher, Bamshad

A2 - Pei, Jian

A2 - Nambiar, Raghunath

PB - Institute of Electrical and Electronics Engineers Inc.

ER -