Decaying telco big data with data postdiction

Constantinos Costa, Andreas Charalampous, Andreas Konstantinidis, Demetrios Zeinalipour-Yazti, Mohamed F. Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages106-115
Number of pages10
ISBN (Electronic)9781538641330
DOIs
StatePublished - Jul 13 2018
Event19th IEEE International Conference on Mobile Data Management, MDM 2018 - Aalborg, Denmark
Duration: Jun 26 2018Jun 28 2018

Publication series

NameProceedings - IEEE International Conference on Mobile Data Management
Volume2018-June
ISSN (Print)1551-6245

Other

Other19th IEEE International Conference on Mobile Data Management, MDM 2018
CountryDenmark
CityAalborg
Period6/26/186/28/18

Keywords

  • big data
  • data decaying
  • data reduction
  • machine learning
  • spatio-temporal analytics
  • telco

Fingerprint Dive into the research topics of 'Decaying telco big data with data postdiction'. Together they form a unique fingerprint.

Cite this