In the realm of smart cities, telecommunication companies (telcos) are expected to play a protagonistic role as these can capture a variety of natural phenomena on an ongoing basis, e.g., traffic in a city, mobility patterns for emergency response or city planning. The key challenges for telcos in this era is to ingest in the most compact manner huge amounts of network logs, perform big data exploration and analytics on the generated data within a tolerable elapsed time. This paper introduces SPATE, an innovative telco big data exploration framework whose objectives are two-fold: (i) minimizing the storage space needed to incrementally retain data over time; and (ii) minimizing the response time for spatiotemporal data exploration queries over recent data. The storage layer of our framework uses lossless data compression to ingest recent streams of telco big data in the most compact manner retaining full resolution for data exploration tasks. The indexing layer of our system then takes care of the progressive loss of detail in information, coined decaying, as data ages with time. The exploration layer provides visual means to explore the generated spatio-Temporal information space. We measure the efficiency of the proposed framework using a 5GB anonymized real telco network trace and a variety of telco-specific tasks, such as OLAP and OLTP querying, privacy-Aware data sharing, multivariate statistics, clustering and regression. We show that out framework can achieve comparable response times to the state-of-The-Art using an order of magnitude less storage space.
|Original language||English (US)|
|Title of host publication||Proceedings - 2017 IEEE 33rd International Conference on Data Engineering, ICDE 2017|
|Publisher||IEEE Computer Society|
|Number of pages||12|
|State||Published - May 16 2017|
|Event||33rd IEEE International Conference on Data Engineering, ICDE 2017 - San Diego, United States|
Duration: Apr 19 2017 → Apr 22 2017
|Name||Proceedings - International Conference on Data Engineering|
|Other||33rd IEEE International Conference on Data Engineering, ICDE 2017|
|Period||4/19/17 → 4/22/17|
Bibliographical noteFunding Information:
This work was supported in part by the University of Cyprus, an industrial sponsorship by MTN Cyprus and EU COST Action IC1304. The third author's research is supported by the Alexander von Humboldt-Foundation, Germany. The last author's research is supported by NSF grants IIS-0952977, IIS-1218168, IIS-1525953, CNS-1512877.
© 2017 IEEE.