Efficient Transmission and Reconstruction of Dependent Data Streams via Edge Sampling

Joel Wolfrath, Abhishek Chandra

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Data stream processing is an increasingly important topic due to the prevalence of smart devices and the demand for real-time analytics. Geo-distributed streaming systems, where cloud-based queries utilize data streams from multiple distributed devices, face challenges since wide-area network (WAN) bandwidth is often scarce or expensive. Edge computing allows us to address these bandwidth costs by utilizing resources close to the devices, e.g. to perform sampling over the incoming data streams, which trades downstream query accuracy to reduce the overall transmission cost. In this paper, we leverage the fact that correlations between data streams may exist across devices located in the same geographical region. Using this insight, we develop a hybrid edge-cloud system which systematically trades off between sampling at the edge and estimation of missing values in the cloud to reduce traffic over the WAN. We present an optimization framework which computes sample sizes at the edge and systematically bounds the number of samples we can estimate in the cloud given the strength of the correlation between streams. Our evaluation with three real-world datasets shows that compared to existing sampling techniques, our system could provide comparable error rates over multiple aggregate queries while reducing WAN traffic by 27-42%.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE International Conference on Cloud Engineering, IC2E 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages47-57
Number of pages11
ISBN (Electronic)9781665491150
DOIs
StatePublished - 2022
Event10th IEEE International Conference on Cloud Engineering, IC2E 2022 - Pacific Grove, United States
Duration: Sep 26 2022Sep 30 2022

Publication series

NameProceedings - 2022 IEEE International Conference on Cloud Engineering, IC2E 2022

Conference

Conference10th IEEE International Conference on Cloud Engineering, IC2E 2022
Country/TerritoryUnited States
CityPacific Grove
Period9/26/229/30/22

Bibliographical note

Funding Information:
This research was supported in part by the NSF under grants CNS-1717834 and CNS-1908566.

Publisher Copyright:
© 2022 IEEE.

Keywords

  • approximate computing
  • big data
  • edge computing
  • Stream processing

Fingerprint

Dive into the research topics of 'Efficient Transmission and Reconstruction of Dependent Data Streams via Edge Sampling'. Together they form a unique fingerprint.

Cite this