Abstract
Data stream processing is an increasingly important topic due to the prevalence of smart devices and the demand for real-time analytics. Geo-distributed streaming systems, where cloud-based queries utilize data streams from multiple distributed devices, face challenges since wide-area network (WAN) bandwidth is often scarce or expensive. Edge computing allows us to address these bandwidth costs by utilizing resources close to the devices, e.g. to perform sampling over the incoming data streams, which trades downstream query accuracy to reduce the overall transmission cost. In this paper, we leverage the fact that correlations between data streams may exist across devices located in the same geographical region. Using this insight, we develop a hybrid edge-cloud system which systematically trades off between sampling at the edge and estimation of missing values in the cloud to reduce traffic over the WAN. We present an optimization framework which computes sample sizes at the edge and systematically bounds the number of samples we can estimate in the cloud given the strength of the correlation between streams. Our evaluation with three real-world datasets shows that compared to existing sampling techniques, our system could provide comparable error rates over multiple aggregate queries while reducing WAN traffic by 27-42%.
Original language | English (US) |
---|---|
Title of host publication | Proceedings - 2022 IEEE International Conference on Cloud Engineering, IC2E 2022 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 47-57 |
Number of pages | 11 |
ISBN (Electronic) | 9781665491150 |
DOIs | |
State | Published - 2022 |
Event | 10th IEEE International Conference on Cloud Engineering, IC2E 2022 - Pacific Grove, United States Duration: Sep 26 2022 → Sep 30 2022 |
Publication series
Name | Proceedings - 2022 IEEE International Conference on Cloud Engineering, IC2E 2022 |
---|
Conference
Conference | 10th IEEE International Conference on Cloud Engineering, IC2E 2022 |
---|---|
Country/Territory | United States |
City | Pacific Grove |
Period | 9/26/22 → 9/30/22 |
Bibliographical note
Funding Information:This research was supported in part by the NSF under grants CNS-1717834 and CNS-1908566.
Publisher Copyright:
© 2022 IEEE.
Keywords
- approximate computing
- big data
- edge computing
- Stream processing