Incremental evaluation of sliding-window queries over data streams

Thanaa M. Ghanem, Moustafa A. Hammad, Mohamed F. Mokbel, Walid G. Aref, Ahmed K. Elmagarmid

Research output: Contribution to journalArticlepeer-review

63 Scopus citations

Abstract

Two research efforts have been conducted to realize sliding-window queries in data stream management systems, namely, query reevaluation and incremental evaluation. In the query reevaluation method, two consecutive windows are processed independently of each other. On the other hand, in the incremental evaluation method, the query answer for a window is obtained incrementally from the answer of the preceding window. In this paper, we focus on the incremental evaluation method. Two approaches have been adopted for the incremental evaluation of sliding-window queries, namely, the input-triggered approach and the negative tuples approach. In the input-triggered approach, only the newly inserted tuples flow in the query pipeline and tuple expiration is based on the timestamps of the newly inserted tuples. On the other hand, in the negative tuples approach, tuple expiration is separated from tuple insertion where a tuple flows in the pipeline for every inserted or expired tuple. The negative tuples approach avoids the unpredictable output delays that result from the input-triggered approach. However, negative tuples double the number of tuples through the query pipeline, thus reducing the pipeline bandwidth. Based on a detailed study of the incremental evaluation pipeline, we classify the incremental query operators into two classes according to whether an operator can avoid the processing of negative tuples or not. Based on this classification, we present several optimization techniques over the negative tuples approach that aim to reduce the overhead of processing negative tuples while avoiding the output delay of the query answer. A detailed experimental study, based on a prototype system implementation, shows the performance gains over the input-triggered approach of the negative tuples approach when accompanied with the proposed optimizations.

Original languageEnglish (US)
Pages (from-to)57-71
Number of pages15
JournalIEEE Transactions on Knowledge and Data Engineering
Volume19
Issue number1
DOIs
StatePublished - Jan 2007

Bibliographical note

Funding Information:
Walid G. Aref’s research is supported by the US National Science Foundation under Grants IIS-0093116 and IIS-0209120.

Keywords

  • Data stream management systems
  • Negative tuples
  • Pipelined query execution

Fingerprint

Dive into the research topics of 'Incremental evaluation of sliding-window queries over data streams'. Together they form a unique fingerprint.

Cite this