Discovering flow anomalies: A SWEET approach

James M. Kang, Shashi Shekhar, Christine Wennen, Paige J Novak

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

Given a percentage-threshold and readings from a pair of consecutive upstream and downstream sensors, flow anomaly discovery identifies dominant time intervals where the fraction of time instants of significantly mis-matched sensor readings exceed the given percentage-threshold. Discovering flow anomalies (FA) is an important problem in environmental flow monitoring networks and early warning detection systems for water quality problems. However, mining FAs is computationally expensive because of the large (potentially infinite) number of time instants of measurement and potentially long delays due to stagnant (e.g. lakes) or slow moving (e.g. wetland) water bodies between consecutive sensors. Traditional outlier detection methods (e.g. t-test) are suited for detecting transient FAs (i.e., time instants of significant mis-matches across consecutive sensors) and cannot detect persistent FAs (i.e., long variable time-windows with a high fraction of time instant transient FAs) due to a lack of a pre-defined window size. In contrast, we propose a Smart Window Enumeration and Evaluation of persistence-Thresholds (SWEET) method to efficiently explore the search space of all possible window lengths. Computation overhead is brought down significantly by restricting the start and end points of a window to coincide with transient FAs, using a smart counter and efficient pruning techniques. Experimental evaluation using a real dataset shows our proposed approach outperforms Naïve alternatives.

Original languageEnglish (US)
Title of host publicationProceedings - 8th IEEE International Conference on Data Mining, ICDM 2008
Pages851-856
Number of pages6
DOIs
StatePublished - 2008
Event8th IEEE International Conference on Data Mining, ICDM 2008 - Pisa, Italy
Duration: Dec 15 2008Dec 19 2008

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
ISSN (Print)1550-4786

Other

Other8th IEEE International Conference on Data Mining, ICDM 2008
CountryItaly
CityPisa
Period12/15/0812/19/08

Fingerprint Dive into the research topics of 'Discovering flow anomalies: A SWEET approach'. Together they form a unique fingerprint.

  • Cite this

    Kang, J. M., Shekhar, S., Wennen, C., & Novak, P. J. (2008). Discovering flow anomalies: A SWEET approach. In Proceedings - 8th IEEE International Conference on Data Mining, ICDM 2008 (pp. 851-856). [4781190] (Proceedings - IEEE International Conference on Data Mining, ICDM). https://doi.org/10.1109/ICDM.2008.117