KBF: A Bloom Filter for key-value storage with an application on approximate state machines

Sisi Xiong, Yanjun Yao, Qing Cao, Tian He

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

Key-value (k-v) storage has been used as a crucial component for many network applications, such as social networks, online retailing, and cloud computing. Such storage usually provides support for operations on key-value pairs, and can be stored in memory to speed up responses to queries. So far, existing methods have been deterministic: they will faithfully return previously inserted key-value pairs. Providing such accuracy, however, comes at the cost of memory and CPU time. In contrast, in this paper, we present an approximate k-v storage that is more compact than existing methods. The tradeoff is that it may, theoretically, return a null value for a valid key with a low probability, or return a valid value for a key that was never inserted. Its design is based on the probabilistic data structure called the 'Bloom Filter', which was originally developed to test element membership in sets. In this paper, we extend the bloom filter concept to support key-value operations, and demonstrate that it still retains the compact nature of the original bloom filter. We call the resulting design as the kBF (key-value bloom filter), and systematically analyze its performance advantages and design tradeoffs. Finally, we apply the kBF to a practical problem of implementing a state machine in network intrusion detection to demonstrate how the kBF can be used as a building block for more complicated software infrastructures.

Original languageEnglish (US)
Title of host publicationIEEE INFOCOM 2014 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1150-1158
Number of pages9
ISBN (Print)9781479933600
DOIs
StatePublished - 2014
Event33rd IEEE Conference on Computer Communications, IEEE INFOCOM 2014 - Toronto, ON, Canada
Duration: Apr 27 2014May 2 2014

Publication series

NameProceedings - IEEE INFOCOM
ISSN (Print)0743-166X

Other

Other33rd IEEE Conference on Computer Communications, IEEE INFOCOM 2014
CountryCanada
CityToronto, ON
Period4/27/145/2/14

Fingerprint Dive into the research topics of 'KBF: A Bloom Filter for key-value storage with an application on approximate state machines'. Together they form a unique fingerprint.

Cite this