ClusterOn: Building highly configurable and reusable clustered data services using simple data nodes

Ali Anwar, Yue Cheng, Hai Huang, Ali R. Butt

Research output: Contribution to conferencePaperpeer-review

6 Scopus citations

Abstract

The growing variety of data storage and retrieval needs is driving the design and development of an increasing number of distributed storage applications such as key-value stores, distributed file systems, object stores, and databases. We observe that, to a large extent, such applications would implement their own way of handling features of data replication, failover, consistency, cluster topology, leadership election, etc. We found that 45-82% of the code in six popular distributed storage applications can be classified as implementations of such common features. While such implementations allow for deeper optimizations tailored for a specific application, writing new applications to satisfy the ever-changing requirements of new types of data or I/O patterns is challenging, as it is notoriously hard to get all the features right in a distributed setting. In this paper, we argue that for most modern storage applications, the common feature implementation (i.e., the distributed part) can be automated and offloaded, so developers can focus on the core application functions. We are designing a framework, ClusterOn, which aims to take care of the messy plumbing of distributed storage applications. The envisioned goal is that a developer simply “drops” a non-distributed application into ClusterOn, which will convert it into a scalable and highly configurable distributed application.

Original languageEnglish (US)
StatePublished - 2016
Externally publishedYes
Event8th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016 - Denver, United States
Duration: Jun 20 2016Jun 21 2016

Conference

Conference8th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016
Country/TerritoryUnited States
CityDenver
Period6/20/166/21/16

Bibliographical note

Publisher Copyright:
© USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016.All right reserved.

Fingerprint

Dive into the research topics of 'ClusterOn: Building highly configurable and reusable clustered data services using simple data nodes'. Together they form a unique fingerprint.

Cite this