Abstract
The growing variety of data storage and retrieval needs is driving the design and development of an increasing number of distributed storage applications such as key-value stores, distributed file systems, object stores, and databases. We observe that, to a large extent, such applications would implement their own way of handling features of data replication, failover, consistency, cluster topology, leadership election, etc. We found that 45-82% of the code in six popular distributed storage applications can be classified as implementations of such common features. While such implementations allow for deeper optimizations tailored for a specific application, writing new applications to satisfy the ever-changing requirements of new types of data or I/O patterns is challenging, as it is notoriously hard to get all the features right in a distributed setting. In this paper, we argue that for most modern storage applications, the common feature implementation (i.e., the distributed part) can be automated and offloaded, so developers can focus on the core application functions. We are designing a framework, ClusterOn, which aims to take care of the messy plumbing of distributed storage applications. The envisioned goal is that a developer simply “drops” a non-distributed application into ClusterOn, which will convert it into a scalable and highly configurable distributed application.
Original language | English (US) |
---|---|
State | Published - 2016 |
Externally published | Yes |
Event | 8th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016 - Denver, United States Duration: Jun 20 2016 → Jun 21 2016 |
Conference
Conference | 8th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016 |
---|---|
Country/Territory | United States |
City | Denver |
Period | 6/20/16 → 6/21/16 |
Bibliographical note
Publisher Copyright:© USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, co-located with USENIX ATC 2016.All right reserved.