Data allocation of large-scale key-value store system using kinetic drives

Nowadays, NoSQL databases become more and more prevalent. As an important category of NoSQL databases, key-value store (KVS) is widely used in many Big Data applications by providing flexible and efficient performance. In a KVS, a key-value pair can be easily accessed by giving a key. Recently, a new Ethernet accessed disk drive called 'Kinetic Drive' was developed by Seagate. This drive can be directly accessed by applications via IP addresses without going through a dedicated storage server. Given the keys from a user, Kinetic Drive can perform key-value pair operations by itself. It can reduce the management complexity, especially in large-scale deployment. Given a set of Kinetic Drives in a data center, it is important to manage the key-value pairs and store them in an organized way. In this paper, we investigate how to allocate data on a large-scale key-value store system using Kinetic Drives. We show the tradeoff among various design factors. We investigate key indexing schemes and allocate data on drives accordingly. We propose efficient approaches to migrate data among drives. We carry out performance evaluation that shows our approaches can handle various key distributions well.