Key-value pairs allocation strategy for kinetic drives

Hebatalla Eldakiky, David H Du

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The explosion in the amount of data with the development of internet and cloud computing prompted much research to develop systems to store and process this data efficiently. Therefore, NoSQL Databases emerged as a solution to be considered due to their flexibility and high performance. key-value object storage, one of the NoSQL Databases categories, is widely used in many Big Data applications, where it treats the object as a single opaque collection identified by a key. This collection may have different fields for each record. Recently, Seagate developed a new drive called 'Kinetic drive'. The kinetic drive is an independent active disk accessible by an Ethernet which enables the applications to directly connect to the drive via IP address without the need to pass through multiple legacy layers using the storage servers. It can also carry out key-value pair operations on its own. So in large scale data management, a set of kinetic drives can be used to exploit parallelism in satisfying user requests and solve the bottleneck caused by queuing of requests in the storage server which manages multiple HDDs/SDDs. On the other hand, Kinetic drive has a limited bandwidth and size. Therefore, in this paper, we develop a key-value pairs allocation algorithm for kinetic drives which takes into consideration the data popularity, limited size and bandwidth of kinetic drive to avoid queuing on the level of the drive, We also carry out performance evaluation that shows the performance of our approach in allocating the data in a near-optimal number of drives and minimizes the drives' index table.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 4th International Conference on Big Data Computing Service and Applications, BigDataService 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages17-24
Number of pages8
ISBN (Electronic)9781538651193
DOIs
StatePublished - Jul 5 2018
Event4th IEEE International Conference on Big Data Computing Service and Applications, BigDataService 2018 - Bamberg, Germany
Duration: Mar 26 2018Mar 29 2018

Publication series

NameProceedings - IEEE 4th International Conference on Big Data Computing Service and Applications, BigDataService 2018

Other

Other4th IEEE International Conference on Big Data Computing Service and Applications, BigDataService 2018
CountryGermany
CityBamberg
Period3/26/183/29/18

Bibliographical note

Funding Information:
In the BigData world with its huge amount of data, there was indeed a need for tools to be able to store and process this huge amount of data efficiently. One of these tools is the key-value store. With the invention of the kinetic drive and being used as an independent key-value store, we can exploit parallelism to increase the performance of the whole storage system. In this paper, we proposed a data allocation approach to allocate key-value pairs on the kinetic drives taking into consideration the bandwidth issue of the kinetic drive to avoid queuing on the level of the drive. We also took the size of the index table into consideration to speed up the search time for the drive’s IP in the meta-data server. The performance evaluation shows that our proposed approach is near-optimal in terms of minimizing the number of assigned drives. Acknowledgments: This work was partially supported by the following NSF awards: 1305237, 1421913, 1439622 and 1525617.

Keywords

  • Data Allocation
  • Key Value Store
  • Kinetic Drives
  • Storage Management

Fingerprint Dive into the research topics of 'Key-value pairs allocation strategy for kinetic drives'. Together they form a unique fingerprint.

Cite this