SDN helps big data to optimize storage

Ali R. Butt, Ali Anwar, Yue Cheng

Research output: Chapter in Book/Report/Conference proceedingChapter


Distributed key-value stores have become the sine qua non for supporting today’s large-scale web services. The extreme latency and throughput requirements of modern web applications are driving the use of distributed in-memory object caches. Similarly, the use of persistent object stores has been growing rapidly as they combine key advantages such as HTTP-based RESTfulAPIs, high availability, elasticity with a pay-as-you-go pricing model that allows applications to scale as needed. Consequently, there is an urgent need for optimizing the emerging software defined cloud data centers to efficiently support such applications at scale. In this chapter, we discuss different techniques to optimize the Big Data processing and data management using key-value stores and software defined networks in virtualized cloud data centers. Specifically, we explore two key questions. (1) How do cloud services users, i.e., tenants, get the most bang-for-the-buck with a distributed in-memory key-value store deployment in a shared multitenant environment? (2) How do tenants enhance cloud object store’s capabilities through fine-grained resource management to effectively meet their SLAs while maximizing resource efficiency? Moreover, we also present the state of the art in this domain and provide a brief analysis of desirable features. We then demonstrate through experiments the impact of SDN-based Big Data storage management solution on improving performance and overall resource efficiency. Finally, we discuss open issues in SDN-based Big Data I/O stacks and future directions.

Original languageEnglish (US)
Title of host publicationBig Data and Software Defined Networks
PublisherInstitution of Engineering and Technology
Number of pages22
ISBN (Electronic)9781785613043
StatePublished - Jan 1 2018
Externally publishedYes

Bibliographical note

Publisher Copyright:
© The Institution of Engineering and Technology 2018.


  • Big data processing
  • Cache storage
  • Cloud computing
  • Cloud object store capabilities
  • Computer centres
  • Computer communications
  • Computer networks and techniques
  • Distributed in-memory key-value store deployment
  • Distributed in-memory object caches
  • Distributed key-value stores
  • Extreme latency
  • File organisation
  • Fine-grained resource management
  • HTTP-based RESTfulAPIs
  • Information networks
  • Internet software
  • Pay-as-you-go pricing model
  • Persistent object stores
  • Resource efficiency maximization
  • SDN-based big data I/O stacks
  • SDN-based big data storage management solution
  • Semiconductor storage
  • Shared multitenant environment
  • Slas
  • Software defined cloud data centers
  • Software defined networking
  • Storage management
  • Storage optimization
  • Throughput requirements
  • Virtualized cloud data centers
  • Web applications
  • Web services
  • Web services


Dive into the research topics of 'SDN helps big data to optimize storage'. Together they form a unique fingerprint.

Cite this