On the use of microservers in supporting hadoop applications

Ali Anwar, K. R. Krish, Ali R. Butt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations

Abstract

The use of economical, low-power microservers comprising of embedded CPUs is on the rise in supporting a myriad of applications. State of the art microservers can already match the performance of low-end traditional servers, and have been advocated as an energy-efficient alternative computing substrate for data centers as well. In this paper, we explore whether cluster comprising microservers can support the popular Hadoop platform. We conduct a quantitative study of six representative Hadoop applications on five hardware configurations. To compare the different clusters, we also define a comprehensive metric, PerfEC, which unifies the performance, energy consumption, and the acquisition and operating costs of the applications, and helps identify appropriate clusters for Hadoop applications. Experiments on our test clusters suggest that for applications such as TeraSort, RandomWriter and Grep microservers offer up to two orders of magnitude better efficiency in terms of PerfEC than traditional clusters. Similarly, a 3000-node cluster simulation driven by a real-world trace from Facebook shows that on average the studied microservers can match the performance of standard servers, while providing up to 31% energy savings at only 60% of the acquisition cost. We also compare PerfEC to the extant Total Cost of Ownership (TCO) metric, and find that our approach is better able to capture the trade-offs involved.

Original languageEnglish (US)
Title of host publication2014 IEEE International Conference on Cluster Computing, CLUSTER 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages66-74
Number of pages9
ISBN (Electronic)9781479955480
DOIs
StatePublished - Nov 26 2014
Externally publishedYes
Event16th IEEE International Conference on Cluster Computing, CLUSTER 2014 - Madrid, Spain
Duration: Sep 22 2014Sep 26 2014

Publication series

Name2014 IEEE International Conference on Cluster Computing, CLUSTER 2014

Conference

Conference16th IEEE International Conference on Cluster Computing, CLUSTER 2014
Country/TerritorySpain
CityMadrid
Period9/22/149/26/14

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

Fingerprint

Dive into the research topics of 'On the use of microservers in supporting hadoop applications'. Together they form a unique fingerprint.

Cite this