Hidra: Statistical multi-dimensional resource discovery for large-scale systems

Michael Cardosa, Abhishek Chandra

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Resource discovery enables applications deployed in heterogeneous large-scale distributed systems to find resources that meet QoS requirements. In particular, most applications need resource requirements to be satisfied simultaneously for multiple resources (such as CPU, memory and network bandwidth). Due to dynamism in many large-scale systems, providing statistical guarantees on such requirements is important to avoid application failures and overheads. However, existing techniques either provide guarantees only for individual resources, or take a static or memoryless approach along multiple dimensions. We present HiDRA, a scalable resource discovery technique providing statistical guarantees for resource requirements spanning multiple dimensions simultaneously. Through trace analysis and a 307-node PlanetLab implementation, we show that HiDRA, while using over 1, 400 times less data, performs nearly as well as a fully-informed algorithm, showing better precision and having recall within 3%. We demonstrate that HiDRA is a feasible, low-overhead approach to statistical resource discovery in a distributed system.

Original languageEnglish (US)
Title of host publication2009 17th International Workshop on Quality of Service, IWQoS 2009
DOIs
StatePublished - 2009
Event2009 17th International Workshop on Quality of Service, IWQoS 2009 - Charleston, SC, United States
Duration: Jul 13 2009Jul 15 2009

Publication series

NameIEEE International Workshop on Quality of Service, IWQoS
ISSN (Print)1548-615X

Other

Other2009 17th International Workshop on Quality of Service, IWQoS 2009
CountryUnited States
CityCharleston, SC
Period7/13/097/15/09

Fingerprint Dive into the research topics of 'Hidra: Statistical multi-dimensional resource discovery for large-scale systems'. Together they form a unique fingerprint.

Cite this