Randomized load balancing with general service time distributions

Maury Bramson, Yi Lu, Balaji Prabhakar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

112 Scopus citations


Randomized load balancing greatly improves the sharing of resources in a number of applications while being simple to implement. One model that has been extensively used to study randomized load balancing schemes is the supermarket model. In this model, jobs arrive according to a rate-nλ Poisson process at a bank of n rate-1 exponential server queues. A notable result, due to Vvedenskaya et.al. (1996), showed that when each arriving job is assigned to the shortest of d ≥ 2 randomly chosen queues, the equilibrium queue sizes decay doubly exponentially in the limit as n → ∞. This is a substantial improvement over the case d = 1, where queue sizes decay exponentially. The method of analysis used in the above paper and in the subsequent literature applies to jobs with exponential service time distributions and does not easily generalize. It is desirable to study load balancing models with more general, especially heavy-tailed, service time distributions since such service times occur widely in practice. This paper describes a modularized program for treating randomized load balancing problems with general service time distributions and service disciplines. The program relies on an ansatz which asserts that any finite set of queues in a randomized load balancing scheme becomes independent as n → ∞. This allows one to derive queue size distributions and other performance measures of interest. We establish the ansatz when the service discipline is FIFO and the service time distribution has a decreasing hazard rate (this includes heavy-tailed service times). Assuming the ansatz, we also obtain the following results: (i) as n → ∞, the process of job arrivals at any fixed queue tends to a Poisson process whose rate depends on the size of the queue, (ii) when the service discipline at each server is processor sharing or LIFO with preemptive resume, the distribution of the number of jobs is insensitive to the service distribution, and (iii) the tail behavior of the queue-size distribution in terms of the service distribution for the FIFO service discipline.

Original languageEnglish (US)
Title of host publicationSIGMETRICS'10 - Proceedings of the 2010 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems
Number of pages12
Edition1 SPEC. ISSUE
StatePublished - 2010
Event2010 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS'10 - New York, NY, United States
Duration: Jun 14 2010Jun 18 2010

Publication series

NamePerformance Evaluation Review
ISSN (Print)0163-5999


Conference2010 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS'10
Country/TerritoryUnited States
CityNew York, NY


  • Asymptotic independence
  • Load balancing
  • Randomized algorithms


Dive into the research topics of 'Randomized load balancing with general service time distributions'. Together they form a unique fingerprint.

Cite this