Job scheduling in the presence of multiple resource requirements

William Leinberger, George Karypis, Vipin Kumar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

49 Scopus citations

Abstract

In past massively parallel processing systems, such as the Intel Paragon and the Thinking Machines CM-5, the scheduling problem consisted of allocating a single type of resource among the waiting jobs; the processing node. A job was allocated the minimum number of nodes required to meet its largest resource requirement (e.g. memory, CPUs, I/O channels, etc.). Recent systems, such as the SUN E10000 and SGI O2K, are made up of pools of independently allocatable hardware and software resources such as shared memory, large disk farms, distinct I/O channels, and software licenses. In order to make efficient use of all the available system resources, the scheduling algorithm must be able to maintain a job working set which fully utilizes all of the resources. Previous work in scheduling multiple resources focused on coordinating the allocation of CPUs and memory, using ad-hoc methods for generating good schedules. We provide new job selection heuristics based on resource balancing which support the construction of generalized K-resource scheduling algorithms. We show through simulation that performance gains of up to 50% in average response time are achievable over classical scheduling methods such as First-Come-First-Served with First-Fit backfill.

Original languageEnglish (US)
Title of host publicationACM/IEEE SC 1999 Conference, SC 1999
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages47
Number of pages1
ISBN (Electronic)1581130910, 9781581130911
DOIs
StatePublished - 1999
Event1999 ACM/IEEE Conference on Supercomputing, SC 1999 - Portland, United States
Duration: Nov 13 1999Nov 19 1999

Publication series

NameACM/IEEE SC 1999 Conference, SC 1999

Other

Other1999 ACM/IEEE Conference on Supercomputing, SC 1999
Country/TerritoryUnited States
CityPortland
Period11/13/9911/19/99

Bibliographical note

Funding Information:
∗This work was supported by NASA NCC2-5268 and by Army High Performance Computing Research Center cooperative agreement number DAAH04-95-2-0003/contract number DAAH04-95-C-0008. Access to computing facilities was provided by AHPCRC, Minnesota Supercomputer Institute.

Publisher Copyright:
© 1999 IEEE.

Keywords

  • High performance computing
  • Multiple resource constraints
  • Parallel job scheduling

Fingerprint

Dive into the research topics of 'Job scheduling in the presence of multiple resource requirements'. Together they form a unique fingerprint.

Cite this