A statistically rigorous approach for improving simulation methodology

J. J. Yi, David J Lilja, Douglas M Hawkins

Research output: Chapter in Book/Report/Conference proceedingConference contribution

100 Scopus citations

Abstract

Due to cost, time, and flexibility constraints, simulators are often used to explore the design space when developing new processor architectures, as well as when evaluating the performance of new processor enhancements. However, despite this dependence on simulators, statistically rigorous simulation methodologies are not typically used in computer architecture research. A formal methodology can provide a sound basis for drawing conclusions gathered from simulation results by adding statistical rigor, and consequently, can increase confidence in the simulation results. This paper demonstrates the application of a rigorous statistical technique to the setup and analysis phases of the simulation process. Specifically, we apply a Plackett and Burman design to: (1) identify key processor parameters; (2) classify benchmarks based on how they affect the processor; and (3) analyze the effect of processor performance enhancements. Our technique expands on previous work by applying a statistical method to improve the simulation methodology instead of applying a statistical model to estimate the performance of the processor.

Original languageEnglish (US)
Title of host publicationProceedings - 9th International Symposium on High-Performance Computer Architecture, HPCA 2003
PublisherIEEE Computer Society
Pages281-291
Number of pages11
ISBN (Electronic)0769518710
DOIs
StatePublished - Jan 1 2003
Event9th IEEE International Symposium on High-Performance Computer Architecture, HPCA 2003 - Anaheim, United States
Duration: Feb 8 2003Feb 12 2003

Publication series

NameProceedings - International Symposium on High-Performance Computer Architecture
Volume12
ISSN (Print)1530-0897

Other

Other9th IEEE International Symposium on High-Performance Computer Architecture, HPCA 2003
CountryUnited States
CityAnaheim
Period2/8/032/12/03

    Fingerprint

Keywords

  • Analytical models
  • Application software
  • Computational modeling
  • Computer architecture
  • Computer simulation
  • Costs
  • Performance analysis
  • Space exploration
  • Statistical analysis
  • Time factors

Cite this

Yi, J. J., Lilja, D. J., & Hawkins, D. M. (2003). A statistically rigorous approach for improving simulation methodology. In Proceedings - 9th International Symposium on High-Performance Computer Architecture, HPCA 2003 (pp. 281-291). [1183546] (Proceedings - International Symposium on High-Performance Computer Architecture; Vol. 12). IEEE Computer Society. https://doi.org/10.1109/HPCA.2003.1183546