TY - GEN
T1 - A statistically rigorous approach for improving simulation methodology
AU - Yi, J. J.
AU - Lilja, David J
AU - Hawkins, Douglas M
PY - 2003/1/1
Y1 - 2003/1/1
N2 - Due to cost, time, and flexibility constraints, simulators are often used to explore the design space when developing new processor architectures, as well as when evaluating the performance of new processor enhancements. However, despite this dependence on simulators, statistically rigorous simulation methodologies are not typically used in computer architecture research. A formal methodology can provide a sound basis for drawing conclusions gathered from simulation results by adding statistical rigor, and consequently, can increase confidence in the simulation results. This paper demonstrates the application of a rigorous statistical technique to the setup and analysis phases of the simulation process. Specifically, we apply a Plackett and Burman design to: (1) identify key processor parameters; (2) classify benchmarks based on how they affect the processor; and (3) analyze the effect of processor performance enhancements. Our technique expands on previous work by applying a statistical method to improve the simulation methodology instead of applying a statistical model to estimate the performance of the processor.
AB - Due to cost, time, and flexibility constraints, simulators are often used to explore the design space when developing new processor architectures, as well as when evaluating the performance of new processor enhancements. However, despite this dependence on simulators, statistically rigorous simulation methodologies are not typically used in computer architecture research. A formal methodology can provide a sound basis for drawing conclusions gathered from simulation results by adding statistical rigor, and consequently, can increase confidence in the simulation results. This paper demonstrates the application of a rigorous statistical technique to the setup and analysis phases of the simulation process. Specifically, we apply a Plackett and Burman design to: (1) identify key processor parameters; (2) classify benchmarks based on how they affect the processor; and (3) analyze the effect of processor performance enhancements. Our technique expands on previous work by applying a statistical method to improve the simulation methodology instead of applying a statistical model to estimate the performance of the processor.
KW - Analytical models
KW - Application software
KW - Computational modeling
KW - Computer architecture
KW - Computer simulation
KW - Costs
KW - Performance analysis
KW - Space exploration
KW - Statistical analysis
KW - Time factors
UR - http://www.scopus.com/inward/record.url?scp=33749057743&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=33749057743&partnerID=8YFLogxK
U2 - 10.1109/HPCA.2003.1183546
DO - 10.1109/HPCA.2003.1183546
M3 - Conference contribution
AN - SCOPUS:33749057743
T3 - Proceedings - International Symposium on High-Performance Computer Architecture
SP - 281
EP - 291
BT - Proceedings - 9th International Symposium on High-Performance Computer Architecture, HPCA 2003
PB - IEEE Computer Society
T2 - 9th IEEE International Symposium on High-Performance Computer Architecture, HPCA 2003
Y2 - 8 February 2003 through 12 February 2003
ER -