A systematic methodology for OS benchmark characterization

Shuo Hung Chen, Hsiao Mei Lin, Kuo Yi Chen, Yuan Hao Chang, Pen Chung Yew, Chien Chung Ho

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Using benchmarks to evaluate operating systems is a common and important approach. However, determining which benchmarks to use for such evaluation requires very careful consideration. It has been found that a seemingly naive change of system configuration or input set could lead to drastic change of benchmark characteristics, and could also lead to misleading or incorrect results. Some OS benchmark suites may also include too many benchmark programs with very similar characteristics that could give biased results against, or in favor of, certain kernel behavior. Hence, we need to determine the characteristics of benchmark programs in order to come up with an appropriate benchmark suite for such evaluation, and to interpret the measured results more precisely and correctly. Although there have been many tools developed to help on profiling an OS and to characterize its run-time behavior, the collected data by those tools are often very large and complex. It is extremely time consuming, labor intensive, and error prone to analyze the large volume of measured results, and to determine the characteristics of a suite of benchmark programs. In this work, we propose to use machine-learning techniques to help on analyzing and characterizing OS benchmark programs based on the traced OS kernel events. In this work, a systematic methodology is proposed to automatically characterize benchmarks. We found that the characterized OS behavior could help developers to choose appropriate applications to benchmark operating systems.

Original languageEnglish (US)
Title of host publicationProceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013
Pages404-409
Number of pages6
DOIs
StatePublished - Dec 1 2013
Event2013 Research in Adaptive and Convergent Systems, RACS 2013 - Montreal, QC, Canada
Duration: Oct 1 2013Oct 4 2013

Publication series

NameProceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013

Other

Other2013 Research in Adaptive and Convergent Systems, RACS 2013
CountryCanada
CityMontreal, QC
Period10/1/1310/4/13

Keywords

  • benchmark
  • multi-core
  • operating system
  • scalability

Fingerprint Dive into the research topics of 'A systematic methodology for OS benchmark characterization'. Together they form a unique fingerprint.

Cite this