Improving the accuracy of oracle verdicts through automated model steering

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

The oraclea judge of the correctness of the system under test (SUT)is a major component of the testing process. Specifying test oracles is challenging for some domains, such as real-time embedded systems, where small changes in timing or sensory input may cause large behavioral differences. Models of such systems, often built for analysis and simulation, are appealing for reuse as oracles. These models, however, typically represent an idealized system, abstracting away certain issues such as non-deterministic timing behavior and sensor noise. Thus, even with the same inputs, the model's behavior may fail to match an acceptable behavior of the SUT, leading to many false positives reported by the oracle. We propose an automated steering framework that can adjust the behavior of the model to better match the behavior of the SUT to reduce the rate of false positives. This model steering is limited by a set of constraints (defining acceptable differences in behavior) and is based on a search process attempting to minimize a dissimilarity metric. This framework allows non-deterministic, but bounded, behavior differences, while preventing future mismatches, by guiding the oraclewithin limitsto match the execution of the SUT. Results show that steering significantly increases SUT-oracle conformance with minimal masking of real faults and, thus, has significant potential for reducing false positives and, consequently, development costs.

Original languageEnglish (US)
Title of host publicationASE 2014 - Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering
PublisherAssociation for Computing Machinery, Inc
Pages527-537
Number of pages11
ISBN (Electronic)9781450330138
DOIs
StatePublished - 2014
Event29th ACM/IEEE International Conference on Automated Software Engineering, ASE 2014 - Vasteras, Sweden
Duration: Sep 15 2014Sep 19 2014

Publication series

NameASE 2014 - Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

Other

Other29th ACM/IEEE International Conference on Automated Software Engineering, ASE 2014
CountrySweden
CityVasteras
Period9/15/149/19/14

Keywords

  • Model-based testing
  • Software testing
  • Test oracles

Fingerprint Dive into the research topics of 'Improving the accuracy of oracle verdicts through automated model steering'. Together they form a unique fingerprint.

Cite this