Programs, tests, and oracles: The foundations of testing revisited

Research output: Chapter in Book/Report/Conference proceedingConference contribution

72 Scopus citations


In previous decades, researchers have explored the formal foundations of program testing. By exploring the foundations of testing largely separate from any specific method of testing, these researchers provided a general discussion of the testing process, including the goals, the underlying problems, and the limitations of testing. Unfortunately, a common, rigorous foundation has not been widely adopted in empirical software testing research, making it difficult to generalize and compare empirical research. We continue this foundational work, providing a framework intended to serve as a guide for future discussions and empirical studies concerning software testing. Specifically, we extend Gourlay's functional description of testing with the notion of a test oracle, an aspect of testing largely overlooked in previous foundational work and only lightly explored in general. We argue additional work exploring the interrelationship between programs, tests, and oracles should be performed, and use our extension to clarify concepts presented in previous work, present new concepts related to test oracles, and demonstrate that oracle selection must be considered when discussing the efficacy of a testing process.

Original languageEnglish (US)
Title of host publicationICSE 2011 - 33rd International Conference on Software Engineering, Proceedings of the Conference
Number of pages10
StatePublished - 2011
Event33rd International Conference on Software Engineering, ICSE 2011 - Waikiki, Honolulu, HI, United States
Duration: May 21 2011May 28 2011

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257


Other33rd International Conference on Software Engineering, ICSE 2011
Country/TerritoryUnited States
CityWaikiki, Honolulu, HI


  • testing formalism
  • theory of testing


Dive into the research topics of 'Programs, tests, and oracles: The foundations of testing revisited'. Together they form a unique fingerprint.

Cite this