Practical aspects of building a constrained random test framework for safety-critical embedded systems

In the safety-critical embedded system industry, one of the key challenges is to demonstrate the robustness and depend- Ability of the product prior to market release, which is typically done using various verification and validation (V&V) strategies. Directed verification testing is a common strategy that performs black-box testing at the system level; how- ever, it only samples a small set of specific system behaviors and requires heavily manual effort. In this paper, we de- scribe our experience and lessons learned of applying the concept of constrained random testing on safety-critical em- bedded systems as a complimentary testing methodology. Constrained random testing enables us to cover many more system behaviors through random input variations, random fault injections, and automatic output comparisons. Addi- Tionally, it can reduce manual effort and increase confidence on the dependability of both firmware and hardware.