Discovering instructions for robust binary-level coverage criteria

Vaibhav B Sharma, Tae Joon Byun, Stephen A McCamant, Sanjai Rayadurgam, Mats Heimdahl

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

Original languageEnglish (US)
Title of host publicationTECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017
EditorsTingting Yu, Darko Marinov
PublisherAssociation for Computing Machinery, Inc
Pages1-4
Number of pages4
ISBN (Electronic)9781450351126
DOIs
StatePublished - Jul 13 2017
Event1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017 - Santa Barbara, United States
Duration: Jul 13 2017 → …

Publication series

NameTECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

Other

Other1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017
CountryUnited States
CitySanta Barbara
Period7/13/17 → …

Fingerprint

coverage
instruction
ability

Keywords

  • Instruction classification
  • Object branch coverage

Cite this

Sharma, V. B., Byun, T. J., McCamant, S. A., Rayadurgam, S., & Heimdahl, M. (2017). Discovering instructions for robust binary-level coverage criteria. In T. Yu, & D. Marinov (Eds.), TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017 (pp. 1-4). (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3107091.3107092

Discovering instructions for robust binary-level coverage criteria. / Sharma, Vaibhav B; Byun, Tae Joon; McCamant, Stephen A; Rayadurgam, Sanjai; Heimdahl, Mats.

TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. ed. / Tingting Yu; Darko Marinov. Association for Computing Machinery, Inc, 2017. p. 1-4 (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sharma, VB, Byun, TJ, McCamant, SA, Rayadurgam, S & Heimdahl, M 2017, Discovering instructions for robust binary-level coverage criteria. in T Yu & D Marinov (eds), TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017, Association for Computing Machinery, Inc, pp. 1-4, 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, TECPS 2017, Santa Barbara, United States, 7/13/17. https://doi.org/10.1145/3107091.3107092
Sharma VB, Byun TJ, McCamant SA, Rayadurgam S, Heimdahl M. Discovering instructions for robust binary-level coverage criteria. In Yu T, Marinov D, editors, TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. Association for Computing Machinery, Inc. 2017. p. 1-4. (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017). https://doi.org/10.1145/3107091.3107092
Sharma, Vaibhav B ; Byun, Tae Joon ; McCamant, Stephen A ; Rayadurgam, Sanjai ; Heimdahl, Mats. / Discovering instructions for robust binary-level coverage criteria. TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017. editor / Tingting Yu ; Darko Marinov. Association for Computing Machinery, Inc, 2017. pp. 1-4 (TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017).
@inproceedings{8fa4a2ec641c46b69553a61bfd2cbfc0,
title = "Discovering instructions for robust binary-level coverage criteria",
abstract = "Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.",
keywords = "Instruction classification, Object branch coverage",
author = "Sharma, {Vaibhav B} and Byun, {Tae Joon} and McCamant, {Stephen A} and Sanjai Rayadurgam and Mats Heimdahl",
year = "2017",
month = "7",
day = "13",
doi = "10.1145/3107091.3107092",
language = "English (US)",
series = "TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017",
publisher = "Association for Computing Machinery, Inc",
pages = "1--4",
editor = "Tingting Yu and Darko Marinov",
booktitle = "TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017",

}

TY - GEN

T1 - Discovering instructions for robust binary-level coverage criteria

AU - Sharma, Vaibhav B

AU - Byun, Tae Joon

AU - McCamant, Stephen A

AU - Rayadurgam, Sanjai

AU - Heimdahl, Mats

PY - 2017/7/13

Y1 - 2017/7/13

N2 - Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

AB - Object-Branch Coverage (OBC) is often used to measure effective-ness of test suites, when source code is unavailable. The traditional OBC definition can be made more resilient to variations in compilers and the structure of generated code by creating more robust definitions. However finding which instructions should be included in each new definition is laborious, error-prone, and architecture-dependent. We automate the discovery of instructions to be included for an improved OBC definition on the X86 and ARM architectures. We discover all possible valid instructions by symbolically executing instruction decoders for X86 and ARM instructions. For each discovered instruction, we translate it to Vine IR, and check if the Vine IR translation satisfies the OBC definition. We verify the correctness of our tool by comparing its output with the X86 and ARM architecture manuals. Our automated instruction clas-sification facilitates development of more robust OBC definitions with better bug-finding ability and lesser sensitivity to compiler variations.

KW - Instruction classification

KW - Object branch coverage

UR - http://www.scopus.com/inward/record.url?scp=85027677773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027677773&partnerID=8YFLogxK

U2 - 10.1145/3107091.3107092

DO - 10.1145/3107091.3107092

M3 - Conference contribution

AN - SCOPUS:85027677773

T3 - TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

SP - 1

EP - 4

BT - TECPS 2017 - Proceedings of the 1st ACM SIGSOFT International Workshop on Testing Embedded and Cyber-Physical Systems, co-located with ISSTA 2017

A2 - Yu, Tingting

A2 - Marinov, Darko

PB - Association for Computing Machinery, Inc

ER -