Use of Test Doubles in Android Testing: An In-Depth Investigation

Mattia Fazzini, Chase Choi, Juan Manuel Copia, Gabriel Lee, Yoshiki Kakehi, Alessandra Gorla, Alessandro Orso

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Android apps interact with their environment extensively, which can result in flaky, slow, or hard-to-debug tests. Developers often address these problems using test doubles - developer-defined objects that replace app or library classes during test execution. Although test doubles are widely used, there is limited understanding of how they are used in practice. To bridge this gap, we present an in-depth empirical study that aims to shed light on how developers create and use test doubles in Android apps. In our study, we first analyze a dataset of 1,006 apps with publicly available test suites to identify which frameworks and approaches developers most commonly use to create test doubles. We then investigate several research questions by studying how test doubles defined using these popular frameworks are created and used in the ten apps in the dataset that define the highest number of test doubles using these frameworks. Our results, based on the analysis of 2,365 test doubles that replace a total of 784 classes, provide insight into the types of test doubles used within Android apps and how they are utilized. Our results also show that test doubles used in Android apps and traditional Java test doubles differ in at least some respect. Finally, our results show that test doubles can introduce test smells and even mistakes in the test code. In the paper, we also discuss some implications of our findings that can help researchers and practitioners working in this area and guide future research.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering, ICSE 2022
PublisherIEEE Computer Society
Pages2266-2278
Number of pages13
ISBN (Electronic)9781450392211
DOIs
StatePublished - 2022
Event44th ACM/IEEE International Conference on Software Engineering, ICSE 2022 - Pittsburgh, United States
Duration: May 22 2022May 27 2022

Publication series

NameProceedings - International Conference on Software Engineering
Volume2022-May
ISSN (Print)0270-5257

Conference

Conference44th ACM/IEEE International Conference on Software Engineering, ICSE 2022
Country/TerritoryUnited States
CityPittsburgh
Period5/22/225/27/22

Bibliographical note

Funding Information:
This work was partially supported by NSF, under grants CCF-1563991 and CCF-0725202, Spanish Government’s SCUM grant RTI2018-102043-B-I00, the Madrid Regional project BLOQUES, DARPA, under contract N66001-21-C-4024, ONR, under contract N00014-18-1-2662, DOE, under contract DE-FOA-0002460, and gifts from Facebook, Google, IBM Research, and Microsoft Research.

Publisher Copyright:
© 2022 ACM.

Keywords

  • Test mocking
  • mobile apps
  • software environment

Fingerprint

Dive into the research topics of 'Use of Test Doubles in Android Testing: An In-Depth Investigation'. Together they form a unique fingerprint.

Cite this