Android apps interact with their environment extensively, which can result in flaky, slow, or hard-to-debug tests. Developers often address these problems using test doubles - developer-defined objects that replace app or library classes during test execution. Although test doubles are widely used, there is limited understanding of how they are used in practice. To bridge this gap, we present an in-depth empirical study that aims to shed light on how developers create and use test doubles in Android apps. In our study, we first analyze a dataset of 1,006 apps with publicly available test suites to identify which frameworks and approaches developers most commonly use to create test doubles. We then investigate several research questions by studying how test doubles defined using these popular frameworks are created and used in the ten apps in the dataset that define the highest number of test doubles using these frameworks. Our results, based on the analysis of 2,365 test doubles that replace a total of 784 classes, provide insight into the types of test doubles used within Android apps and how they are utilized. Our results also show that test doubles used in Android apps and traditional Java test doubles differ in at least some respect. Finally, our results show that test doubles can introduce test smells and even mistakes in the test code. In the paper, we also discuss some implications of our findings that can help researchers and practitioners working in this area and guide future research.
|Original language||English (US)|
|Title of host publication||Proceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering, ICSE 2022|
|Publisher||IEEE Computer Society|
|Number of pages||13|
|State||Published - 2022|
|Event||44th ACM/IEEE International Conference on Software Engineering, ICSE 2022 - Pittsburgh, United States|
Duration: May 22 2022 → May 27 2022
|Name||Proceedings - International Conference on Software Engineering|
|Conference||44th ACM/IEEE International Conference on Software Engineering, ICSE 2022|
|Period||5/22/22 → 5/27/22|
Bibliographical noteFunding Information:
This work was partially supported by NSF, under grants CCF-1563991 and CCF-0725202, Spanish Government’s SCUM grant RTI2018-102043-B-I00, the Madrid Regional project BLOQUES, DARPA, under contract N66001-21-C-4024, ONR, under contract N00014-18-1-2662, DOE, under contract DE-FOA-0002460, and gifts from Facebook, Google, IBM Research, and Microsoft Research.
© 2022 ACM.
- Test mocking
- mobile apps
- software environment