Skip to main navigation Skip to search Skip to main content

Human-Agent versus Human Pull Requests: A Testing-Focused Characterization and Comparison

  • Roberto Milanese
  • , Francesco Salzano
  • , Angelica Spina
  • , Antonio Vitale
  • , Remo Pareschi
  • , Fausto Fasano
  • , Mattia Fazzini

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

AI-based coding agents are increasingly integrated into software development workflows, collaborating with developers to create pull requests (PRs). Despite their growing adoption, the role of human-agent collaboration in software testing remains poorly understood. This paper presents an empirical study of 6,582 human-agent PRs (HAPRs) and 3,122 human PRs (HPRs) from the AIDev dataset. We compare HAPRs and HPRs along three dimensions: (i) testing frequency and extent, (ii) types of testing-related changes (code-and-test co-evolution vs. test-focused), and (iii) testing quality, measured by test smells. Our findings reveal that, although the likelihood of including tests is comparable (42.9% for HAPRs vs. 40.0% for HPRs), HAPRs exhibit a larger extent of testing, nearly doubling the test-to-source line ratio found in HPRs. While test-focused task distributions are comparable, HAPRs are more likely to add new tests during co-evolution (OR = 1.79), whereas HPRs prioritize modifying existing tests. Finally, although some test smell categories differ statistically, negligible effect sizes suggest no meaningful differences in quality. These insights provide the first characterization of how human-agent collaboration shapes testing practices.

Original languageEnglish (US)
Title of host publicationProceedings - 2026 IEEE/ACM 23rd International Conference on Mining Software Repositories, MSR 2026
PublisherAssociation for Computing Machinery, Inc
Pages999-1003
Number of pages5
ISBN (Electronic)9798400724749
DOIs
StatePublished - Jul 31 2026
Event23rd International Conference on Mining Software Repositories, MSR 2026 - Rio de Janeiro, Brazil
Duration: Apr 13 2026 → Apr 14 2026

Publication series

NameProceedings - 2026 IEEE/ACM 23rd International Conference on Mining Software Repositories, MSR 2026

Conference

Conference23rd International Conference on Mining Software Repositories, MSR 2026
Country/TerritoryBrazil
CityRio de Janeiro
Period4/13/26 → 4/14/26

Bibliographical note

Publisher Copyright:
© 2026 Copyright held by the owner/author(s).

Keywords

  • Coding Agents
  • Empirical Study
  • Mining Software Repositories
  • Pull Requests
  • Software Testing
  • Test Smells

Fingerprint

Dive into the research topics of 'Human-Agent versus Human Pull Requests: A Testing-Focused Characterization and Comparison'. Together they form a unique fingerprint.

Cite this