A randomized evaluation of on-site monitoring nested in a multinational randomized trial

Nicole Wyman Engen, Kathy Huppler Hullsiek, Waldo H. Belloso, Elizabeth Finley, Fleur Hudson, Eileen Denning, Catherine Carey, Mary Pearson, Jonathan Kagan

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. Methods: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. Results: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1–2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8–85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1–3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. Conclusion: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.

Original languageEnglish (US)
Pages (from-to)3-14
Number of pages12
JournalClinical Trials
Volume17
Issue number1
DOIs
StatePublished - Feb 1 2020

Fingerprint

Odds Ratio
Confidence Intervals
Fraud
Costs and Cost Analysis
Budgets
Random Allocation
Informed Consent
Hospitalization
Logistic Models

Keywords

  • On-site monitoring
  • central monitoring
  • cluster randomization
  • fixed effects
  • generalized estimating equation
  • hierarchical logistic regression
  • local monitoring
  • proc genmod

PubMed: MeSH publication types

  • Journal Article

Cite this

A randomized evaluation of on-site monitoring nested in a multinational randomized trial. / Wyman Engen, Nicole; Huppler Hullsiek, Kathy; Belloso, Waldo H.; Finley, Elizabeth; Hudson, Fleur; Denning, Eileen; Carey, Catherine; Pearson, Mary; Kagan, Jonathan.

In: Clinical Trials, Vol. 17, No. 1, 01.02.2020, p. 3-14.

Research output: Contribution to journalArticle

Wyman Engen, Nicole ; Huppler Hullsiek, Kathy ; Belloso, Waldo H. ; Finley, Elizabeth ; Hudson, Fleur ; Denning, Eileen ; Carey, Catherine ; Pearson, Mary ; Kagan, Jonathan. / A randomized evaluation of on-site monitoring nested in a multinational randomized trial. In: Clinical Trials. 2020 ; Vol. 17, No. 1. pp. 3-14.
@article{638ed1b77c274205822f50eb7751f97a,
title = "A randomized evaluation of on-site monitoring nested in a multinational randomized trial",
abstract = "Background: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. Methods: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95{\%} confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. Results: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4{\%}) in 56 of 99 sites (57{\%}) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8{\%}) in 34 of 97 sites (35{\%}) had a primary monitoring outcome (odds ratio = 1.7; 95{\%} confidence interval: 1.1–2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95{\%} confidence interval: 1.8–85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95{\%} confidence interval: 1.1–3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. Conclusion: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.",
keywords = "On-site monitoring, central monitoring, cluster randomization, fixed effects, generalized estimating equation, hierarchical logistic regression, local monitoring, proc genmod",
author = "{Wyman Engen}, Nicole and {Huppler Hullsiek}, Kathy and Belloso, {Waldo H.} and Elizabeth Finley and Fleur Hudson and Eileen Denning and Catherine Carey and Mary Pearson and Jonathan Kagan",
year = "2020",
month = "2",
day = "1",
doi = "10.1177/1740774519881616",
language = "English (US)",
volume = "17",
pages = "3--14",
journal = "Clinical Trials",
issn = "1740-7745",
publisher = "SAGE Publications Ltd",
number = "1",

}

TY - JOUR

T1 - A randomized evaluation of on-site monitoring nested in a multinational randomized trial

AU - Wyman Engen, Nicole

AU - Huppler Hullsiek, Kathy

AU - Belloso, Waldo H.

AU - Finley, Elizabeth

AU - Hudson, Fleur

AU - Denning, Eileen

AU - Carey, Catherine

AU - Pearson, Mary

AU - Kagan, Jonathan

PY - 2020/2/1

Y1 - 2020/2/1

N2 - Background: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. Methods: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. Results: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1–2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8–85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1–3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. Conclusion: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.

AB - Background: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. Methods: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. Results: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1–2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8–85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1–3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. Conclusion: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.

KW - On-site monitoring

KW - central monitoring

KW - cluster randomization

KW - fixed effects

KW - generalized estimating equation

KW - hierarchical logistic regression

KW - local monitoring

KW - proc genmod

UR - http://www.scopus.com/inward/record.url?scp=85074583600&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074583600&partnerID=8YFLogxK

U2 - 10.1177/1740774519881616

DO - 10.1177/1740774519881616

M3 - Article

C2 - 31647325

AN - SCOPUS:85074583600

VL - 17

SP - 3

EP - 14

JO - Clinical Trials

JF - Clinical Trials

SN - 1740-7745

IS - 1

ER -