Assessing and Visualizing Simultaneous Simulation Error

Nathan Robertson, James M. Flegal, Dootika Vats, Galin L. Jones

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Monte Carlo experiments produce samples to estimate features such as means and quantiles of a given distribution. However, simultaneous estimation of means and quantiles has received little attention. In this setting, we establish a multivariate central limit theorem for any finite combination of sample means and quantiles under the assumption of a strongly mixing process, which includes the standard Monte Carlo and Markov chain Monte Carlo settings. We build on this to provide a fast algorithm for constructing hyperrectangular confidence regions having the desired simultaneous coverage probability and a convenient marginal interpretation. The methods are incorporated into standard ways of visualizing the results of Monte Carlo experiments enabling the practitioner to more easily assess the reliability of the results. We demonstrate the utility of this approach in various Monte Carlo settings including simulation studies based on independent and identically distributed samples and Bayesian analyses using Markov chain Monte Carlo sampling. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)324-334
Number of pages11
JournalJournal of Computational and Graphical Statistics
Issue number2
StatePublished - 2020

Bibliographical note

Publisher Copyright:
© 2020 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.


  • Markov chain Monte Carlo
  • Monte Carlo
  • Quantile limit theorems
  • Simulation studies
  • Strong mixing
  • Visualizations


Dive into the research topics of 'Assessing and Visualizing Simultaneous Simulation Error'. Together they form a unique fingerprint.

Cite this