Samples in Applied Psychology: Over a Decade of Research in Review

Winny Shen, Thomas B. Kiger, Stacy E. Davies, Rena L. Rasch, Kara M. Simon, Deniz S. Ones

Research output: Contribution to journalArticle

73 Scopus citations

Abstract

This study examines sample characteristics of articles published in Journal of Applied Psychology (JAP) from 1995 to 2008. At the individual level, the overall median sample size over the period examined was approximately 173, which is generally adequate for detecting the average magnitude of effects of primary interest to researchers who publish in JAP. Samples using higher units of analyses (e.g., teams, departments/work units, and organizations) had lower median sample sizes (Mdn ≈ 65), yet were arguably robust given typical multilevel design choices of JAP authors despite the practical constraints of collecting data at higher units of analysis. A substantial proportion of studies used student samples (~40%); surprisingly, median sample sizes for student samples were smaller than working adult samples. Samples were more commonly occupationally homogeneous (~70%) than occupationally heterogeneous. U.S. and English-speaking participants made up the vast majority of samples, whereas Middle Eastern, African, and Latin American samples were largely unrepresented. On the basis of study results, recommendations are provided for authors, editors, and readers, which converge on 3 themes: (a) appropriateness and match between sample characteristics and research questions, (b) careful consideration of statistical power, and (c) the increased popularity of quantitative synthesis. Implications are discussed in terms of theory building, generalizability of research findings, and statistical power to detect effects.

Original languageEnglish (US)
Pages (from-to)1055-1064
Number of pages10
JournalJournal of Applied Psychology
Volume96
Issue number5
DOIs
StatePublished - Sep 1 2011

Keywords

  • Experimental design
  • Methodology
  • Sample size
  • Statistical power

Fingerprint Dive into the research topics of 'Samples in Applied Psychology: Over a Decade of Research in Review'. Together they form a unique fingerprint.

  • Cite this

    Shen, W., Kiger, T. B., Davies, S. E., Rasch, R. L., Simon, K. M., & Ones, D. S. (2011). Samples in Applied Psychology: Over a Decade of Research in Review. Journal of Applied Psychology, 96(5), 1055-1064. https://doi.org/10.1037/a0023322