Expert panel reviews of research centers: The site visit process

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Site visits are used extensively in a variety of settings within the evaluation community. They are especially common in making summative value decisions about the quality and worth of research programs/centers. However, there has been little empirical research and guidance about how to appropriately conduct evaluative site visits of research centers. We review the processes of two site visit examples using an expert panel review: (1) a process to evaluate four university research centers and (2) a process to review a federally sponsored research center. A set of 14 categories describing the expert panel review process was obtained through content analysis and participant observation. Most categories were addressed differently through the two processes highlighting the need for more research about the most effective processes to use within different contexts. Decisions about how to structure site visits appear to depend on the research context, practical considerations, the level at which the review is being conducted and the intended impact of the report. Future research pertaining to the selection of site visitors, the autonomy of the visitors in data collection and report writing, and the amount and type of information provided would be particularly valuable.

Original languageEnglish (US)
Pages (from-to)390-397
Number of pages8
JournalEvaluation and Program Planning
Volume35
Issue number3
DOIs
StatePublished - Aug 1 2012

Fingerprint

expert
Research
autonomy
research program
Empirical Research
university research
participant observation
research centre
Research center
empirical research
content analysis
Observation
evaluation
community
decision
Values
need
analysis

Keywords

  • Expert panel review/evaluation
  • Research center
  • Site visit

Cite this

Expert panel reviews of research centers : The site visit process. / Lawrenz, Frances P; Thao, Mao; Johnson, Kelli.

In: Evaluation and Program Planning, Vol. 35, No. 3, 01.08.2012, p. 390-397.

Research output: Contribution to journalArticle

@article{31cb4aaae34a40c287773d64ce5e5b2d,
title = "Expert panel reviews of research centers: The site visit process",
abstract = "Site visits are used extensively in a variety of settings within the evaluation community. They are especially common in making summative value decisions about the quality and worth of research programs/centers. However, there has been little empirical research and guidance about how to appropriately conduct evaluative site visits of research centers. We review the processes of two site visit examples using an expert panel review: (1) a process to evaluate four university research centers and (2) a process to review a federally sponsored research center. A set of 14 categories describing the expert panel review process was obtained through content analysis and participant observation. Most categories were addressed differently through the two processes highlighting the need for more research about the most effective processes to use within different contexts. Decisions about how to structure site visits appear to depend on the research context, practical considerations, the level at which the review is being conducted and the intended impact of the report. Future research pertaining to the selection of site visitors, the autonomy of the visitors in data collection and report writing, and the amount and type of information provided would be particularly valuable.",
keywords = "Expert panel review/evaluation, Research center, Site visit",
author = "Lawrenz, {Frances P} and Mao Thao and Kelli Johnson",
year = "2012",
month = "8",
day = "1",
doi = "10.1016/j.evalprogplan.2012.01.003",
language = "English (US)",
volume = "35",
pages = "390--397",
journal = "Evaluation and Program Planning",
issn = "0149-7189",
publisher = "Elsevier Limited",
number = "3",

}

TY - JOUR

T1 - Expert panel reviews of research centers

T2 - The site visit process

AU - Lawrenz, Frances P

AU - Thao, Mao

AU - Johnson, Kelli

PY - 2012/8/1

Y1 - 2012/8/1

N2 - Site visits are used extensively in a variety of settings within the evaluation community. They are especially common in making summative value decisions about the quality and worth of research programs/centers. However, there has been little empirical research and guidance about how to appropriately conduct evaluative site visits of research centers. We review the processes of two site visit examples using an expert panel review: (1) a process to evaluate four university research centers and (2) a process to review a federally sponsored research center. A set of 14 categories describing the expert panel review process was obtained through content analysis and participant observation. Most categories were addressed differently through the two processes highlighting the need for more research about the most effective processes to use within different contexts. Decisions about how to structure site visits appear to depend on the research context, practical considerations, the level at which the review is being conducted and the intended impact of the report. Future research pertaining to the selection of site visitors, the autonomy of the visitors in data collection and report writing, and the amount and type of information provided would be particularly valuable.

AB - Site visits are used extensively in a variety of settings within the evaluation community. They are especially common in making summative value decisions about the quality and worth of research programs/centers. However, there has been little empirical research and guidance about how to appropriately conduct evaluative site visits of research centers. We review the processes of two site visit examples using an expert panel review: (1) a process to evaluate four university research centers and (2) a process to review a federally sponsored research center. A set of 14 categories describing the expert panel review process was obtained through content analysis and participant observation. Most categories were addressed differently through the two processes highlighting the need for more research about the most effective processes to use within different contexts. Decisions about how to structure site visits appear to depend on the research context, practical considerations, the level at which the review is being conducted and the intended impact of the report. Future research pertaining to the selection of site visitors, the autonomy of the visitors in data collection and report writing, and the amount and type of information provided would be particularly valuable.

KW - Expert panel review/evaluation

KW - Research center

KW - Site visit

UR - http://www.scopus.com/inward/record.url?scp=84856488619&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84856488619&partnerID=8YFLogxK

U2 - 10.1016/j.evalprogplan.2012.01.003

DO - 10.1016/j.evalprogplan.2012.01.003

M3 - Article

C2 - 22306932

AN - SCOPUS:84856488619

VL - 35

SP - 390

EP - 397

JO - Evaluation and Program Planning

JF - Evaluation and Program Planning

SN - 0149-7189

IS - 3

ER -