Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. Study Design and Setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.
Bibliographical noteFunding Information:
Conflict of interest statement: This project was funded under Contract No. 290-02-0009 , from the AHRQ, U.S. Department of Health and Human Services . The authors are responsible for its content. Statements in the study should not be construed as endorsement by the AHRQ or the U.S. Department of Health and Human Services. The authors do not have any conflict of interest.
- Bias (epidemiology)
- Epidemiologic studies
- Evidence-based medicine
- Reproducibility of results
- Review literature as topic
- Risk factors