Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases

Tatyana Shamliyan; Robert L Kane; Stacy Jansen

doi:10.1016/j.jclinepi.2011.10.017

Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases

Tatyana Shamliyan, Robert L Kane, Stacy Jansen

Health Policy and Management

Research output: Contribution to journal › Review article › peer-review

11 Scopus citations

Abstract

Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. Study Design and Setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.

Original language	English (US)
Pages (from-to)	610-618
Number of pages	9
Journal	Journal of Clinical Epidemiology
Volume	65
Issue number	6
DOIs	https://doi.org/10.1016/j.jclinepi.2011.10.017
State	Published - Jun 2012

Bibliographical note

Funding Information:
Conflict of interest statement: This project was funded under Contract No. 290-02-0009 , from the AHRQ, U.S. Department of Health and Human Services . The authors are responsible for its content. Statements in the study should not be construed as endorsement by the AHRQ or the U.S. Department of Health and Human Services. The authors do not have any conflict of interest.

Keywords

Bias (epidemiology)
Checklist/methods
Epidemiologic studies
Evidence-based medicine
Reproducibility of results
Review literature as topic
Risk factors

Publisher link

10.1016/j.jclinepi.2011.10.017

Find It

Find It at U of M Libraries

Cite this

@article{89d9652d938b40fb95c16bebef1896f1,

title = "Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases",

abstract = "Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. Study Design and Setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.",

keywords = "Bias (epidemiology), Checklist/methods, Epidemiologic studies, Evidence-based medicine, Reproducibility of results, Review literature as topic, Risk factors",

author = "Tatyana Shamliyan and Kane, {Robert L} and Stacy Jansen",

note = "Funding Information: Conflict of interest statement: This project was funded under Contract No. 290-02-0009 , from the AHRQ, U.S. Department of Health and Human Services . The authors are responsible for its content. Statements in the study should not be construed as endorsement by the AHRQ or the U.S. Department of Health and Human Services. The authors do not have any conflict of interest. ",

year = "2012",

month = jun,

doi = "10.1016/j.jclinepi.2011.10.017",

language = "English (US)",

volume = "65",

pages = "610--618",

journal = "Journal of Chronic Diseases",

issn = "0895-4356",

publisher = "Elsevier USA",

number = "6",

}

TY - JOUR

T1 - Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases

AU - Shamliyan, Tatyana

AU - Kane, Robert L

AU - Jansen, Stacy

N1 - Funding Information: Conflict of interest statement: This project was funded under Contract No. 290-02-0009 , from the AHRQ, U.S. Department of Health and Human Services . The authors are responsible for its content. Statements in the study should not be construed as endorsement by the AHRQ or the U.S. Department of Health and Human Services. The authors do not have any conflict of interest.

PY - 2012/6

Y1 - 2012/6

N2 - Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. Study Design and Setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.

AB - Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. Study Design and Setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.

KW - Bias (epidemiology)

KW - Checklist/methods

KW - Epidemiologic studies

KW - Evidence-based medicine

KW - Reproducibility of results

KW - Review literature as topic

KW - Risk factors

UR - http://www.scopus.com/inward/record.url?scp=84860237468&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860237468&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2011.10.017

DO - 10.1016/j.jclinepi.2011.10.017

M3 - Review article

C2 - 22424987

AN - SCOPUS:84860237468

VL - 65

SP - 610

EP - 618

JO - Journal of Chronic Diseases

JF - Journal of Chronic Diseases

SN - 0895-4356

IS - 6

ER -

Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this