Skip to Main Content

Systematic Reviews and Knowledge Syntheses

Common Review Types

"A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. Researchers conducting systematic reviews use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making." Cochrane Library

Compared to traditional literature reviews, systematic reviews require a team of researchers and a significant investment in time to complete. For more information on how systematic reviews compare to traditional reviews please refer to the table in the Introduction to this guide.

Systematic approaches to literature reviews: Systematic reviews should not be confused with systematic approaches to literature reviews. Traditional literature reviews can be systematic in their approach and may incorporate some elements common to systematic reviews. For example, a traditional literature review can involve a comprehensive search of the literature. Full systematic reviews, however, require extensive searching across multiple resources for both published and nonpublished or grey literature. The search strategies used in a systematic review are often complex and must be clearly documented in the methods section of the review. For more information on the differences between systematic reviews and systematic approaches to literature reviews, please consult the following document from the University of Ottawa Library: 

Systematic reviews and systematic approaches to reviews

Steps to conducting a systematic review: Conducting a systematic review requires a great deal of planning and comprises several steps including: formulating the research question, developing a plan (protocol), searching for and screening studies for inclusion/exclusion, extracting data from selected studies, analyzing and synthesizing the data, and reporting the findings.

The following video from The Evidence Synthesis Academy outlines the steps of a systematic review:

 

The following infographic provides on overview of the sytematic review process:

An infographic titled: What authors DO. Consists of an illustration of a funnel with a wide opening at the top, narrowing to a small opening at the bottom. A strainer is positioned above the funnel. Research studies, as depicted by small squares, triangles and circles, are being sifted through the strainer. Only triangles are falling through the strainer. A systematic review, as depicted by a triangle composed of several small triangles, is positioned at the bottom of the funnel. The following steps are identified starting above the funnel and ending at the bottom of the funnel: Identify the issues and determine the question; Write a plan for the review (protocol); Search for studies; Sift and select studies; Extract data from the studies; Assess the quality of the studies; Combine the data (synthesis or meta-analysis); Discuss and conclude overall findings; Systematic Review; Dissemination.

Designed by Jessica Kaufman, Cochrane Consumers & Communication Review Group, Centre for Health Communication & Participation, La Trobe University, 2011. Licensed under Creative Commons CC BY-NC 4.0 DEED.

 

Systematic Review Standards and Guidelines

There are well-established standards and guidelines on how to conduct systematic reviews. Some of the most commonly used sources are listed below:

Campbell systematic reviews: Policies and guidelines (Version 1.8, February 2021). Campbell Collaboration. https://onlinelibrary.wiley.com/pb-assets/assets/18911803/Campbell%20Policies%20and%20Guidelines%20_May3%202022-1653054593497.pdf 

  • The Campbell Collaboration is an international network that supports the preparation and dissemination of high quality systematic reviews of research evidence on the effectiveness of social programs, policies, and practices.

 

Cochrane-Campbell handbook for qualitative evidence synthesis (Version 1.0, 2023). Cochrane. https://training.cochrane.org/cochrane-campbell-handbook-qualitative-evidence-synthesis

  • A guide for conducting systematic reviews of qualitative evidence with particular relevance to Cochrane and Campbell qualitative systematic reviews. Requires user to set-up a Cochrane account (free) to view chapters. 

 

Cochrane handbook for systematic reviews of interventions (Version 6.4, updated August 2023). Cochrane. https://www.training.cochrane.org/handbook.

  • The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions. 

 

Guidelines and standards for evidence synthesis in environmental management (Version 5.1, 2022). Collaboration for Environmental Evidence. https://environmentalevidence.org/information-for-authors/

  • Guidelines for the planning and conduct of CEE Evidence Syntheses in environmental management.

 

JBI manual for evidence synthesis (2024 Edition). JBI. https://jbi-global-wiki.refined.site/space/MANUAL

  • Provides guidance to authors for the conduct and preparation of JBI systematic and scoping reviews. 
 
Resources

Daniels, K. (2019). Guidance on conducting and reviewing systematic reviews (and meta-analysis) in work and organizational psychology. European Journal of Work and Organizational Psychology, 28(1), 1–10. https://doi.org/10.1080/1359432X.2018.1547708

National Collaborating Centre for Methods and Tools. (2017). Anatomy of a systematic review [Fact sheet]. http://www.
nccmt.ca/pubs/FactSheet_AnatomySR_EN_WEB.pdf.

Princeton University Library. (2023, December 27). Systematic reviews & meta-analyses: Standards & guidelines.  https://libguides.princeton.edu/c.php?g=1019850&p=9611752

Siddaway, A.P., Wood, A.M., & Hedges, L.V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70, 747–770. https://doi.org/10.1146/annurev-psych-010418-102803

A scoping review "provides a preliminary assessment of the potential size and scope of available research literature. It aims to identify the nature and extent of research evidence (usually including ongoing research)." (Grant & Booth, 2009, p. 101)

Reasons for conducting a scoping review:

  • To identify the types of available evidence in a given field
  • To clarify key concepts/definitions in the literature
  • To examine how research is conducted on a certain topic or field
  • To identify key characteristics or factors related to a concept
  • As a precursor to a systematic review
  • To identify and analyze knowledge gaps

(Munn et al., 2018, p. 2)

Like systematic reviews, scoping reviews are conducted using rigorous and transparent methods to systematically identify existing evidence in response to a question. However, there are key differences between the two types of reviews. The following video and resources provide guidance on when to use a scoping review and how scoping reviews compare to other types of evidence synthesis: 

(video by Carrie Price, Towson University, Albert S. Cook Library)

 
Resources
Arksey, H. & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616
 
Grant, M.J. & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26, 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
 
JBI Scoping Review Network
  • Contains resources and publications about scoping reviews including a link to the JBI Manual for Evidence Synthesis Chapter 10: Scoping reviews (Last updated 26 March, 2024).
 
Munn, Z., et al. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, Article 143. https://doi.org/10.1186/s12874-018-0611-x
 
Munn, Z., et al. (2022). What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesisJBI Evidence Synthesis, 20(4), 950–952. https://doi.org/10.11124/JBIES-21-00483

Peters, M.D.J., et al. (2021). Scoping reviews: Reinforcing and advancing the methodology and application. Systematic Reviews, 10, Article 263. https://doi.org/10.1186/s13643-021-01821-3
 
 

Rapid Reviews follow an expedited or simplified review process. Various terms have been used to describe rapid reviews (e.g., rapid evidence assessments, restricted reviews, etc.). The definitions and methods that characterize rapid reviews also vary. Hamel et al. (2021) suggest this broad definition: "A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting a variety of methods to produce evidence in a resource-efficient manner."

 

Resources

Barends, E., Rousseau, D.M . & Briner, R.B. (Eds). (2017). CEBMa guideline for rapid evidence assessments in management and organizations (Version 1.0). Center for Evidence Based Management. https://cebma.org/assets/Uploads/CEBMa-REA-Guideline.pdf

Dobbins, M. (2017). Rapid review guidebook. National Collaborating Centre for Methods and Tools. https://www.nccmt.ca/tools/rapid-review-guidebook

Garritty, C., et al. (2024). Updated recommendations for the Cochrane rapid review methods guidance for rapid reviews of effectiveness. BMJ, 384, Article e076335. https://doi.org/10.1136/bmj-2023-076335

Hamel, C., et al. (2021). Defining rapid reviews: A Systematic scoping review and thematic analysis of definitions and defining characteristics or rapid reviews. Journal of Clinical Epidemiology, 129, 74–85. https://doi.org/10.1016/j.jclinepi.2020.09.041

Tricco, A.C., Langlois, E.V., Straus, S.E., (Eds.). (2017).  Rapid reviews to strengthen health policy and systems: A practical guide. World Health Organization. https://ahpsr.who.int/publications/i/item/2017-08-10-rapid-reviews-to-strengthen-health-policy-and-systems-a-practical-guide

Tricco, A.C., et al. (2015). A scoping review of rapid review methods. BMC Medicine, 13, Article 224. https://doi.org/10.1186/s12916-015-0465-6

University of Sheffield. (2019). STARR Decision Tool: Selecting approaches for rapid review.  https://www.sheffield.ac.uk/scharr/research/themes/systematic-reviewing#STARR

Meta-analyses is the "statistical analysis of a collection of analysis results from individual studies for the purpose of integrating the findings." (Cooper et al., 2019, p. 532). A meta-analysis may be part of a systematic review, but it can also be a stand-alone study.

 

The following video from TARG (Tobacco and Alcohol Research Group), University of Bristol provides a brief overview of meta-analysis.

 
Resources

Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555, 175–182.  https://doi.org/10.1038/nature25753

Hansen, C., Steinmetz, H., & Block, J. (2022). How to conduct a meta-analysis in eight steps: A practical guideManagement Review Quarterly, 72, 1–19. https://doi.org/10.1007/s11301-021-00247-4

Meta-Essentials: Workbooks for meta-analysis (Erasmus Research Institute of Management, Erasmus University, Rotterdam)

  • A free tool that facilitates the integration and synthesis of effect sizes from different studies.

Methods of Synthesis and Integration Center (MOSAIC)

  • An initiative at the American Institutes for Research; provides resources and tools to assist with evidence synthesis and meta-analysis.

Practical Meta-Analysis Effect Size Calculator (Campbell Collaboration)

Umbrella reviews review and summarize the findings of existing systematic reviews or meta-analyses. Sometimes known by other names, such as overviews of reviews or reviews of reviews, “the aim of an umbrella review is not to repeat the searches, assessment of study eligibility, assessment of risk of bias or meta-analyses from the included reviews, but rather to provide an overall picture of findings for particular questions or phenomenon” (Aromataris, 2015, p. 133).

 

Resources

Aromataris, E., et al. (2020). Chapter 9: Umbrella reviews. In JBI manual for evidence synthesis. (Last updated 18 April, 2024). JBI. https://doi.org/10.46658/JBIMES-24-08

Aromataris, E., et al. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare, 13(3), 132–140. https://doi.org/10.1097/XEB.0000000000000055

Fusar-Poli, P., & Radua, J. (2018). Ten simple rules for conducting umbrella reviews. Evidence-Based Mental Health, 21(3), 95–100. https://doi.org/10.1136/ebmental-2018-300014

Gates, M., et al. (2020). Guidance for overviews of reviews continues to accumulate, but important challenges remain: A scoping review. Systematic Reviews, 9, Article 254. https://doi.org/10.1186/s13643-020-01509-0

Pollock, M., et al. (2023). Chapter V: Overviews of reviews. In Cochrane handbook for systematic reviews of interventions (Version 6.4 updated August 2023). Cochrane. https://training.cochrane.org/handbook/current/chapter-v