Skip to Main Content

Systematic Review

Searching for existing reviews/protocols

It is important to conduct a preliminary search to see if others have done a review or are working towards a review on a similar topic. This will eliminate duplication and may help you identify/finalise a review question.

Find existing systematic reviews:

Cochrane Reviews: systematic reviews in the field of health care and health policy, available in the "Cochrane Library" database. 

Campbell Collaboration: an international social science research network providing useful resources including Campbell reviews, evidence and gap maps (EGMs), and Campbell systematic reviews journal.

CINAHAL: a library database providing full text of nursing and allied health journals.

DoPHER: hosted by the EPPI-Centre, DoPHER covers reviews of effectiveness in health promotion and public health.

Joanna Briggs Institute EBP Database (via OVID): primarily covers nursing, medical and allied health disciplines.

MEDLINE: access via OVID or EBSCO from the Library website.

OTseeker: a database with abstracts of systematic reviews and randomised controlled trials relevant to occupational therapy.

PEDro: covers over 50,000 trials, reviews and guidelines evaluating physiotherapy interventions.

PsycBITE: a database of systematic reviews, evidence-based guidelines, and randomised controlled trials (RCTs) in physiotherapy.

PsycINFO: a library database providing abstracts and citations of academic articles in the behavioural sciences and mental health.

PROSPERO: a database of prospectively registered systematic reviews in health and other areas where there is a health outcome.

peechBITE: a database of intervention studies across the scope of speech pathology practice.

PubMed: contains all MEDLINE content and additional resources, e.g. books, citations ahead of print and that predate MEDLINE.

Trip: a high quality clinical research evidence database.

Library databases: select Databases by Subject to retrieve systematic review articles form your subject databases. 

 

Find registered protocols

Cochrane protocols: find Cochrane registered protocols from Cochrane (Wiley) or Cochrane (via OVID, select Protocols under "Limits" before searching protocols). 

Joanna Briggs Institute EBP Database (via OVID): use keyword search and select "Systematic review protocols" in the Publication types.

Research Registry: search or browse registered systematic review protocols.

Develop a protocol

What is a systematic review protocol?

A systematic review protocol describes the stages of your systematic review process. A protocol usually includes objective, review question, score of the review, the research group, search strategy, methods, inclusion/exclusion criteria, quality assessment, procedures for data extraction and data synthesis, and project timeline etc. 

McMahon, E.J., Campbell, K.L., Bauer, J.D., Mudge, D.W., & Kelly, J.T.. (2021). Altered dietary salt intake for people with chronic kidney disease. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.CD010070.pub3

Khan, H. H., Mahrin, M. N. b., & Chuprat, S. b. (2013). Situational requirement engineering: A systematic literature review protocol. 2013 IEEE Conference on Open Systems (ICOS), 123-126. https://doi.org/10.1109/ICOS.2013.6735060

Azeem, M. I., & Khan, S. U. (2011, December 13-14). Intercultural challenges in offshore software development outsourcing relationships: A systematic literature review protocol. 2011 Malaysian Conference in Software Engineering, 475-480. https://doi.org/10.1109/MySEC.2011.6140719

Guidelines for preparing a protocol:  

You may find guidelines for other research fields online.  

Registration

Systematic reviews protocols, especially health/medical science related reviews, are often registered in registries at the protocol stage. This will avoid duplication and enable peer reviews. 

Registries:

PROSERO: an international database of prospectively registered systematic reviews in health and social care. Find out how to register your review with PROSERO and check the registration form from this webpage.

Cochrane reviews: register your title with a Cochrane Review Group first. Peer review and other supports will be given by the groups. 

Joanna Briggs Institute: for the use of JBI affiliated entities only.

BEME Collaboration: for medical and health reviews. Check out their review protocol guide.

Formulate a research question

Commonly systematic review questions are narrowly focused and well defined but can be more broadly formulated to scope the existing knowledge in a research area.

To formulate a question, elements of PICO(T) can be applied. PICO(T) or PECO(T) stands for the following:

P = people/population; problem 
I/E = intervention; exposure/event 
C = comparison; control
O = outcome  
T = time, if applicable  

The fewer elements you use, the broader the formulation of the question. 

Broad questions

  • can be asked to review what is generally known about a problem or topic, and 
  • are often used in a scoping review to provide a thorough understanding of the problem or topic 

Generally, a broad question contains two elements, e.g. an intervention or event (I/E) and population or problem (P).

For example, “What types of programmes can build mental resilience (I) among student nurses (P)?”

Narrow questions

A narrow question usually covers a specific topic and helps define the key terms used in searching databases. They tend to involve many of the PICO(t) elements, as it is illustrated below:

“"In patients undergoing a surgical biopsy (P), how does mindfulness meditation (I) compared with hypnosis (C) affect the use of pain killers (O) in the first 24 hours after the biopsy (T)?" 

This video (by the Medical College of Wisconsin Libraries) introduces how to use PICO elements to refine your research question.

Inclusion/exclusion criteria

Inclusion/exclusion criteria are important for systematic reviews. They are often applied to the creation of a search strategy and are used to define which studies will be included.

Noted that exclusion criteria are far harder to include as part of a search, and are most often applied at the article selecting/screening stage.   

Inclusion/exclusion criteria will ensure that:

  • your literature search will focus on your review question
  • selected studies will be able to answer your question
  • the selection bias will be minimised

When are they created?

Inclusion and exclusion criteria are most commonly determined after setting the research question and before conducting the search. A preliminary search in library databases is often conducted for determining these criteria.

It is possible to reconsider inclusion/exclusion criteria after seeing the search results.

Common inclusion and/or exclusion criteria

Inclusion/exclusion criteria often include the following details (but not limit to these): 

Date e.g.   only those items from 2011 to 2020
Geographic location e.g.   only items whose data includes New Zealand
Participants e.g.   limited to only a particular age or gender group; location of participants
Design/methodology e.g.   review limited to randomised control trials
Search methods e.g. identify databases and search strategy; some reviews may include grey literature
Types of interventions e.g. using products contained Manuka honey for wound dressing 

Where are they reported?

Inclusion and exclusion criteria are usually reported within the Methodology section of a review, possibly with a fuller explanation within an appendix.

The Cochrane handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis

Meline, T. (2006). Selecting studies for systemic review: Inclusion and exclusion criteria. Contemporary issues in communication science and disorders, 33(Spring), 21–27. https://doi.org/10.1044/cicsd_33_S_21 

University of Melbourne. (2021). Systematic reviews. https://unimelb.libguides.com/c.php?g=492361&p=3368110

Checklists & guidelines

Guidelines and checklists are critical tools for the stages of searching, selecting, appraising and reporting. 

Reduce the risk of biases:

  • ROBIS tool -  for assessing the risk of bias in systematic reviews
  • SYRCLE's ROB tool - by the Systematic Review Center for Laboratory Animal Experimentation.

Other useful guides:

Frameworks

The following are popular frameworks/models used by systematic reviews: 

PICO/PICOT/PICOS

P = people/population; problem
I/E = intervention; exposure/event
C = comparison; control
O = outcome
T = time, if applicable
S= study design

Other popular frameworks

PIC: Population, Phenomena of Interest, Context

SPIDER: Sample, Phenomenon of Interest, Design, Evaluation, Research type  

PEO: Population, Exposure, Outcomes

SPICE: Setting , Population or Perspective, Intervention, Comparison, Evaluation

Eriksen, M. B. (2018). The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: A systematic review. Journal of the Medical Library Association, 106(4), 420–431. https://doi.org/10.5195/jmla.2018.345

Methley, A. M., Campbell, S., Chew-Graham, C., McNall, R, Cheraghi, S. (2014). PICO, PICOS and SPIDR: A comparison stydy of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Services Research, 14, Article 579. https://doi.org/10.1186/s12913-014-0579-0

Recording SR process

A systematic review must include details of its review methodology. It is important to record each step of your search and selection process. One of the checklists and guidelines may be used or consulted. This will ensure that the systematic review is comprehensive, transparent and replicable. 

Documentation includes:

  • the search strategy for database searching and search results
  • the selection process
  • details of the studies included (and excluded) in the review 

PRISMA flow diagram:

The PRISMA flow diagram is widely used for reporting the process of systematic reviews.

Find more information on the PRISMA website.

 

Note. The PRISMA 2020 flow diagram template for systematic review was published in Page, et al. (2021), with a CC BY 4.0 license.

Reference

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal, 372, Article 71. https://doi.org/10.1136/bmj.n71 

Examples of recording sheets:

Useful tools

Managing your data

Systematic reviews are often conducted by a group of researchers. It is critical to maintain a good data management practice during your review process. Some simple guidelines can make a large difference:

  • file naming and location (primary and backup)
  • backups  (when, where, by whom)
  • reducing unnecessary duplication