Synthesising and summarising evidence

About how to synthesise and summarise the evidence for users

Overview of the topic

The Knowledge for Healthcare Framework (2021) identifies the growing demand for knowledge and library staff to search and present evidence that is synthesised and summarised.

Lots of terminology is used interchangeably to describe the products we create and share with users:

  • literature search
  • evidence search
  • literature review
  • evidence summary
  • evidence synthesis
  • evidence brief
  • search with summary
  • systematic review

the list goes on…

The range of language used, the way they overlap, and the inconsistencies in how they are applied, can make understanding the differences between them all difficult.

A “search” is the strategic approach to finding and collating evidence, references, and results, relevant to the question posed by a user.

Often searchers scan databases containing studies relevant to their topic, for example MEDLINE, a general medical database.

Searches will often also involve scanning other resources on the internet to retrieve “grey literature” (see separate Learning Hub page), for example case studies or reports.

The search process, and the collation of the results into a document, might be described as a literature search or evidence search, and would contain:

  • a list of the references retrieved
  • citation details
  • an abstract briefly describing the findings of each result

A search often includes a description of the search strategy, resources searched, and potentially some short comments from the searcher.

When we talk about summarising or synthesising, we refer to the process of taking the list of results or references retrieved as part of a literature search and manipulating them to produce a more detailed report or document that is sent to users to help with their decision-making.

A “summary” might include:

  • a description of the search strategy
  • comments from the searcher on the search process or results
  • brief descriptions of headlines from key results, or an indication of themes that have emerged.

Booth (2015) describes an evidence summary as functioning “like an honest broker who neutrally gathers factual information to support the decision-making process.” 

Summaries may or may not include a quality assessment of individual documents (Maden, 2021).

Summaries might range from a few sentences to a paragraph or two presented before the list of results.

A “synthesis” goes a step further. Analysis is undertaken across the result set, the findings from multiple studies are considered to identify themes, similarities, and differences, and they may or may not include quality assessment across multiple documents (Maden, 2021).

Synthesis is usually a key part of the process of a systemic review, a type of literature review where the best available evidence relating to a specific research question is located, appraised, and synthesised (Boland et al., 2017), and overall conclusions can be provided.

When producing a synthesis as part of a systematic review a quality assessment (also known as critical appraisal) of each individual study may be required using a checklist appropriate to the study design.

A list of checklists for critical appraisal can be found on the CASP website.

It is possible then for searchers to conduct a search and present:

  • the results in a list (search),
  • the results with additional commentary potentially highlighting key documents or themes, (summary)
  • an analysis of the results in more detail providing a more considered discussion of the results as a whole (synthesis).

Example from practice

Below are some examples of different kinds of searches, summaries, and synthesis to convey the wide variety of ways the results of a search can be pulled together and evaluated.

This example on “incidence of septicaemia in Covid patients” from the COVID-19 search bank includes some search notes with a list of retrieved results organised by result type, for example guidance, systematic reviews, and original research. Details of the search strategy and resources searched are included.

This search with summary to answer the question “does hospital admission for people with dementia lead to long term nursing/ residential care?” contains some comments from the searcher followed by key quotes and descriptions of conclusions from some of the papers retrieved. Details of search strategies and sources are again included.

This evidence synthesis on the “NHS Health Check Programme” includes the facets of the search questions, details on design (systematic review), and study selection, as well as inclusion and exclusion criteria for the papers included in the analysis followed by a detailed description of the results of the synthesis organised by relevant themes, for example “characteristics of people who have had an NHS Check”. Detailed descriptions of the search strategies and sources searched are included.

Based on the definitions above this example [Employee Reward and Recognition – Recent Research] is of a summary, almost synthesis, whilst this one [Use of patient restraints in critical care or HDU/ICU] is more of a brief summary/ search.

Resources

Blogs

Rapid-Reviews.info

Blog about rapid reviewing by Jon Brassey, who also runs the TRIP database [Last checked: 30/6/20]

Courses

GRADE Online Learning Modules (McMaster University)

Online learning modules to help guideline developers and authors of systematic reviews learn how to use the GRADE approach to grade the evidence in systematic reviews [Last checked: 30/6/20]

Sheffield University: SchARR (School of Health and Related Research)

Qualitative and quantitative synthesis. Fee paying bookable courses. [Last checked: 30/6/20]

E-books

Cochrane Library Handbook

The Cochrane Library contains six databases that contain different types of high-quality evidence. The Cochrane handbook explains how Cochrane reviews are produced [Last checked: 30/6/20]

University of York CRD: Systematic Reviews

In-depth guidance on how to carry out a systematic review [Last checked: 30/6/20]

Journal Articles

Adams, J. et al. (2016) Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: Critical reflections on three case studies. Systematic Reviews, 5 (1)

On three previous occasions, we have attempted to systematically search for and synthesise public health grey literature and information—both to summarise the extent and nature of particular classes of interventions and to synthesise results of evaluations. Here, we briefly describe these three ‘case studies’ but focus on our post hoc critical reflections on searching for and synthesising grey literature and information garnered from our experiences of these case studies. [Last checked: 30/6/20]

Booth, A. (2015). EVIDENT Guidance for Reviewing the evidence: a compendium of methodological literature and websites. DOI: 10.13140/RG.2.1.1562.9842

Dias, S. et al (2013) Evidence Synthesis for Decision Making, Parts 1 -7. Medical Decision Making

A series of seven tutorial papers on evidence synthesis methods for decision making. [Last checked: 30/6/20]

Websites

Evidence summaries process guide, NICE 

PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis)

Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. Provides a checklist and flow diagram. This also helps to develop understanding of research design. [Last checked: 30/6/20]

University of York Centre for Reviews and Dissemination (CRD)

CRD is a world renowned institute that produces policy relevant research and innovative methods that advance the use of research evidence to improve population health. [Last checked: 30/6/20]