Skip to Main Content

Qualitative Evidence Synthesis (QES)

A guide intended to introduce researchers to the principles of QES, the associated methodologies, and guidance on engaging in this mode of evidence synthesis.

Data extraction

There are many ways to extract data from qualitative studies. Decisions about which approach to use may depend on several factors, including the number of studies that are under consideration and the availability of software that can assist. The research team will likely want to test drive different approaches before settling on the one that is most appropriate for your QES. Options available are as follows:

  1. A bespoke universal, standardized or adapted data extraction template: There are many of these available for adaptation or standard use.  This is a link to a Google Sheets template that you can modify as appropriate to suit the needs of your QES. HINT: click make a copy to create your own. Otherwise, search the web for "data extraction template" to see what other formats may be useful. 
  2. Using an a priori theory or predetermined framework to extract data: If you know that you will be using a framework synthesis, you may want to incorporate that into your extraction process. For more information on framework synthesis, read the box below. 
  3. Using a software program to code original studies inductively: If available to the team, you can consider using software like NVIVO, Covidence or JBI SUMARI. There is less flexibility for adaptation, however, if you are dealing with a large number of studies, and standardization of your data is crucial in your analysis, this may be a wise option. For an example of what this approach entails, take a look at the forms that are used in the JBI Qualitative data extraction tool

 

Choosing an approach to synthesis

It is recommended that you spend some time considering your topic and research question through the RETREAT framework in order to choose an appropriate approach to synthesis. The seven domains of the RETREAT framework are as follows:

Domain Definition
Review question A clear and detailed specification of the research question(s) to be addressed by the review
Epistemology The assumptions on the nature of knowledge that underpin the synthesis method and the extent to which these permit the review team to achieve their purpose
Time/timeframe Logistic constraints regarding the expected completion date of the synthesis and the cumulative amount of effort required to deliver the review
Resources Financial and physical support and infrastructure required to deliver the review
Expertise Knowledge and skill domains required by the review team and the wider network supporting the review
Audience and purpose Requirements and expectations of the intended recipients of the review and how review findings are intended to be used
Type of data The richness, thickness, type (quantitative/qualitative), quality, and quantity of data available to address the review question.

 

For further guidance on how to apply this framework, and to see some worked examples, read this article:

Booth, A., Noyes, J., Flemming, K., Gerhardus, A., Wahlster, P., van der Wilt, G. J., Mozygemba, K., Refolo, P., Sacchini, D., Tummers, M., & Rehfuess, E. (2018). Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches. Journal of Clinical Epidemiology, 99, 41–52. https://doi.org/10.1016/j.jclinepi.2018.03.003

Once you have worked through the RETREAT framework, you will be able to select the synthesis method that is best suited to your project goals. There are three synthesis methods that are most commonly used. They are outlined below. For more in-depth information on each method, read this article:

Flemming, K., & Noyes, J. (2021). Qualitative Evidence Synthesis: Where Are We at? International Journal of Qualitative Methods, 20, https://doi.org/10.1177/1609406921993276

 

Thematic synthesis

Thematic synthesis is perhaps the most commonly used approach. It is a three-stage approach involving preliminary line-by-line coding to identify potential areas of commonality, the subsequent development of descriptive themes, and finally - if the data is rich enough - the development of new frameworks, theories and/or hypotheses. It is an interpretive approach and is based on methods used in primary research and is ideal for novice reviewers due to its flexibility. However, for this same reason, it can tend toward simplification or reductive conclusions. 

An example of thematic synthesis in action:

Walker, R. C., Tong, A., Howard, K., & Palmer, S. C. (2019). Patient expectations and experiences of remote monitoring for chronic diseases: Systematic review and thematic synthesis of qualitative studies. International Journal of Medical Informatics (Shannon, Ireland), 124, 78–85. https://doi.org/10.1016/j.ijmedinf.2019.01.013

Further (and highly recommended) reading
Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8(1), 45–45. https://doi.org/10.1186/1471-2288-8-45

Framework synthesis

Also known as best fit framework synthesis, this is a deductive approach that makes use of an existing or a priori framework. These frameworks could be sourced from a previous review, policy framework, logic model or conceptual model. It is less suited for theory generation, but very appropriate for interpretation of what is happening in a particular setting or context. When a suitable framework is identified, it can be a time saver for the research team, however, if not readily available, the time spent searching for a relevant framework can be a drawback. 

An example of framework synthesis in action:

Trivedi, D. (2015). Cochrane review summary: barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Primary Health Care Research & Development, 16(4), 326–328. https://doi.org/10.1017/S1463423615000341
 

Meta-ethnography

Meta-ethnography is the oldest synthesis tradition, theorized originally by Noblitt and Hare in 1988. It is interpretive and seeks to create new understanding from distinct bodies of work. The meta-ethnographer deconstructs, compares and analyzes texts to create substantive and novel interpretations. There are three main approaches to meta-ethnography: reciprocal translation, refutational synthesis, and lines-of-argument synthesis. Deciding which approach to take will largely depend on the substance of the studies under consideration. Is there consensus? Different conclusions? Do the findings refute each other? The answers to these questions will help you determine which approach to take. 

When selecting this approach, the expertise and experience of the research team must be considered. Meta-ethnography demands an in-depth understanding of theory, the breadth and depth of a subject area, and sound knowledge and experience with qualitative methods. 

An example of meta-ethnography in action

Toye, F., Jenkins, C., & Barker, K. L. (2020). The Experience of Living to an Extreme Age: A Meta-Ethnography. Qualitative Health Research, 30(1), 3–22. https://doi.org/10.1177/1049732319880537

Further (and highly recommended) reading

Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography : synthesizing qualitative studies. SAGE. https://methods-sagepub-com.ezproxy.lib.torontomu.ca/book/meta-ethnography