Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

  • Formulate a research question
  • Develop a protocol
  • Search for all relevant studies
  • Apply the selection criteria
  • Extract the data
  • Synthesize the data
  • Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

  • Academic style
  • Vague sentences
  • Style consistency

See an example

what is systematic review research design

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

  • A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
  • Access to databases and journal archives. Often, your educational institution provides you with access.
  • Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
  • Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

  • They minimize research bias by considering all available evidence and evaluating each study for bias.
  • Their methods are transparent , so they can be scrutinized by others.
  • They’re thorough : they summarize all available evidence.
  • They can be replicated and updated by others.

Systematic reviews also have a few cons .

  • They’re time-consuming .
  • They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

  • Allow you to more effectively communicate your research to other researchers and practitioners
  • Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

  • Population(s) or problem(s)
  • Intervention(s)
  • Comparison(s)

You can rearrange these four components to write your research question:

  • What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

  • Type of study design(s)
  • The population of patients with eczema
  • The intervention of probiotics
  • In comparison to no treatment, placebo , or non-probiotic treatment
  • The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
  • Randomized control trials, a type of study design

Their research question was:

  • What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

  • Background information : Provide the context of the research question, including why it’s important.
  • Research objective (s) : Rephrase your research question as an objective.
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

  • Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
  • Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
  • Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
  • Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

  • Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
  • Handsearch: Conference proceedings and reference lists of articles
  • Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
  • Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

  • Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
  • Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

  • Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
  • Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

  • Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
  • Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

  • Abstract : A summary of the review
  • Introduction : Including the rationale and objectives
  • Methods : Including the selection criteria, search method, data extraction method, and synthesis method
  • Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
  • Discussion : Including interpretation of the results and limitations of the review
  • Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

  • Student’s  t -distribution
  • Normal distribution
  • Null and Alternative Hypotheses
  • Chi square tests
  • Confidence interval
  • Quartiles & Quantiles
  • Cluster sampling
  • Stratified sampling
  • Data cleansing
  • Reproducibility vs Replicability
  • Peer review
  • Prospective cohort study

Research bias

  • Implicit bias
  • Cognitive bias
  • Placebo effect
  • Hawthorne effect
  • Hindsight bias
  • Affect heuristic
  • Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved February 19, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

  • Knowledge Base
  • Methodology
  • Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 17 October 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

  • Formulate a research question
  • Develop a protocol
  • Search for all relevant studies
  • Apply the selection criteria
  • Extract the data
  • Synthesise the data
  • Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

  • A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
  • Access to databases and journal archives. Often, your educational institution provides you with access.
  • Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
  • Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

  • They minimise research b ias by considering all available evidence and evaluating each study for bias.
  • Their methods are transparent , so they can be scrutinised by others.
  • They’re thorough : they summarise all available evidence.
  • They can be replicated and updated by others.

Systematic reviews also have a few cons .

  • They’re time-consuming .
  • They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

  • Allow you to more effectively communicate your research to other researchers and practitioners
  • Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

  • Population(s) or problem(s)
  • Intervention(s)
  • Comparison(s)

You can rearrange these four components to write your research question:

  • What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

  • Type of study design(s)
  • The population of patients with eczema
  • The intervention of probiotics
  • In comparison to no treatment, placebo , or non-probiotic treatment
  • The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
  • Randomised control trials, a type of study design

Their research question was:

  • What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

  • Background information : Provide the context of the research question, including why it’s important.
  • Research objective(s) : Rephrase your research question as an objective.
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

  • Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
  • Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
  • Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
  • Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

  • Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
  • Handsearch: Conference proceedings and reference lists of articles
  • Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
  • Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

  • Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
  • Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

  • Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
  • Your judgement of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

  • Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
  • Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

  • Abstract : A summary of the review
  • Introduction : Including the rationale and objectives
  • Methods : Including the selection criteria, search method, data extraction method, and synthesis method
  • Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
  • Discussion : Including interpretation of the results and limitations of the review
  • Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarise yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2022, October 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 19 February 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

Study Design 101: Systematic Review

  • Case Report
  • Case Control Study
  • Cohort Study
  • Randomized Controlled Trial
  • Practice Guideline
  • Systematic Review
  • Meta-Analysis
  • Helpful Formulas
  • Finding Specific Study Types

A document often written by a panel that provides a comprehensive review of all relevant studies on a particular clinical or health-related topic/question. The systematic review is created after reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings.

  • Exhaustive review of the current literature and other sources (unpublished studies, ongoing research)
  • Less costly to review prior studies than to create a new study
  • Less time required than conducting a new study
  • Results can be generalized and extrapolated into the general population more broadly than individual studies
  • More reliable and accurate than individual studies
  • Considered an evidence-based resource

Disadvantages

  • Very time-consuming
  • May not be easy to combine studies

Design pitfalls to look out for

Studies included in systematic reviews may be of varying study designs, but should collectively be studying the same outcome.

Is each study included in the review studying the same variables?

Some reviews may group and analyze studies by variables such as age and gender; factors that were not allocated to participants.

Do the analyses in the systematic review fit the variables being studied in the original studies?

Fictitious Example

Does the regular wearing of ultraviolet-blocking sunscreen prevent melanoma? An exhaustive literature search was conducted, resulting in 54 studies on sunscreen and melanoma. Each study was then evaluated to determine whether the study focused specifically on ultraviolet-blocking sunscreen and melanoma prevention; 30 of the 54 studies were retained. The thirty studies were reviewed and showed a strong positive relationship between daily wearing of sunscreen and a reduced diagnosis of melanoma.

Real-life Examples

Yang, J., Chen, J., Yang, M., Yu, S., Ying, L., Liu, G., ... Liang, F. (2018). Acupuncture for hypertension. The Cochrane Database of Systematic Reviews, 11 (11), CD008821. https://doi.org/10.1002/14651858.CD008821.pub2

This systematic review analyzed twenty-two randomized controlled trials to determine whether acupuncture is a safe and effective way to lower blood pressure in adults with primary hypertension. Due to the low quality of evidence in these studies and lack of blinding, it is not possible to link any short-term decrease in blood pressure to the use of acupuncture. Additional research is needed to determine if there is an effect due to acupuncture that lasts at least seven days.

Parker, H.W. and Vadiveloo, M.K. (2019). Diet quality of vegetarian diets compared with nonvegetarian diets: a systematic review. Nutrition Reviews , https://doi.org/10.1093/nutrit/nuy067

This systematic review was interested in comparing the diet quality of vegetarian and non-vegetarian diets. Twelve studies were included. Vegetarians more closely met recommendations for total fruit, whole grains, seafood and plant protein, and sodium intake. In nine of the twelve studies, vegetarians had higher overall diet quality compared to non-vegetarians. These findings may explain better health outcomes in vegetarians, but additional research is needed to remove any possible confounding variables.

Related Terms

Cochrane Database of Systematic Reviews

A highly-regarded database of systematic reviews prepared by The Cochrane Collaboration , an international group of individuals and institutions who review and analyze the published literature.

Exclusion Criteria

The set of conditions that characterize some individuals which result in being excluded in the study (i.e. other health conditions, taking specific medications, etc.). Since systematic reviews seek to include all relevant studies, exclusion criteria are not generally utilized in this situation.

Inclusion Criteria

The set of conditions that studies must meet to be included in the review (or for individual studies - the set of conditions that participants must meet to be included in the study; often comprises age, gender, disease type and status, etc.).

Now test yourself!

1. Systematic Reviews are similar to Meta-Analyses, except they do not include a statistical analysis quantitatively combining all the studies.

a) True b) False

2. The panels writing Systematic Reviews may include which of the following publication types in their review?

a) Published studies b) Unpublished studies c) Cohort studies d) Randomized Controlled Trials e) All of the above

Evidence Pyramid - Navigation

  • Meta- Analysis
  • Case Reports
  • << Previous: Practice Guideline
  • Next: Meta-Analysis >>

Creative Commons License

  • Last Updated: Sep 25, 2023 10:59 AM
  • URL: https://guides.himmelfarb.gwu.edu/studydesign101

GW logo

  • Himmelfarb Intranet
  • Privacy Notice
  • Terms of Use
  • GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
  • Himmelfarb Health Sciences Library
  • 2300 Eye St., NW, Washington, DC 20037
  • Phone: (202) 994-2850
  • [email protected]
  • https://himmelfarb.gwu.edu
  • Open access
  • Published: 01 August 2019

A step by step guide for conducting a systematic review and meta-analysis with simulation data

  • Gehad Mohamed Tawfik 1 , 2 ,
  • Kadek Agus Surya Dila 2 , 3 ,
  • Muawia Yousif Fadlelmola Mohamed 2 , 4 ,
  • Dao Ngoc Hien Tam 2 , 5 ,
  • Nguyen Dang Kien 2 , 6 ,
  • Ali Mahmoud Ahmed 2 , 7 &
  • Nguyen Tien Huy 8 , 9 , 10  

Tropical Medicine and Health volume  47 , Article number:  46 ( 2019 ) Cite this article

1.71m Accesses

254 Citations

94 Altmetric

Metrics details

The massive abundance of studies relating to tropical medicine and health has increased strikingly over the last few decades. In the field of tropical medicine and health, a well-conducted systematic review and meta-analysis (SR/MA) is considered a feasible solution for keeping clinicians abreast of current evidence-based medicine. Understanding of SR/MA steps is of paramount importance for its conduction. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, this methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly conduct a SR/MA, in which all the steps here depicts our experience and expertise combined with the already well-known and accepted international guidance.

We suggest that all steps of SR/MA should be done independently by 2–3 reviewers’ discussion, to ensure data quality and accuracy.

SR/MA steps include the development of research question, forming criteria, search strategy, searching databases, protocol registration, title, abstract, full-text screening, manual searching, extracting data, quality assessment, data checking, statistical analysis, double data checking, and manuscript writing.

Introduction

The amount of studies published in the biomedical literature, especially tropical medicine and health, has increased strikingly over the last few decades. This massive abundance of literature makes clinical medicine increasingly complex, and knowledge from various researches is often needed to inform a particular clinical decision. However, available studies are often heterogeneous with regard to their design, operational quality, and subjects under study and may handle the research question in a different way, which adds to the complexity of evidence and conclusion synthesis [ 1 ].

Systematic review and meta-analyses (SR/MAs) have a high level of evidence as represented by the evidence-based pyramid. Therefore, a well-conducted SR/MA is considered a feasible solution in keeping health clinicians ahead regarding contemporary evidence-based medicine.

Differing from a systematic review, unsystematic narrative review tends to be descriptive, in which the authors select frequently articles based on their point of view which leads to its poor quality. A systematic review, on the other hand, is defined as a review using a systematic method to summarize evidence on questions with a detailed and comprehensive plan of study. Furthermore, despite the increasing guidelines for effectively conducting a systematic review, we found that basic steps often start from framing question, then identifying relevant work which consists of criteria development and search for articles, appraise the quality of included studies, summarize the evidence, and interpret the results [ 2 , 3 ]. However, those simple steps are not easy to be reached in reality. There are many troubles that a researcher could be struggled with which has no detailed indication.

Conducting a SR/MA in tropical medicine and health may be difficult especially for young researchers; therefore, understanding of its essential steps is crucial. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, we recommend a flow diagram (Fig. 1 ) which illustrates a detailed and step-by-step the stages for SR/MA studies. This methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly and succinctly conduct a SR/MA; all the steps here depicts our experience and expertise combined with the already well known and accepted international guidance.

figure 1

Detailed flow diagram guideline for systematic review and meta-analysis steps. Note : Star icon refers to “2–3 reviewers screen independently”

Methods and results

Detailed steps for conducting any systematic review and meta-analysis.

We searched the methods reported in published SR/MA in tropical medicine and other healthcare fields besides the published guidelines like Cochrane guidelines {Higgins, 2011 #7} [ 4 ] to collect the best low-bias method for each step of SR/MA conduction steps. Furthermore, we used guidelines that we apply in studies for all SR/MA steps. We combined these methods in order to conclude and conduct a detailed flow diagram that shows the SR/MA steps how being conducted.

Any SR/MA must follow the widely accepted Preferred Reporting Items for Systematic Review and Meta-analysis statement (PRISMA checklist 2009) (Additional file 5 : Table S1) [ 5 ].

We proposed our methods according to a valid explanatory simulation example choosing the topic of “evaluating safety of Ebola vaccine,” as it is known that Ebola is a very rare tropical disease but fatal. All the explained methods feature the standards followed internationally, with our compiled experience in the conduct of SR beside it, which we think proved some validity. This is a SR under conduct by a couple of researchers teaming in a research group, moreover, as the outbreak of Ebola which took place (2013–2016) in Africa resulted in a significant mortality and morbidity. Furthermore, since there are many published and ongoing trials assessing the safety of Ebola vaccines, we thought this would provide a great opportunity to tackle this hotly debated issue. Moreover, Ebola started to fire again and new fatal outbreak appeared in the Democratic Republic of Congo since August 2018, which caused infection to more than 1000 people according to the World Health Organization, and 629 people have been killed till now. Hence, it is considered the second worst Ebola outbreak, after the first one in West Africa in 2014 , which infected more than 26,000 and killed about 11,300 people along outbreak course.

Research question and objectives

Like other study designs, the research question of SR/MA should be feasible, interesting, novel, ethical, and relevant. Therefore, a clear, logical, and well-defined research question should be formulated. Usually, two common tools are used: PICO or SPIDER. PICO (Population, Intervention, Comparison, Outcome) is used mostly in quantitative evidence synthesis. Authors demonstrated that PICO holds more sensitivity than the more specific SPIDER approach [ 6 ]. SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) was proposed as a method for qualitative and mixed methods search.

We here recommend a combined approach of using either one or both the SPIDER and PICO tools to retrieve a comprehensive search depending on time and resources limitations. When we apply this to our assumed research topic, being of qualitative nature, the use of SPIDER approach is more valid.

PICO is usually used for systematic review and meta-analysis of clinical trial study. For the observational study (without intervention or comparator), in many tropical and epidemiological questions, it is usually enough to use P (Patient) and O (outcome) only to formulate a research question. We must indicate clearly the population (P), then intervention (I) or exposure. Next, it is necessary to compare (C) the indicated intervention with other interventions, i.e., placebo. Finally, we need to clarify which are our relevant outcomes.

To facilitate comprehension, we choose the Ebola virus disease (EVD) as an example. Currently, the vaccine for EVD is being developed and under phase I, II, and III clinical trials; we want to know whether this vaccine is safe and can induce sufficient immunogenicity to the subjects.

An example of a research question for SR/MA based on PICO for this issue is as follows: How is the safety and immunogenicity of Ebola vaccine in human? (P: healthy subjects (human), I: vaccination, C: placebo, O: safety or adverse effects)

Preliminary research and idea validation

We recommend a preliminary search to identify relevant articles, ensure the validity of the proposed idea, avoid duplication of previously addressed questions, and assure that we have enough articles for conducting its analysis. Moreover, themes should focus on relevant and important health-care issues, consider global needs and values, reflect the current science, and be consistent with the adopted review methods. Gaining familiarity with a deep understanding of the study field through relevant videos and discussions is of paramount importance for better retrieval of results. If we ignore this step, our study could be canceled whenever we find out a similar study published before. This means we are wasting our time to deal with a problem that has been tackled for a long time.

To do this, we can start by doing a simple search in PubMed or Google Scholar with search terms Ebola AND vaccine. While doing this step, we identify a systematic review and meta-analysis of determinant factors influencing antibody response from vaccination of Ebola vaccine in non-human primate and human [ 7 ], which is a relevant paper to read to get a deeper insight and identify gaps for better formulation of our research question or purpose. We can still conduct systematic review and meta-analysis of Ebola vaccine because we evaluate safety as a different outcome and different population (only human).

Inclusion and exclusion criteria

Eligibility criteria are based on the PICO approach, study design, and date. Exclusion criteria mostly are unrelated, duplicated, unavailable full texts, or abstract-only papers. These exclusions should be stated in advance to refrain the researcher from bias. The inclusion criteria would be articles with the target patients, investigated interventions, or the comparison between two studied interventions. Briefly, it would be articles which contain information answering our research question. But the most important is that it should be clear and sufficient information, including positive or negative, to answer the question.

For the topic we have chosen, we can make inclusion criteria: (1) any clinical trial evaluating the safety of Ebola vaccine and (2) no restriction regarding country, patient age, race, gender, publication language, and date. Exclusion criteria are as follows: (1) study of Ebola vaccine in non-human subjects or in vitro studies; (2) study with data not reliably extracted, duplicate, or overlapping data; (3) abstract-only papers as preceding papers, conference, editorial, and author response theses and books; (4) articles without available full text available; and (5) case reports, case series, and systematic review studies. The PRISMA flow diagram template that is used in SR/MA studies can be found in Fig. 2 .

figure 2

PRISMA flow diagram of studies’ screening and selection

Search strategy

A standard search strategy is used in PubMed, then later it is modified according to each specific database to get the best relevant results. The basic search strategy is built based on the research question formulation (i.e., PICO or PICOS). Search strategies are constructed to include free-text terms (e.g., in the title and abstract) and any appropriate subject indexing (e.g., MeSH) expected to retrieve eligible studies, with the help of an expert in the review topic field or an information specialist. Additionally, we advise not to use terms for the Outcomes as their inclusion might hinder the database being searched to retrieve eligible studies because the used outcome is not mentioned obviously in the articles.

The improvement of the search term is made while doing a trial search and looking for another relevant term within each concept from retrieved papers. To search for a clinical trial, we can use these descriptors in PubMed: “clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH terms] OR “clinical trial”[All Fields]. After some rounds of trial and refinement of search term, we formulate the final search term for PubMed as follows: (ebola OR ebola virus OR ebola virus disease OR EVD) AND (vaccine OR vaccination OR vaccinated OR immunization) AND (“clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH Terms] OR “clinical trial”[All Fields]). Because the study for this topic is limited, we do not include outcome term (safety and immunogenicity) in the search term to capture more studies.

Search databases, import all results to a library, and exporting to an excel sheet

According to the AMSTAR guidelines, at least two databases have to be searched in the SR/MA [ 8 ], but as you increase the number of searched databases, you get much yield and more accurate and comprehensive results. The ordering of the databases depends mostly on the review questions; being in a study of clinical trials, you will rely mostly on Cochrane, mRCTs, or International Clinical Trials Registry Platform (ICTRP). Here, we propose 12 databases (PubMed, Scopus, Web of Science, EMBASE, GHL, VHL, Cochrane, Google Scholar, Clinical trials.gov , mRCTs, POPLINE, and SIGLE), which help to cover almost all published articles in tropical medicine and other health-related fields. Among those databases, POPLINE focuses on reproductive health. Researchers should consider to choose relevant database according to the research topic. Some databases do not support the use of Boolean or quotation; otherwise, there are some databases that have special searching way. Therefore, we need to modify the initial search terms for each database to get appreciated results; therefore, manipulation guides for each online database searches are presented in Additional file 5 : Table S2. The detailed search strategy for each database is found in Additional file 5 : Table S3. The search term that we created in PubMed needs customization based on a specific characteristic of the database. An example for Google Scholar advanced search for our topic is as follows:

With all of the words: ebola virus

With at least one of the words: vaccine vaccination vaccinated immunization

Where my words occur: in the title of the article

With all of the words: EVD

Finally, all records are collected into one Endnote library in order to delete duplicates and then to it export into an excel sheet. Using remove duplicating function with two options is mandatory. All references which have (1) the same title and author, and published in the same year, and (2) the same title and author, and published in the same journal, would be deleted. References remaining after this step should be exported to an excel file with essential information for screening. These could be the authors’ names, publication year, journal, DOI, URL link, and abstract.

Protocol writing and registration

Protocol registration at an early stage guarantees transparency in the research process and protects from duplication problems. Besides, it is considered a documented proof of team plan of action, research question, eligibility criteria, intervention/exposure, quality assessment, and pre-analysis plan. It is recommended that researchers send it to the principal investigator (PI) to revise it, then upload it to registry sites. There are many registry sites available for SR/MA like those proposed by Cochrane and Campbell collaborations; however, we recommend registering the protocol into PROSPERO as it is easier. The layout of a protocol template, according to PROSPERO, can be found in Additional file 5 : File S1.

Title and abstract screening

Decisions to select retrieved articles for further assessment are based on eligibility criteria, to minimize the chance of including non-relevant articles. According to the Cochrane guidance, two reviewers are a must to do this step, but as for beginners and junior researchers, this might be tiresome; thus, we propose based on our experience that at least three reviewers should work independently to reduce the chance of error, particularly in teams with a large number of authors to add more scrutiny and ensure proper conduct. Mostly, the quality with three reviewers would be better than two, as two only would have different opinions from each other, so they cannot decide, while the third opinion is crucial. And here are some examples of systematic reviews which we conducted following the same strategy (by a different group of researchers in our research group) and published successfully, and they feature relevant ideas to tropical medicine and disease [ 9 , 10 , 11 ].

In this step, duplications will be removed manually whenever the reviewers find them out. When there is a doubt about an article decision, the team should be inclusive rather than exclusive, until the main leader or PI makes a decision after discussion and consensus. All excluded records should be given exclusion reasons.

Full text downloading and screening

Many search engines provide links for free to access full-text articles. In case not found, we can search in some research websites as ResearchGate, which offer an option of direct full-text request from authors. Additionally, exploring archives of wanted journals, or contacting PI to purchase it if available. Similarly, 2–3 reviewers work independently to decide about included full texts according to eligibility criteria, with reporting exclusion reasons of articles. In case any disagreement has occurred, the final decision has to be made by discussion.

Manual search

One has to exhaust all possibilities to reduce bias by performing an explicit hand-searching for retrieval of reports that may have been dropped from first search [ 12 ]. We apply five methods to make manual searching: searching references from included studies/reviews, contacting authors and experts, and looking at related articles/cited articles in PubMed and Google Scholar.

We describe here three consecutive methods to increase and refine the yield of manual searching: firstly, searching reference lists of included articles; secondly, performing what is known as citation tracking in which the reviewers track all the articles that cite each one of the included articles, and this might involve electronic searching of databases; and thirdly, similar to the citation tracking, we follow all “related to” or “similar” articles. Each of the abovementioned methods can be performed by 2–3 independent reviewers, and all the possible relevant article must undergo further scrutiny against the inclusion criteria, after following the same records yielded from electronic databases, i.e., title/abstract and full-text screening.

We propose an independent reviewing by assigning each member of the teams a “tag” and a distinct method, to compile all the results at the end for comparison of differences and discussion and to maximize the retrieval and minimize the bias. Similarly, the number of included articles has to be stated before addition to the overall included records.

Data extraction and quality assessment

This step entitles data collection from included full-texts in a structured extraction excel sheet, which is previously pilot-tested for extraction using some random studies. We recommend extracting both adjusted and non-adjusted data because it gives the most allowed confounding factor to be used in the analysis by pooling them later [ 13 ]. The process of extraction should be executed by 2–3 independent reviewers. Mostly, the sheet is classified into the study and patient characteristics, outcomes, and quality assessment (QA) tool.

Data presented in graphs should be extracted by software tools such as Web plot digitizer [ 14 ]. Most of the equations that can be used in extraction prior to analysis and estimation of standard deviation (SD) from other variables is found inside Additional file 5 : File S2 with their references as Hozo et al. [ 15 ], Xiang et al. [ 16 ], and Rijkom et al. [ 17 ]. A variety of tools are available for the QA, depending on the design: ROB-2 Cochrane tool for randomized controlled trials [ 18 ] which is presented as Additional file 1 : Figure S1 and Additional file 2 : Figure S2—from a previous published article data—[ 19 ], NIH tool for observational and cross-sectional studies [ 20 ], ROBINS-I tool for non-randomize trials [ 21 ], QUADAS-2 tool for diagnostic studies, QUIPS tool for prognostic studies, CARE tool for case reports, and ToxRtool for in vivo and in vitro studies. We recommend that 2–3 reviewers independently assess the quality of the studies and add to the data extraction form before the inclusion into the analysis to reduce the risk of bias. In the NIH tool for observational studies—cohort and cross-sectional—as in this EBOLA case, to evaluate the risk of bias, reviewers should rate each of the 14 items into dichotomous variables: yes, no, or not applicable. An overall score is calculated by adding all the items scores as yes equals one, while no and NA equals zero. A score will be given for every paper to classify them as poor, fair, or good conducted studies, where a score from 0–5 was considered poor, 6–9 as fair, and 10–14 as good.

In the EBOLA case example above, authors can extract the following information: name of authors, country of patients, year of publication, study design (case report, cohort study, or clinical trial or RCT), sample size, the infected point of time after EBOLA infection, follow-up interval after vaccination time, efficacy, safety, adverse effects after vaccinations, and QA sheet (Additional file 6 : Data S1).

Data checking

Due to the expected human error and bias, we recommend a data checking step, in which every included article is compared with its counterpart in an extraction sheet by evidence photos, to detect mistakes in data. We advise assigning articles to 2–3 independent reviewers, ideally not the ones who performed the extraction of those articles. When resources are limited, each reviewer is assigned a different article than the one he extracted in the previous stage.

Statistical analysis

Investigators use different methods for combining and summarizing findings of included studies. Before analysis, there is an important step called cleaning of data in the extraction sheet, where the analyst organizes extraction sheet data in a form that can be read by analytical software. The analysis consists of 2 types namely qualitative and quantitative analysis. Qualitative analysis mostly describes data in SR studies, while quantitative analysis consists of two main types: MA and network meta-analysis (NMA). Subgroup, sensitivity, cumulative analyses, and meta-regression are appropriate for testing whether the results are consistent or not and investigating the effect of certain confounders on the outcome and finding the best predictors. Publication bias should be assessed to investigate the presence of missing studies which can affect the summary.

To illustrate basic meta-analysis, we provide an imaginary data for the research question about Ebola vaccine safety (in terms of adverse events, 14 days after injection) and immunogenicity (Ebola virus antibodies rise in geometric mean titer, 6 months after injection). Assuming that from searching and data extraction, we decided to do an analysis to evaluate Ebola vaccine “A” safety and immunogenicity. Other Ebola vaccines were not meta-analyzed because of the limited number of studies (instead, it will be included for narrative review). The imaginary data for vaccine safety meta-analysis can be accessed in Additional file 7 : Data S2. To do the meta-analysis, we can use free software, such as RevMan [ 22 ] or R package meta [ 23 ]. In this example, we will use the R package meta. The tutorial of meta package can be accessed through “General Package for Meta-Analysis” tutorial pdf [ 23 ]. The R codes and its guidance for meta-analysis done can be found in Additional file 5 : File S3.

For the analysis, we assume that the study is heterogenous in nature; therefore, we choose a random effect model. We did an analysis on the safety of Ebola vaccine A. From the data table, we can see some adverse events occurring after intramuscular injection of vaccine A to the subject of the study. Suppose that we include six studies that fulfill our inclusion criteria. We can do a meta-analysis for each of the adverse events extracted from the studies, for example, arthralgia, from the results of random effect meta-analysis using the R meta package.

From the results shown in Additional file 3 : Figure S3, we can see that the odds ratio (OR) of arthralgia is 1.06 (0.79; 1.42), p value = 0.71, which means that there is no association between the intramuscular injection of Ebola vaccine A and arthralgia, as the OR is almost one, and besides, the P value is insignificant as it is > 0.05.

In the meta-analysis, we can also visualize the results in a forest plot. It is shown in Fig. 3 an example of a forest plot from the simulated analysis.

figure 3

Random effect model forest plot for comparison of vaccine A versus placebo

From the forest plot, we can see six studies (A to F) and their respective OR (95% CI). The green box represents the effect size (in this case, OR) of each study. The bigger the box means the study weighted more (i.e., bigger sample size). The blue diamond shape represents the pooled OR of the six studies. We can see the blue diamond cross the vertical line OR = 1, which indicates no significance for the association as the diamond almost equalized in both sides. We can confirm this also from the 95% confidence interval that includes one and the p value > 0.05.

For heterogeneity, we see that I 2 = 0%, which means no heterogeneity is detected; the study is relatively homogenous (it is rare in the real study). To evaluate publication bias related to the meta-analysis of adverse events of arthralgia, we can use the metabias function from the R meta package (Additional file 4 : Figure S4) and visualization using a funnel plot. The results of publication bias are demonstrated in Fig. 4 . We see that the p value associated with this test is 0.74, indicating symmetry of the funnel plot. We can confirm it by looking at the funnel plot.

figure 4

Publication bias funnel plot for comparison of vaccine A versus placebo

Looking at the funnel plot, the number of studies at the left and right side of the funnel plot is the same; therefore, the plot is symmetry, indicating no publication bias detected.

Sensitivity analysis is a procedure used to discover how different values of an independent variable will influence the significance of a particular dependent variable by removing one study from MA. If all included study p values are < 0.05, hence, removing any study will not change the significant association. It is only performed when there is a significant association, so if the p value of MA done is 0.7—more than one—the sensitivity analysis is not needed for this case study example. If there are 2 studies with p value > 0.05, removing any of the two studies will result in a loss of the significance.

Double data checking

For more assurance on the quality of results, the analyzed data should be rechecked from full-text data by evidence photos, to allow an obvious check for the PI of the study.

Manuscript writing, revision, and submission to a journal

Writing based on four scientific sections: introduction, methods, results, and discussion, mostly with a conclusion. Performing a characteristic table for study and patient characteristics is a mandatory step which can be found as a template in Additional file 5 : Table S3.

After finishing the manuscript writing, characteristics table, and PRISMA flow diagram, the team should send it to the PI to revise it well and reply to his comments and, finally, choose a suitable journal for the manuscript which fits with considerable impact factor and fitting field. We need to pay attention by reading the author guidelines of journals before submitting the manuscript.

The role of evidence-based medicine in biomedical research is rapidly growing. SR/MAs are also increasing in the medical literature. This paper has sought to provide a comprehensive approach to enable reviewers to produce high-quality SR/MAs. We hope that readers could gain general knowledge about how to conduct a SR/MA and have the confidence to perform one, although this kind of study requires complex steps compared to narrative reviews.

Having the basic steps for conduction of MA, there are many advanced steps that are applied for certain specific purposes. One of these steps is meta-regression which is performed to investigate the association of any confounder and the results of the MA. Furthermore, there are other types rather than the standard MA like NMA and MA. In NMA, we investigate the difference between several comparisons when there were not enough data to enable standard meta-analysis. It uses both direct and indirect comparisons to conclude what is the best between the competitors. On the other hand, mega MA or MA of patients tend to summarize the results of independent studies by using its individual subject data. As a more detailed analysis can be done, it is useful in conducting repeated measure analysis and time-to-event analysis. Moreover, it can perform analysis of variance and multiple regression analysis; however, it requires homogenous dataset and it is time-consuming in conduct [ 24 ].

Conclusions

Systematic review/meta-analysis steps include development of research question and its validation, forming criteria, search strategy, searching databases, importing all results to a library and exporting to an excel sheet, protocol writing and registration, title and abstract screening, full-text screening, manual searching, extracting data and assessing its quality, data checking, conducting statistical analysis, double data checking, manuscript writing, revising, and submitting to a journal.

Availability of data and materials

Not applicable.

Abbreviations

Network meta-analysis

Principal investigator

Population, Intervention, Comparison, Outcome

Preferred Reporting Items for Systematic Review and Meta-analysis statement

Quality assessment

Sample, Phenomenon of Interest, Design, Evaluation, Research type

Systematic review and meta-analyses

Bello A, Wiebe N, Garg A, Tonelli M. Evidence-based decision-making 2: systematic reviews and meta-analysis. Methods Mol Biol (Clifton, NJ). 2015;1281:397–416.

Article   Google Scholar  

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Rys P, Wladysiuk M, Skrzekowska-Baran I, Malecki MT. Review articles, systematic reviews and meta-analyses: which can be trusted? Polskie Archiwum Medycyny Wewnetrznej. 2009;119(3):148–56.

PubMed   Google Scholar  

Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Gross L, Lhomme E, Pasin C, Richert L, Thiebaut R. Ebola vaccine development: systematic review of pre-clinical and clinical studies, and meta-analysis of determinants of antibody response variability after vaccination. Int J Infect Dis. 2018;74:83–96.

Article   CAS   Google Scholar  

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, ... Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Giang HTN, Banno K, Minh LHN, Trinh LT, Loc LT, Eltobgy A, et al. Dengue hemophagocytic syndrome: a systematic review and meta-analysis on epidemiology, clinical signs, outcomes, and risk factors. Rev Med Virol. 2018;28(6):e2005.

Morra ME, Altibi AMA, Iqtadar S, Minh LHN, Elawady SS, Hallab A, et al. Definitions for warning signs and signs of severe dengue according to the WHO 2009 classification: systematic review of literature. Rev Med Virol. 2018;28(4):e1979.

Morra ME, Van Thanh L, Kamel MG, Ghazy AA, Altibi AMA, Dat LM, et al. Clinical outcomes of current medical approaches for Middle East respiratory syndrome: a systematic review and meta-analysis. Rev Med Virol. 2018;28(3):e1977.

Vassar M, Atakpo P, Kash MJ. Manual search approaches used by systematic reviewers in dermatology. Journal of the Medical Library Association: JMLA. 2016;104(4):302.

Naunheim MR, Remenschneider AK, Scangas GA, Bunting GW, Deschler DG. The effect of initial tracheoesophageal voice prosthesis size on postoperative complications and voice outcomes. Ann Otol Rhinol Laryngol. 2016;125(6):478–84.

Rohatgi AJaiWa. Web Plot Digitizer. ht tp. 2014;2.

Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol. 2005;5(1):13.

Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.

Van Rijkom HM, Truin GJ, Van’t Hof MA. A meta-analysis of clinical studies on the caries-inhibiting effect of fluoride gel treatment. Carries Res. 1998;32(2):83–92.

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

Tawfik GM, Tieu TM, Ghozy S, Makram OM, Samuel P, Abdelaal A, et al. Speech efficacy, safety and factors affecting lifetime of voice prostheses in patients with laryngeal cancer: a systematic review and network meta-analysis of randomized controlled trials. J Clin Oncol. 2018;36(15_suppl):e18031-e.

Wannemuehler TJ, Lobo BC, Johnson JD, Deig CR, Ting JY, Gregory RL. Vibratory stimulus reduces in vitro biofilm formation on tracheoesophageal voice prostheses. Laryngoscope. 2016;126(12):2752–7.

Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355.

RevMan The Cochrane Collaboration %J Copenhagen TNCCTCC. Review Manager (RevMan). 5.0. 2008.

Schwarzer GJRn. meta: An R package for meta-analysis. 2007;7(3):40-45.

Google Scholar  

Simms LLH. Meta-analysis versus mega-analysis: is there a difference? Oral budesonide for the maintenance of remission in Crohn’s disease: Faculty of Graduate Studies, University of Western Ontario; 1998.

Download references

Acknowledgements

This study was conducted (in part) at the Joint Usage/Research Center on Tropical Disease, Institute of Tropical Medicine, Nagasaki University, Japan.

Author information

Authors and affiliations.

Faculty of Medicine, Ain Shams University, Cairo, Egypt

Gehad Mohamed Tawfik

Online research Club http://www.onlineresearchclub.org/

Gehad Mohamed Tawfik, Kadek Agus Surya Dila, Muawia Yousif Fadlelmola Mohamed, Dao Ngoc Hien Tam, Nguyen Dang Kien & Ali Mahmoud Ahmed

Pratama Giri Emas Hospital, Singaraja-Amlapura street, Giri Emas village, Sawan subdistrict, Singaraja City, Buleleng, Bali, 81171, Indonesia

Kadek Agus Surya Dila

Faculty of Medicine, University of Khartoum, Khartoum, Sudan

Muawia Yousif Fadlelmola Mohamed

Nanogen Pharmaceutical Biotechnology Joint Stock Company, Ho Chi Minh City, Vietnam

Dao Ngoc Hien Tam

Department of Obstetrics and Gynecology, Thai Binh University of Medicine and Pharmacy, Thai Binh, Vietnam

Nguyen Dang Kien

Faculty of Medicine, Al-Azhar University, Cairo, Egypt

Ali Mahmoud Ahmed

Evidence Based Medicine Research Group & Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam

Nguyen Tien Huy

Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam

Department of Clinical Product Development, Institute of Tropical Medicine (NEKKEN), Leading Graduate School Program, and Graduate School of Biomedical Sciences, Nagasaki University, 1-12-4 Sakamoto, Nagasaki, 852-8523, Japan

You can also search for this author in PubMed   Google Scholar

Contributions

NTH and GMT were responsible for the idea and its design. The figure was done by GMT. All authors contributed to the manuscript writing and approval of the final version.

Corresponding author

Correspondence to Nguyen Tien Huy .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:.

Figure S1. Risk of bias assessment graph of included randomized controlled trials. (TIF 20 kb)

Additional file 2:

Figure S2. Risk of bias assessment summary. (TIF 69 kb)

Additional file 3:

Figure S3. Arthralgia results of random effect meta-analysis using R meta package. (TIF 20 kb)

Additional file 4:

Figure S4. Arthralgia linear regression test of funnel plot asymmetry using R meta package. (TIF 13 kb)

Additional file 5:

Table S1. PRISMA 2009 Checklist. Table S2. Manipulation guides for online database searches. Table S3. Detailed search strategy for twelve database searches. Table S4. Baseline characteristics of the patients in the included studies. File S1. PROSPERO protocol template file. File S2. Extraction equations that can be used prior to analysis to get missed variables. File S3. R codes and its guidance for meta-analysis done for comparison between EBOLA vaccine A and placebo. (DOCX 49 kb)

Additional file 6:

Data S1. Extraction and quality assessment data sheets for EBOLA case example. (XLSX 1368 kb)

Additional file 7:

Data S2. Imaginary data for EBOLA case example. (XLSX 10 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Tawfik, G.M., Dila, K.A.S., Mohamed, M.Y.F. et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health 47 , 46 (2019). https://doi.org/10.1186/s41182-019-0165-6

Download citation

Received : 30 January 2019

Accepted : 24 May 2019

Published : 01 August 2019

DOI : https://doi.org/10.1186/s41182-019-0165-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Tropical Medicine and Health

ISSN: 1349-4147

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

what is systematic review research design

Banner

Systematic Reviews

Who is this guide for and what can be found in it, what are systematic reviews, how do systematic reviews differ from narrative literature reviews.

  • Types of Systematic Reviews
  • Reading Systematic Reviews
  • Resources for Conducting Systematic Reviews
  • Getting Help with Systematic Reviews from the Library
  • History of Systematic Reviews
  • Acknowledgements

Contact Us!

Ask a question

This guide aims to support all OHSU members' systematic review education and activities, orienting OHSU members who are new to systematic reviews and facilitating the quality, rigor, and reproducibility of systematic reviews produced by OHSU members.

In it you will find:

  • A definition of what systematic reviews are, how they compare to other evidence, and how they differ from narrative literature reviews
  • Descriptions of the different types of systematic reviews , with links to resources on methods, protocols, reporting, additional information, and selecting the right type of systematic review for your research question
  • Guidance on how to read and evaluate systematic reviews for strength, quality, and potential for bias
  • A high-level overview of how systematic reviews are conducted , including team size and roles, standards, and processes
  • Links to resources and tools for conducting systematic reviews
  • Information about how to get assistance with conducting a systematic review from the OHSU Library
  • A history of systematic reviews to provide contextual understanding of how they have developed over time
"A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors."

Gopalakrishnan S, Ganeshkumar P. Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare . J Family Med Prim Care . 2013;2(1):9-14. doi:10.4103/2249-4863.109934

Systematic Reviews are a vital resource used in the pursuit of Evidence-Based Practice (EBP):

  • These studies can be found near the top of the Evidence Pyramid , which ranks sources of information and study designs by the level of evidence contained within them
  • This ranking is based on the level of scientific rigor employed in their methods and the quality and reliability of the evidence contained within these sources
  • A higher ranking means that we can be more confident that their conclusions are accurate and have taken measures to limit bias

Research design and evidence , by CFCF , CC BY-SA 4.0 , via Wikimedia Commons

Things to know about systematic reviews:

  • Systematic reviews are a type of research study
  • Systematic reviews aim to provide a comprehensive and unbiased summary of the existing evidence on a particular research question
  • There are many types of systematic reviews , each designed to address a specific type of research purpose and with their own strengths and weaknesses
  • The choice of what type of review to produce typically will depend on the nature of the research question and the resources that are available on the topic

The practice of producing systematic reviews is sometimes referred to by other names such as:

  • Evidence Synthesis
  • Knowledge Synthesis
  • Research Synthesis

This guide tries to stick with the term "Systematic Reviews" unless a specific type of systematic review is being discussed.

While all reviews combat information overload in the health sciences by summarizing the literature on a topic, different types of reviews have different approaches. The term systematic review is often conflated with narrative literature reviews , which can lead to confusion and misunderstandings when seeking help with conducting them. This table helps clarify the differences.

  • Next: Types of Systematic Reviews >>
  • Last Updated: Feb 12, 2024 5:59 PM
  • URL: https://libguides.ohsu.edu/systematic-reviews

University of Maryland Libraries Logo

Systematic Review

  • Library Help
  • What is a Systematic Review (SR)?

Steps of a Systematic Review

  • Framing a Research Question
  • Developing a Search Strategy
  • Searching the Literature
  • Managing the Process
  • Meta-analysis
  • Publishing your Systematic Review

Forms and templates

Logos of MS Word and MS Excel

Image: David Parmenter's Shop

  • PICO Template
  • Inclusion/Exclusion Criteria
  • Database Search Log
  • Review Matrix
  • Cochrane Tool for Assessing Risk of Bias in Included Studies

   • PRISMA Flow Diagram  - Record the numbers of retrieved references and included/excluded studies. You can use the Create Flow Diagram tool to automate the process.

   •  PRISMA Checklist - Checklist of items to include when reporting a systematic review or meta-analysis

PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram

  • PROSPERO Template
  • Manuscript Template
  • Steps of SR (text)
  • Steps of SR (visual)
  • Steps of SR (PIECES)

Adapted from  A Guide to Conducting Systematic Reviews: Steps in a Systematic Review by Cornell University Library

Source: Cochrane Consumers and Communications  (infographics are free to use and licensed under Creative Commons )

Check the following visual resources titled " What Are Systematic Reviews?"

  • Video  with closed captions available
  • Animated Storyboard
  • << Previous: What is a Systematic Review (SR)?
  • Next: Framing a Research Question >>
  • Last Updated: Jan 26, 2024 4:35 PM
  • URL: https://lib.guides.umd.edu/SR

Study designs: Part 7 - Systematic reviews

Affiliations.

  • 1 Department of Anaesthesiology, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India.
  • 2 Director, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India.
  • PMID: 32670836
  • PMCID: PMC7342340
  • DOI: 10.4103/picr.PICR_84_20

In this series on research study designs, we have so far looked at different types of primary research designs which attempt to answer a specific question. In this segment, we discuss systematic review, which is a study design used to summarize the results of several primary research studies. Systematic reviews often also use meta-analysis, which is a statistical tool to mathematically collate the results of various research studies to obtain a pooled estimate of treatment effect; this will be discussed in the next article.

Keywords: Research design; review [publication type]; systematic review [publication type].

Copyright: © 2020 Perspectives in Clinical Research.

  • Research Consultation
  • How-to Guides
  • People Search
  • Library Databases
  • Research Guides
  • Find Periodicals
  • Google Scholar
  • Library Map
  • Texts for Vets
  • Interlibrary Loan - ILL
  • Research & Instruction
  • Computers and Laptops
  • Services For...
  • Reserve a Room
  • APSU Records Management
  • Library Collections
  • Digital Collections
  • Special Collections
  • Veterans' Oral History
  • Government Resources
  • Library Information
  • Woodward Library Society
  • New @ Woodward Library
  • Celebration of Scholarship
  • Join Woodward Library Society

Systematic Reviews: What is a systematic review?

  • What is a systematic review?
  • How do you perform a systematic review?
  • Tools & Resources

What is a Systematic Review?

In the health-related professions, systematic reviews are considered the most reliable resources.  They are studies that make up the top level of the evidence-based information pyramid, and as a result, they are the most sought after information for questions about health. Systematic reviews are not quite the same as literature reviews. Literature reviews, also known as narrative reviews, attempt to find all published materials on a subject, whereas systematic reviews try to find everything that focuses on answering a specific question.  Since systematic reviews are generally associated with health related fields, their main objective is to ensure the results of the review provide accurate evidence that answers relevant questions.  If you are looking for information about literature reviews, please check the library's guide on the topic  here .

Why use systematic reviews to answer questions?

When looking for answers to health questions, systematic reviews are considered the best resources to use for evidence-based information.  The predefined protocols, the amount of information reviewed, the evaluation process involved, and the efforts to eliminate bias are all a part of what makes health professionals consider systematic reviews to be the highest level of evidence based information available.  As a part of the process, systematic reviews tend to look at and evaluate all the randomized controlled trials, or all the cohort studies, for their specific topic.  By looking at and evaluating a vast amount of comparable studies, a systematic review is able to provide answers that have a much stronger level of evidence than any individual study.

Why perform a systematic review?

These reviews collect large amounts of information that fit within the predetermined parameters, so performing a systematic review is an excellent way to develop expertise on a topic.  Setting up the criteria, searching for the information, and evaluating the information found, gives the reviewer an extremely strong understanding of the process needed to create a review as well as how to evaluate its various elements.  Creating a systematic review gives the reviewer an opportunity to further the discussion on a topic.  In the health fields, performing and then publishing these reviews provides more evidence on topics that can be used for making decisions in a clinical environment.

How to find a systematic review

There are a number of databases that focus on health related resources, and most of them search through journals that include systematic reviews.  In these cases, you can include the words “systematic review” and the results will include entries that have the words “systematic review” in them somewhere.  Many of these results will be systematic reviews; however, some of the results may include these words, but are not systematic reviews.  A few databases that are used by researchers have added in limitation features that make it easier to find systematic reviews, and ensure a specific article/document/publication type are found.  Here are three examples of databases and how to limit their search results to systematic reviews:

  • Scroll down the main page and in the middle column, click on Clinical Queries.  In the provided search box type in your search terms.  The middle row of results is systematic reviews.  You can type your search terms in on the main page, and limit the results after they are listed.  To do this, find the section for “article types” on the left side of the page and click “customize”.  On the list that pops up, clear all the check boxes except the one for “systematic reviews”, then click “Show”.  When the main page is displayed again, click on “systematic reviews” to reset the search using that limit.
  • In CINAHL (EBSCO version) it is easy to limit search results to systematic reviews from the initial search page.  Scroll down to find “Publication types” on the right side of the page.  In the Publication Types box, search through the list until you find systematic reviews and highlight it.  After this has been done, add in any other limits needed and then search as usual.
  • This is the easiest of the three to use for finding systematic reviews.  On the main search page, type in the terms, run the search and wait for the results page.  On the results page, the right side has a limit that says “systematic review”.  Click on it and the results will all be systematic reviews.

Types of systematic reviews

These are a few of the various types of systematic reviews.

Critical review

Critical reviews are often thought to be the same as a literature review.  They involve a comprehensive review of a topic that involves a high level of analysis that seeks to identify the most important aspects of a subject.

The overview is a look at the literature available on the topic.  They may or may not have an analytical component that provides a synthesis of the information found, but this depends on how systematic the review has been.

Qualitative systematic review

This type of study looks at as many qualitative studies and tries to find themes among these studies that lie across the range of reviews. The information is then organized into a narrative that is frequently accompanied by a conceptual model.

Rapid review

The number of studies included in a rapid review are dictated by time constraints. Most often these reviews are conducted to determine what is known about policies or practices, and then determine how much these are based on evidence.

Scoping review

These reviews are intended to quickly assess the availability of research on a specific topic, and then determine whether the evidence provided in these research articles is sufficient and appropriate, 

Umbrella review

Umbrella reviews research broad topic, and look at a wide range of reviews that have been done for a number of potential interventions. It then triest to establish the pros and cons of each treatment and what the potential results would be.

Subject Guide

Profile Photo

  • Next: How do you perform a systematic review? >>
  • Last Updated: Jun 30, 2023 2:03 PM
  • URL: https://libguides.apsu.edu/systematic-reviews

Home

  • Duke NetID Login
  • 919.660.1100
  • Duke Health Badge: 24-hour access
  • Accounts & Access
  • Databases, Journals & Books
  • Request & Reserve
  • Training & Consulting
  • Request Articles & Books
  • Renew Online
  • Reserve Spaces
  • Reserve a Locker
  • Study & Meeting Rooms
  • Course Reserves
  • Digital Health Device Collection
  • Pay Fines/Fees
  • Recommend a Purchase
  • Access From Off Campus
  • Building Access
  • Computers & Equipment
  • Wifi Access
  • My Accounts
  • Mobile Apps
  • Known Access Issues
  • Report an Access Issue
  • All Databases
  • Article Databases
  • Basic Sciences
  • Clinical Sciences
  • Dissertations & Theses
  • Drugs, Chemicals & Toxicology
  • Grants & Funding
  • Interprofessional Education
  • Non-Medical Databases
  • Search for E-Journals
  • Search for Print & E-Journals
  • Search for E-Books
  • Search for Print & E-Books
  • E-Book Collections
  • Biostatistics
  • Global Health
  • MBS Program
  • Medical Students
  • MMCi Program
  • Occupational Therapy
  • Path Asst Program
  • Physical Therapy
  • Researchers
  • Community Partners

Conducting Research

  • Archival & Historical Research
  • Black History at Duke Health
  • Data Analytics & Viz Software
  • Data: Find and Share
  • Evidence-Based Practice
  • NIH Public Access Policy Compliance
  • Publication Metrics
  • Qualitative Research
  • Searching Animal Alternatives

Systematic Reviews

  • Test Instruments

Using Databases

  • JCR Impact Factors
  • Web of Science

Finding & Accessing

  • COVID-19: Core Clinical Resources
  • Health Literacy
  • Health Statistics & Data
  • Library Orientation

Writing & Citing

  • Creating Links
  • Getting Published
  • Reference Mgmt
  • Scientific Writing

Meet a Librarian

  • Request a Consultation
  • Find Your Liaisons
  • Register for a Class
  • Request a Class
  • Self-Paced Learning

Search Services

  • Literature Search
  • Systematic Review
  • Animal Alternatives (IACUC)
  • Research Impact

Citation Mgmt

  • Other Software

Scholarly Communications

  • About Scholarly Communications
  • Publish Your Work
  • Measure Your Research Impact
  • Engage in Open Science
  • Libraries and Publishers
  • Directions & Maps
  • Floor Plans

Library Updates

  • Annual Snapshot
  • Conference Presentations
  • Contact Information
  • Gifts & Donations
  • What is a Systematic Review?

Types of Reviews

  • Manuals and Reporting Guidelines
  • Our Service
  • 1. Assemble Your Team
  • 2. Develop a Research Question
  • 3. Write and Register a Protocol
  • 4. Search the Evidence
  • 5. Screen Results
  • 6. Assess for Quality and Bias
  • 7. Extract the Data
  • 8. Write the Review
  • Additional Resources
  • Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

  • Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

  • << Previous: What is a Systematic Review?
  • Next: Manuals and Reporting Guidelines >>
  • Last Updated: Feb 8, 2024 5:16 PM
  • URL: https://guides.mclibrary.duke.edu/sysreview
  • Duke Health
  • Duke University
  • Duke Libraries
  • Medical Center Archives
  • Duke Directory
  • Seeley G. Mudd Building
  • 10 Searle Drive
  • [email protected]

Dahlgren Memorial Library

The Graduate Health & Life Sciences Research Library at Georgetown University Medical Center

Systematic reviews.

  • Should I do a systematic review?
  • Writing the Protocol
  • Building a Systematic Search
  • Where to Search
  • Managing Project Data
  • How can a DML librarian help?

Guides and Standards

  • The Cochrane Handbook The Cochrane Handbook has become the de facto standard for planning and carrying out a systematic review. Chapter 6, Searching for Studies, is most helpful in planning your review.
  • Finding What Works in Health Care: Standards for Systematic Reviews The IOM standards promote objective, transparent, and scientifically valid systematic reviews. They address the entire systematic review process, from locating, screening, and selecting studies for the review, to synthesizing the findings (including meta-analysis) and assessing the overall quality of the body of evidence, to producing the final review report.
  • PRISMA Standards The Preferred Reporting Items for Systematic Reviews and Meta-Analyses is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. A 27-item checklist, PRISMA focuses on randomized trials but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

What is a systematic review?

A systematic literature review is a research methodology designed to answer a focused research question. Authors conduct a methodical and comprehensive literature synthesis focused on a well-formulated research question. Its aim is to identify and synthesize all of the scholarly research on a particular topic, including both published and unpublished studies. Systematic reviews are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making and identify gaps in research.  Every step of the review, including the search, must be documented for reproducibility. 

Researchers in medicine may be most familiar with Cochrane Reviews, which synthesize randomized controlled trials to evaluate specific medical interventions. Systematic reviews are conducted in many other fields, though the type of evidence analyzed varies with the research question. 

When to use systematic review methodology

Systematic reviews require more time and manpower than traditional literature reviews. Before beginning a systematic review, researchers should address these questions:

Is there is enough literature published on the topic to warrant a review? 

Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?

Can your research question be answered by a systematic review?

Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria, and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.

Do you have a protocol outlining the review plan?

The protocol is the roadmap for the review project. A good protocol outlines study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.

Do you have a team of experts?

A systematic review is team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams are often composed of subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data. 

Do you have the time that it takes to properly conduct a systematic review?  

Systematic reviews typically take 12-18 months. 

Do you have a method for discerning bias?  

There are many types of bias, including selection, performance, & reporting bias, and assessing the risk of bias of individual studies is an important part of your study design.

Can you afford to have articles in languages other than English translated?  

You should include all relevant studies in your systematic review, regardless of the language they were published in, so as to avoid language bias. 

Which review is right for you?

If your project does not meet the above criteria, there are many more options for conducting a synthesis of the literature. The chart below highlights several review methodologies. Reproduced from: Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x  . Review. PubMed PMID: 19490148 

  • Next: Writing the Protocol >>
  • Last Updated: Aug 7, 2023 3:36 PM
  • URL: https://guides.dml.georgetown.edu/systematicreviews

The Responsible Use of Electronic Resources policy governs the use of resources provided on these guides. © Dahlgren Memorial Library, Georgetown University Medical Center. Unless otherwise stated, these guides may be used for educational or academic purposes as long as proper attribution is given. Please seek permission for any modifications, adaptations, or for commercial purposes. Email [email protected] to request permission. Proper attribution includes: Written by or adapted from, Dahlgren Memorial Library, URL.

SMU Libraries logo

  •   SMU Libraries
  • Scholarship & Research
  • Teaching & Learning
  • Bridwell Library
  • Business Library
  • DeGolyer Library
  • Fondren Library
  • Hamon Arts Library
  • Underwood Law Library
  • Fort Burgwin Library
  • Exhibits & Digital Collections
  • SMU Scholar
  • Special Collections & Archives
  • Connect With Us
  • Research Guides by Subject
  • How Do I . . . ? Guides
  • Find Your Librarian
  • Writing Support

Evidence Syntheses and Systematic Reviews: Overview

  • Choosing a Review

Analyze and Report

What is evidence synthesis.

Evidence Synthesis: general term used to refer to any method of identifying, selecting, and combining results from multiple studies. There are several types of reviews which fall under this term; the main ones are in the table below: 

Types of Reviews

General steps for conducting systematic reviews.

The number of steps for conducting Evidence Synthesis varies a little, depending on the source that one consults. However, the following steps are generally accepted in how Systematic Reviews are done:

  • Identify a gap in the literature and form a well-developed and answerable research question which will form the basis of your search
  • Select a framework that will help guide the type of study you’re undertaking
  • Different guidelines are used for documenting and reporting the protocols of your systematic review before the review is conducted. The protocol is created following whatever guideline you select.
  • Select Databases and Grey Literature Sources
  • For steps 3 and 4, it is advisable to consult a librarian before embarking on this phase of the review process. They can recommend databases and other sources to use and even help design complex searches.
  • A protocol is a detailed plan for the project, and after it is written, it should be registered with an appropriate registry.
  • Search Databases and Other Sources
  • Not all databases use the same search syntax, so when searching multiple databases, use search syntaxes that would work in individual databases.
  • Use a citation management tool to help store and organize your citations during the review process; great help when de-duplicating your citation results
  • Inclusion and exclusion criteria already developed help you remove articles that are not relevant to your topic. 
  • Assess the quality of your findings to eliminate bias in either the design of the study or in the results/conclusions (generally not done outside of Systematic Reviews).

Extract and Synthesize

  • Extract the data from what's left of the studies that have been analyzed
  • Extraction tools are used to get data from individual studies that will be analyzed or summarized. 
  • Synthesize the main findings of your research

Report Findings

Report the results using a statistical approach or in a narrative form.

Need More Help?

Librarians can:

  • Provide guidance on which methodology best suits your goals
  • Recommend databases and other information sources for searching
  • Design and implement comprehensive and reproducible database-specific search strategies 
  • Recommend software for article screening
  • Assist with the use of citation management
  • Offer best practices on documentation of searches

Related Guides

  • Literature Reviews
  • Choose a Citation Manager
  • Project Management

Steps of a Systematic Review - Video

  • Next: Choosing a Review >>
  • Last Updated: Feb 16, 2024 5:40 PM
  • URL: https://guides.smu.edu/evidencesyntheses
  • Open access
  • Published: 16 February 2024

Barriers and facilitators to health technology adoption by older adults with chronic diseases: an integrative systematic review

  • Alessia Bertolazzi 1 ,
  • Valeria Quaglia 1 &
  • Ramona Bongelli 1  

BMC Public Health volume  24 , Article number:  506 ( 2024 ) Cite this article

139 Accesses

Metrics details

In recent years, healthcare systems have progressively adopted several technologies enhancing access to healthcare for older adults and support the delivery of efficient and effective care for this specific population. These technologies include both assistive technologies designed to maintain or improve the independence, social participation and functionality of older people at home, as well as health information technology developed to manage long-term conditions. Examples of such technologies include telehealth, wearable devices and mobile health. However, despite the great promise that health technology holds for promoting independent living among older people, its actual implementation remains challenging.

This study aimed to conduct an integrative systematic review of the research evidence on the factors that facilitate or hinder the adoption of different types of technology by older individuals with chronic diseases. For this purpose, four electronic databases (PsycArticles, Scopus, Web of Science and PubMed) were queried to search for indexed published studies. The methodological quality of the selected papers has been assessed using the Mixed Methods Appraisal Tool (MMAT).

Twenty-nine articles were selected, including 6.213 adults aged 60 or older. The studies have been synthesised considering the types of technological interventions and chronic diseases, as well as the main barriers and facilitators in technology acceptance. The results revealed that the majority of the selected articles focused on comorbid conditions and the utilisation of telemedicine tools. With regard to hindering and facilitating factors, five main domains were identified: demographic and socioeconomic, health-related, dispositional, technology-related and social factors.

The study results have practical implications not only for technology developers but also for all the social actors involved in the design and implementation of healthcare technologies, including formal and informal caregivers and policy stakeholders. These actors could use this work to enhance their understanding of the utilisation of technology by the ageing population. This review emphasises the factors that facilitate technology adoption and identifies barriers that impede it, with the ultimate goal of promoting health and independent living.

Peer Review reports

Over the last few decades, the elderly population has grown significantly, and it is projected that the proportion of people aged 65 and above will continue to increase from 10% in 2022 to 16% in 2050 [ 1 ]. This demographic shift has led to increased pressure on healthcare systems’ ability to plan and provide effective healthcare services for older adults. In fact, the ageing of the population has resulted in an increase in long-term diseases such as diabetes, chronic respiratory diseases (e.g., chronic obstructive pulmonary disease and asthma), neurological disorders (e.g., dementia, Alzheimer’s disease and Parkinson’s disease) and cardiovascular disease (e.g., ischaemic heart disease, cerebrovascular disease and hypertensive heart disease) [ 2 ]. Along Europe, 72.5% of people aged 85 years or older reported the presence of at least one health problem [ 3 ]. In the U.S., 23.9% of the population aged 65 or older has one chronic condition, while 63.7% has two or more [ 4 ]. The growing prevalence of multimorbidity is associated with increased utilisation and cost of healthcare services [ 5 ]. The growth of the elderly population with multiple long-term diseases has implications not only at the societal level but also at the individual level. Older people have specific health needs that need to be met in a timely manner, as complications and limitations related to illness can impact their independence, autonomy and overall well-being [ 6 ].

However, this social group faces specific difficulties in accessing healthcare services. Several studies have investigated the factors influencing access to healthcare. These studies suggest that sociodemographic determinants (e.g., female gender, older age, etc.) [ 7 ], age-related factors (e.g., limited mobility, sensory impairments, and disability) [ 8 ], socioeconomic variables (such as lower income, lack of complementary insurance, cost, and transportation) [ 7 , 9 ], as well as organisational features of healthcare systems (e.g., extended waiting periods for medical examinations) [ 10 ], play significant roles in influencing access to healthcare.

To overcome these barriers, healthcare systems have progressively implemented various types of digital health technologies aimed at enhancing elderly care. The literature presents various terminologies in this regard. The World Health Organization defined digital health as ‘the field of knowledge and practice associated with the development and use of digital technologies to improve health’ [ 11 , 12 ]. Particularly, digital technology refers to both the software, which includes computer coding programmes that provide instructions for computer operations, and the hardware, which consists of physical computer devices. These components work together using digital coding, also known as binary coding. Additionally, digital technology encompasses the infrastructure that supports these software and hardware components [ 13 ].

Furthermore, specific terms have been coined to describe digital health technologies for older adults. For instance, the umbrella term ‘gerontechnology’ has emerged to define the set of technologies intended to promote the independence of older adults, facilitate ageing in place and accommodate age-related declines and impairments [ 14 ]. These technologies include both assistive technologies designed to maintain or improve the independence, social participation and functionality of older people at home, as well as health information technology for managing long-term conditions. Examples of these technologies include telehealth, wearable devices, and mobile health [ 15 , 16 ]. In the Nordic European welfare state systems, the term ‘welfare technology’ has been coined to emphasise the public and universalistic nature that these devices should have. This term refers to a range of digital tools with integrated platforms that are adopted by public care services to promote welfare among individuals [ 17 ].

In this paper, the expression ‘health technologies’ will be employed to encompass a range of digital technologies designed for the management of health conditions. These include the electronic health record, mobile health apps, wearable devices, telehealth and telemedicine [ 11 , 12 ].

Health technologies can be applied in multiple areas, such as self-management of chronic diseases and the sharing and transfer of clinical data. These applications can improve adherence to therapeutic regimens, facilitate communication with healthcare professionals and enable timely interventions. Previous research on telemedicine has shown that it can reduce travel time and costs, making it an important resource for older patients living in underserved areas. It can also diminish patients’ waiting times for medical encounters, thus shortening the time for diagnosis. Lastly, especially during pandemic times, it can also reduce the risk of contagion and infections [ 18 , 19 ]. Moreover, health technologies can promote aging in place, enabling older people to safely remain in their homes (or in appropriate housing, depending on their health conditions), thereby reducing hospitalization and avoiding institutionalization. A recent systematic review revealed that smart home technologies used for the management of chronic diseases in older adults can improve several health outcomes [ 20 ]. The monitoring of daily living activities, such as mobility, posture, falls or sleeping disorders can facilitate personalized and timely interventions. Furthermore, it promotes physical activity, enhances the quality of life and fosters a sense of security and well-being in older people. Other instruments, such as external memory aids and telemedicine, can support chronic patients in managing medication, or enhance the control of vital signs [ 20 ].

However, despite the great promise of health technology in enabling older people to maintain their independence for longer, its actual implementation remains challenging. The literature highlights several factors that may have a negative impact on the adoption of health technologies in the elderly population.

One line of research has focused on socio-structural characteristics that may increase inequalities in technology use. Previous studies have found that older individuals with lower income and education levels have limited access to broadband, lower health literacy and lower digital competencies, which in turn leads to limited technology adoption [ 21 , 22 , 23 , 24 ].

The second research line has concentrated on individual factors that may influence the acceptance of technology, such as age or health conditions. Cognitive deficits, as well as physical impairments (e.g., vision and hearing loss and mobility limitations), can pose significant challenges in the use of technology [ 15 , 25 ]. In addition, other studies have examined the attitudes of the elderly towards health technologies. For instance, some of them have focused on the strategies employed by individuals to resist stereotypes associated with old age. The rejection of technology may indeed be associated with negative (self)perceptions related to the loss of dignity and autonomy, as well as the fear of being stigmatised as someone no longer able to take care of oneself [ 26 , 27 , 28 ]. Furthermore, perceptions of the usefulness and ease of use of technology, as well as beliefs about privacy, may influence its adoption. While privacy concerns have not been observed for some technologies, such as telemedicine [ 26 ], they could be a barrier for other devices, including fall detection or bed occupancy sensors [ 29 , 30 ].

The third line of study focused on aspects related to the technology itself, including the role of users in the technology design process and the socio-material characteristics of the technology, as well as users’ adaptation strategies. Regarding the first aspect, while participatory technology design methods are becoming more widespread, there are difficulties in implementing these methods in practice. Users are often still perceived as passive consumers, and their needs may not be fully addressed [ 31 , 32 ]. Concerning the second aspect, research has emphasised that in order to understand the strategies for the adoption or refusal of technologies, they should be considered within the context of the usual practices of the elderly population [ 33 ]. In fact, research has shown that older people tend to adapt technologies to suit their needs through ‘bricolage’ arrangements, using devices in ways that were not originally intended [ 33 , 34 , 35 ].

Across these strands of research, however, tension emerges between two different perspectives regarding the impact that health technologies would have on social actors. On one side, there is the self-surveillance effect of these technologies, while on the other side, there is the empowerment effect that would be embedded in health technologies [ 36 , 37 , 38 ]. The first perspective emphasises the disciplinary effects of health technologies, which could engender behavioural changes through continuous data generation and transmission [ 39 , 40 , 41 , 42 ]. This ultimately would promote both an expansion of the ‘medical gaze’ into the everyday lives of self-tracked patients [ 37 ] and an individualistic dimension of health, shifting the responsibility from healthcare systems to individuals [ 43 , 44 ].

The second point of view presents the individualisation of responsibility for one’s health condition in a positive manner and highlights the empowering potential of technologies. Health technologies empower patients by instilling in them a greater awareness of their health status. This awareness would thus lead to a greater sense of responsibility for one’s own health and trigger a virtuous cycle [ 45 – 46 ].

Therefore, although much research has been conducted, a variety of factors is at play in the acceptance of technology by older adults. Additionally, there are ambivalent points of view about the impact of health technologies on people. Upon examining recent reviews, it appears that studies are narrow in focus, as they only consider one set of factors at a time or a single technology, or they are not systematic and are based on a scoping review [ 47 ]. Particularly, the research appears to be limited to specific technologies, such as falls-prevention interventions [ 48 ], m-Health technology [ 49 ], telemedicine [ 18 , 50 ] and electronic personal health records [ 51 ]. Moreover, it solely examines hurdles and barriers without considering facilitating factors [ 51 ]. It also fails to indicate the specific medical condition or set of conditions that the technology is targeted at [ 47 , 52 , 53 ].

To shed light on this topic, the present integrative systematic review aims to identify barriers and facilitators that impact the adoption of different health technologies by older individuals with chronic diseases. Considering the multitude of diseases that could be included in the review and the wide range of technologies addressed in the health management of elderly individuals with chronic conditions, we have chosen to adopt a broad conceptualisation of ‘facilitators’ and ‘barriers’. With the first term, we refer to factors that support the adoption of technology and provide an incentive to continue using it. Additionally, we consider factors that have been identified in the literature as not hindering the utilisation of technology. Barriers, on the other hand, consist of all the elements that hinder the adoption of technology or discourage its use.

Thus, the research questions that guided the review were as follows:

What are the main factors hindering the adoption of technology by older adults with chronic diseases?

What are the main factors facilitating the adoption of technology by older adults with chronic diseases?

An integrative systematic review was conducted by implementing a search strategy that allowed for a comprehensive examination of the barriers and facilitators to the adoption of chronic disease-related technology in the elderly population. This method can have direct applicability for practical implementation and policymaking, and ‘allows for the inclusion of diverse methodologies (e.g., experimental and non-experimental research)’ [ 54 ]. Therefore, our analysis includes qualitative research, randomised and non-randomised quantitative studies and mixed method studies. It also comprises papers focused on different technologies and different types of chronic diseases.

Inclusion and exclusion criteria

In order to identify eligible articles, inclusion and exclusion criteria were established before starting the literature search. These criteria were based on the exploratory research questions in the review. The pre-defined inclusion criteria for study selection were as follows: (a) the sample must include participants aged 60 or older; (b) the sample must include participants affected by chronic disease; (c) the studies must focus on facilitators or barriers to the adoption of technologies related to chronic disease management; (d) the studies must be empirical and use qualitative, quantitative or mixed methods; (e) the studies must be published in English; (f) the studies must focus on technology targeting older people. The exclusion criteria adopted were as follows: (a) mixed sample population with participants above and below 60 years of age; (b) publications such as theoretical contributions, letters to the editor, systematic or scoping reviews, dissertations, conference proceedings, or those adopting non-standardised techniques and lacking sufficient analytical rigour (e.g., narrative reviews); (c) studies that evaluated a healthcare service instead of digital technology tools enabling the service (e.g., studies that evaluated the general telemedicine service without focusing on the platform enabling telemedicine services).

Search strategy

Four electronic databases were queried to search for published studies: PsycArticles, Scopus, Web of Science and PubMed. Records published between January 2012 and April 2022 were considered, operating with the following PICo framework [ 55 ] and using a Boolean search strategy through keywords and Medical Subject Headings (MeSH) terms (see Table  1 ). The full search strategy can be found in Additional file 1 .

The timeframe was chosen based on the findings from previous reviews [47; 53] and the evolution of the digital health tools under consideration. Publications reporting was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement (PRISMA) flow diagram [ 56 ].

figure 1

PRISMA flow diagram

As shown in Fig.  1 , a total of 9.906 publications have been retrieved from the databases. After removing the duplicates, 9.370 have been screened. All the authors reviewed the titles and abstracts of the records to identify relevant studies that met the inclusion/exclusion criteria. Disagreements were resolved through discussions until a consensus was reached.

Data extraction and analysis

As a result of the first screening, fifty-four publications were identified for possible inclusion. The methodological quality of the selected records has been assessed using the Mixed Methods Appraisal Tool (MMAT) [ 57 ]. All authors independently undertook the quality assessment process by performing blind MMAT evaluations of the same articles and comparing the results.

Regarding data extraction, the full texts of the included articles were obtained to extract pertinent details such as the authors’ names, year of publication, study objectives, study design, country of study, study setting (e.g., home or hospital), type of chronic disease and type of technology used. Each author independently identified facilitators and barriers through the content analysis of the included articles. One author independently (A.B.) grouped together homogeneous factors and redefined similar factors with different names. The final list of factors was discussed among authors until reaching a consensus. To synthesise and organise the results more effectively, these factors were further grouped into macro-categories, which are referred to as domains. The five identified domains emerged through a combination of inductive and deductive approaches. Some domains were deduced from previous systematic reviews [ 47 , 51 ], including technology-related factors and social factors. Some factors emerged inductively from the data itself, such as socioeconomic factors, health-related factors and dispositional factors.

During the quality assessment process, the authors agreed to exclude twenty-five articles for various reasons. These reasons included poor methodological quality, inadequate focus on barriers and facilitating factors (such as factors that were mere inferences of the authors) or articles that solely presented protocols for developing technology. Twenty-nine articles were selected, including 6.213 adults aged 60 or older. The characteristics of the screened studies are shown in Additional File 2 . As for the nomenclature of technological interventions, the original designations used within each article have been preserved to avoid inappropriate simplifications that could have resulted from their reclassification. Regarding the research design of the included studies, fifteen were based on research trials, presenting different lengths of the evaluation period: two weeks [ 58 ], five weeks [ 59 ], two months [ 60 , 61 , 62 ], three months [ 63 , 64 ], nine months [ 65 ], one year [ 66 , 67 ], and fifteen months [ 68 ]. However, one trial was conducted in a laboratory [ 69 ], so no evaluation period was planned. Additionally, three studies relied on data derived from prior trials, with subsequent secondary analyses performed [ 70 , 71 , 72 ]. The remaining articles had adopted different methodologies, encompassing qualitative approaches– such as interviews [ 73 , 74 , 75 , 76 ], focus groups [ 77 , 78 , 79 ], or both [ 80 ]–, quantitative ones through cross-sectional studies [ 81 , 82 , 83 , 84 ], or mixed methods [ 85 ].

The analysis of the included articles focused on (a) types of technological interventions, (b) types of chronic diseases and (c) the main barriers and facilitators in technology acceptance.

Concerning the types of technological interventions for managing chronic diseases, eight studies have examined telemedicine, which includes telecare, telemonitoring, telehealth, and telerehabilitation programmes [ 60 , 61 , 62 , 63 , 65 , 67 , 71 , 86 ]. Seven studies have focused on digital health platforms, such as web portals and video conferencing for home-based education [ 59 , 64 , 66 , 78 , 79 , 83 , 85 ]. Five studies have explored wearable technology, including the use of pedometers and self-tracking technology [ 58 , 72 , 73 , 75 , 77 ]; instead, m-health services have been examined by three studies [ 70 , 74 , 84 ]. Three studies have investigated information and communication technology (ICT) [ 80 , 81 , 82 ], that is services obtained through ICT (e.g., messaging services, using email to communicate with doctors, medication services and reminders, online tools, etc.); instead, one article focused on the Internet for health information seeking [ 71 ], as it specifically refers to the Internet exclusively for searching for health information online. Lastly, two articles have examined home assistive technologies (smart home) [ 68 , 69 ], two studies have focused on assistive robots [ 69 , 76 ], and one on active video games [ 75 ].

In terms of the specific chronic diseases targeted by technological interventions, 16 articles included aged people with multiple chronic conditions for technological intervention [ 59 , 60 , 61 , 63 , 65 , 66 , 67 , 69 , 73 , 78 , 79 , 80 , 82 , 83 , 85 , 86 ]. While, other articles focused on a specific chronic disease: COPD [ 71 , 72 , 75 ], cognitive impairments [ 68 , 70 , 76 ], heart failure [ 74 , 84 ], Parkinson’s disease [ 77 , 81 ], hypertension [ 58 ], vestibular dysfunction [ 64 ], and diabetes [ 62 ].

The identified barriers and facilitators are shown in Table  2 .

The identified factors were grouped into five domains, which have been used to organise the results: demographic and socioeconomic factors, health-related factors, dispositional factors, technology-related factors and social factors. Evidently, some of the factors placed in one of the five identified domains could simultaneously fall into another domain or present aspects of continuity with other domains. Therefore, the classification made is not rigid, but it is intended to make the results as intelligible as possible.

Demographic and socioeconomic factors

The impact of age on the adoption of technologies appears to be conflicting, considering demographic and socioeconomic patients’ characteristics. Several studies indicate that older age is a barrier to the utilisation of technology [ 81 , 82 , 85 ], mainly when it comes to using ICT tools for communication with healthcare providers or health education. However, other research did not find that increasing age is a limiting factor for technological interventions, including devices such as a pedometer [ 73 ], mobile applications for reporting health outcomes [ 84 ] and a telemedicine service [ 62 ].

In contrast, the impact of educational level on technology use appears to be more consistently supported by the literature [ 7 , 8 , 71 , 73 , 82 ]. Well-educated older adults seem to have an advantage in adopting various technologies [ 73 , 82 ], whereas poor education limits their use and acceptance [ 71 ]. An additional factor that has hindered the adoption of technologies by the elderly is related to the economic aspect of technology use. Several investigations consider the cost of technology as a barrier [ 58 , 69 , 85 ], as well as having a low income or not having an adequate socioeconomic status [ 74 ]. In the same vein, another study highlights the repercussions of low socioeconomic status, such as the absence of an Internet connection or insufficient space in the household for technological devices [ 58 ]. On the other hand, home-based technologies offer the advantage of being cost-effective, as they allow people to save time and money on transportation to medical examinations [ 59 , 65 , 74 , 79 ].

Health-related factors

Studies examining factors associated with the health status of the elderly converge, highlighting that age-related physical limitations can be a significant barrier to the adoption of technological interventions. More specifically, the analysis has revealed a high prevalence of sensory impairments, such as poor vision [ 71 ] or hearing loss [ 76 ], motor deficits [ 73 ] and cognitive disorders, including poor memory [ 58 , 71 ], limited learning skills [ 71 ] and general cognitive degeneration [ 76 ].

Furthermore, the lived experience of various health conditions plays a significant role in determining older adults’ acceptance of technology, both positively and negatively. On the one hand, technology seems to be more likely to be accepted if it has been adopted in the early stages of the disease [ 65 ]. On the other hand, research has underlined the detrimental effect of comorbidities or complex health conditions on the adoption of technology [ 60 , 66 , 86 ]. Different types of long-term diseases can also affect technology acceptance. For example, Rodríguez-Fernández et al. [ 86 ] found that a history of cancer, arthritis and hypertension was positively associated with the effective utilisation of telemedicine. On the contrary, depressive symptoms were negatively associated with it, as well as technical difficulties in using telemedicine were associated with a history of diabetes, heart disease and anxiety symptoms. States of severe anxiety about their illness appear to hinder the use of technology, as observed in the study conducted by Middlemass et al. [ 65 ]. Whereas a complex disease experience may lead to perceiving the activity of controlling one’s condition as ‘work’ [ 73 ] or an ‘extra burden’ [ 66 ], other studies have indicated that self-monitoring technologies appear to be capable of triggering a virtuous circle. In fact, older patients reported greater awareness and understanding of their condition [ 65 , 71 ], as well as an improvement in health and quality of life through the support they received [ 66 , 69 ]. Continuous monitoring enables both patients and healthcare providers to gain insights and learn about the medical condition, as well as to make timely adjustments (e.g., modifying diet or medications) [ 55 , 66 , 69 , 77 ].

Dispositional factors

Further aspects are investigated in the literature pertaining to dispositional factors, specifically the beliefs, attitudes and behaviours exhibited by the ageing population towards technologies. A conservative mindset, as well as a strong attachment to daily routines, have been identified as obstacles to the adoption of various technologies [ 71 , 76 , 79 ]. Similarly, a lack of motivation and interest in learning new ways to manage their conditions is an important barrier [ 60 , 64 , 65 , 71 , 76 , 79 ]. Resistance to the adoption of technology may also be the result of a general fear of using new services that could potentially alter their lives [ 80 ]. Additionally, specific fears, such as the fear of online fraud [ 71 ], the fear of sudden device malfunction [ 58 , 80 ] and the fear of making mistakes when using healthcare devices, can contribute to this resistance [ 75 ].

In addition, privacy seems to be a significant factor in hindering access to technology, especially when it comes to the use of the Internet for health information seeking [ 71 ] and patient web portals [ 78 , 83 ]. Since the ageing population tends to be unfamiliar with new technologies, training is a crucial factor in promoting and enhancing access to eHealth. The knowledge necessary to use technology can be gained through targeted training [ 58 ] or acquired through previous experience [ 74 , 80 ]. However, reluctance to learn how to operate technology, as well as perceived difficulties in the learning process, can make that process challenging.

Technical factors

As for technology-related factors, the analysed publications consistently emphasise the importance of the perceived ease of use of the various technologies discussed [ 59 , 74 , 82 , 80 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 61 ]. Conversely, the perception of difficulties in using devices or programmes may hinder the acceptance of such technologies [ 60 , 75 ]. The reviewed studies highlight several technical factors that patients report as problematic. These include connection problems [ 59 , 60 , 65 ], which are particularly evident for patients living in rural or isolated areas. Additionally, interface design issues, such as low-quality graphics [ 85 ] and unclear navigation buttons in portals [ 79 , 85 ], are also mentioned.

Therefore, a simple design [ 80 ] and clear presentation and organisation of information are features that facilitate the use of technology [ 85 ]. Besides, the technical factors that promote or hinder access to health technologies vary depending on their specificities and their different uses. For instance, in the case of wearable devices, it is important for older people that the technologies are non-invasive and that users perceive them as comfortable [ 77 ].

Similarly, in the case of assistive robots, the literature highlights the significance of the robot’s appearance as a relevant factor [ 76 ], particularly in terms of how the robot is perceived as trustworthy and likeable [ 69 ]. The robot’s speech interaction capabilities have also emerged as particularly important, especially the presence of a speech recognition system and the implementation of robot eye contact and validated gestures to accompany the speech [ 69 ].

Interestingly, a technical aspect that seems to encourage the use of both wearable devices and portals concerns the self-tracking functions. These functions include the ability of the tools to set measurable goals for physical activity, quantify users’ health status and activities, receive reminders, allow users to see long-term improvements and share data with healthcare providers [ 60 , 73 , 78 , 83 ].

Moreover, another important feature concerns the ability to communicate effectively with healthcare providers, specifically in the case of telehealth. Since telehealth is considered useful in managing medical conditions [ 58 ], facilitating factors include the device’s ability to function correctly, transmit accurate and reliable information and provide prompt intervention [ 58 ].

Social factors

Lastly, the analysis has revealed several social factors that are considered relevant based on the literature. First, an important facilitating factor is the perception that health technologies help enhance social ties with nurses and clinicians [ 63 , 66 ]. Moreover, in the case of telehealth, there is a perception of receiving social support and an improvement in communication between patients and healthcare providers [ 67 ].

Research has highlighted ambivalent perspectives on the role played by social networks in the lives of the elderly. On the one hand, having offspring or partners to rely on is considered a facilitating factor for using technology [ 65 , 71 ], especially when they actively encourage the elderly person to use these devices [ 65 ]. In addition, certain technologies seem to enhance connectedness with others; for example, active video games allow people to play with others [ 75 ] and videoconferencing platforms are used for home-based education [ 59 ]. The latter helped older adults who lived alone to meet new people, and being part of a group allowed them to share information and knowledge with others who had the same condition. Additionally, individuals who suffered from anxiety or depression found it less challenging to participate in online groups rather than to interact with people in person [ 59 ].

On the other hand, having more than one person in the house is instead seen as a barrier to technology use because it alters the data registered by sensors [ 76 ]. Furthermore, another hindering factor in the use of health technologies is the dependence on others for their use. This is evident in the cases of telehealth [ 65 ], ICT [ 80 ] or video games for physical activity that require other individuals to participate. The concern that technologies could replace in-person contact/visits by both clinicians and relatives constitutes a barrier, particularly in the case of ICT [ 80 ] and telehealth [ 65 ].

This study contributes to the understanding of the key factors that influence the acceptance or rejection of health technologies among elderly individuals with chronic diseases or conditions. This is achieved through an analysis of recent empirical literature. Selected studies examine a variety of chronic conditions (i.e. Parkinson’s disease, heart failure, COPD, cognitive impairment, etc.), investigating older adults’ acceptance or rejection of different technologies (i.e. wearable and mobile technologies, telemedicine, assistive robots, etc.). Our review suggests that the technology acceptance or refusal by aged people depends on a wide range of factors, grouped as follows: demographic-socioeconomic, health-related, dispositional, technology-related and social factors.

Regarding demographic and socioeconomic factors, a low educational level [ 71 ] and low income [ 58 , 69 , 85 ] have emerged as the main obstacles, which is consistent with previous findings [ 7 , 9 ]. Instead, the results concerning the impact of age appear more controversial, as some studies included in the review emphasize the negative effect of older age on the adoption of technologies [ 81 , 82 ], while others did not demonstrate statistically significant differences in technology adoption with advancing age [ 62 , 72 , 84 ]. This inconsistency can be explained by the fact that the samples considered in the examined studies include a target group– individuals over 60 years old– with heterogeneous characteristics. Differences may exist between age groups (for example, between those under 75 and those over 75), as well as within the same age group, given the variation in the aging process and the progression of chronic diseases from person to person.

Therefore, policymakers and technology developers should consider the needs of the most vulnerable and underprivileged social groups, particularly those with low household income and limited educational achievement [ 22 , 23 , 24 ]. To enhance access to health technologies, costs should be minimized or even eliminated, for example, by providing support to low-income individuals to access broadband [ 22 , 23 ]. Specific interventions should be aimed at improving technological competencies and digital health literacy among the elderly. For example, short e-learning courses have been found to be useful for enhancing their technological skills [ 87 ] and could be a suitable solution to bridge the digital divide.

Our review highlighted several aspects of older people’s health status, one of the individual-level factors that can negatively impact healthcare technology access. These factors include poor vision [ 71 ], hearing loss [ 76 ], motor deficits [ 73 ], cognitive issues such as poor memory [ 58 , 71 ] and limited learning skills [ 71 ], general cognitive degeneration [ 76 ] and the presence of comorbidities or complex health conditions [ 60 , 66 , 86 ]. However, considering the facilitating factors outlined in the review, it is possible to identify some strategies to mitigate these barriers. Firstly, healthcare professionals should recommend the early adoption of health technologies to ensure that older people have the opportunity to learn how to use them before the progression of the disease(s) [ 65 ]. Secondly, the developers should provide devices with a design that is as accessible as possible. For example, they could increase screen contrast and use an adequate font size to allow individuals with poor vision to read [ 63 , 65 , 79 , 83 , 84 ]. Additionally, robots and devices should incorporate sound alerts and a language that can be heard and understood by the elderly with hearing loss and cognitive impairment [ 58 , 59 , 69 , 76 , 85 ]. Findings also showed several dispositional factors that influence the acceptance or refusal to adopt technologies. For those individuals from older generations who are less familiar with technology, there is a higher likelihood of encountering resistance when it comes to using digital health tools. In fact, a conservative mindset, and a lack of interest in learning new methods of managing their conditions have been identified as significant barriers to the adoption of technologies among the elderly [ 65 , 71 , 79 ]. These findings are consistent with previous studies, which indicate that older adults are often hesitant to embrace new technologies [ 50 , 51 , 75 ]. This reluctance may stem from their familiarity with alternative methods of managing their diseases and their perceived lack of necessity for these devices [ 88 , 89 ]. Thus, healthcare providers should consider introducing digital health devices to the elderly through personalised and easy-to-understand training [ 65 , 66 , 79 , 83 ]. They should also reassure prospective users about privacy issues and provide constant and timely support in case of doubts or device malfunctions [ 58 , 65 , 66 , 78 ].

Technical factors emerged as crucial in either promoting or hindering older people’s access to technology. Literature has shown that, in order to be accepted and used over time, devices should be non-invasive and perceived as comfortable by users [ 77 ]. They also have functions to measure and quantify body functions and health status, set measurable goals for physical activity, send reminders to users, allow users to track long-term improvements and share data with physicians [ 75 ]. As mentioned earlier, technologies should have a simple design [ 80 ], and the organisation of the information should be as clear as possible [ 85 ]. Technologies for elderly healthcare have to accommodate the needs of individuals with different dis/abilities and physical/cognitive limitations. Furthermore, developers should consider the possibility of involving end users in the design and development process of digital health devices. Because of egocentric bias, younger designers might indeed face challenges in envisioning the product’s usage from the standpoint of an elderly adult [ 90 ]. Patients involved in the technology development process are more satisfied and inclined to adopt the technology [ 58 ]. In particular, if wearable devices are designed in collaboration with patients, it is easier to avoid issues regarding comfort, size, and ease of fitting, which often pose a barrier to adoption [ 77 ]. The involvement of patients in the early stages of technology development can enable the design of more user-centred technologies, and contribute to the early identification of potential issues, thus avoiding the addition of features that patients do not need [ 31 , 58 , 66 ].

The current integrative systematic review has highlighted a domain that is often overlooked but could actually play a crucial role in facilitating technology adoption– the social factors domain. The factors that we have classified in this domain indicate that health technologies can enhance connectivity with others, including nurses, physicians and relatives or patients with a similar health condition. In other words, besides helping to manage the disease, certain technologies can unintentionally have a positive effect by improving the social relationships of older people and encouraging them to embrace technology. Telecare technologies are perceived as useful for receiving social support and improving communication between patients and healthcare providers [ 67 ]. Active video games allow people to play with others [ 75 ]. Videoconferencing platforms used for home-based education help older adults, especially those who live alone, meet new people and become part of a group [ 59 , 71 , 75 , 80 ]. This allows them to share information and knowledge with other people who have the same condition [ 59 ]. Additionally, individuals who have experienced anxiety or depression found it less challenging to participate in online groups than to interact with others in person [ 59 ]. However, studies have reported that patients fear technologies could replace in-person visits from both clinicians and relatives [ 65 , 80 ]. Therefore, formal and informal caregivers should receive proper training in the use of healthcare technologies and should be encouraged to alternate between remote and in-person consultations, as this is in the best interest of the older person.

Lastly, our results contribute to a deeper understanding of the ongoing debate surrounding the impact of health technology use, specifically the ‘self-surveillance/empowerment dichotomy’.

On the one hand, the perspective of empowerment positively frames the individual responsibility in managing the disease [ 45 – 46 ]. The ‘empowered patient’ gains power through a better understanding of their illness, which in turn produces a greater sense of responsibility towards self-management of the disease. Patient empowerment thus enhances motivation and adherence to the use of health technologies. As demonstrated in this review, research on platforms for home-based telerehabilitation and health education programs, as well as on wearable activity trackers and active video games, has indicated that motivation can be fostered by various factors.

First, several studies included in the current review have shown that older adults appreciate the self-tracking functions enabled by various types of technology [ 60 , 67 , 73 , 75 , 78 ]. Health information technologies for self-tracking stimulate individuals to engage in physical or monitoring activities and evaluate their progress towards a goal [ 60 , 73 , 75 , 78 ]. Reminders for goal setting have been shown to yield motivational benefits for older adults [ 60 , 66 , 74 , 75 ], as well as to receive positive feedback on the accomplishment of personal objectives [ 69 ].

Second, m-health devices and telemonitoring platforms also provide patients with the ability to learn more about their disease, and access information and data [ 60 , 73 , 75 , 78 ]. Patients perceive greater control over the self-management of their disease and a better understanding of their condition [ 67 ]. An increased self-understanding of one’s body and illness can trigger in individuals an attitude of heightened awareness and self-efficacy, which could increase motivation, enable positive coping actions towards self-care, and improve health behaviours. Specifically, seniors show greater motivation to engage with technology when they perceive a clear connection between improved health behaviours and better health outcomes. They can recognize the additional health benefits it offers, such as enhanced autonomy and an improved quality of life [ 58 , 66 , 67 , 75 ].

Third, the motivating factors can be socially focused, as an increased sense of connectedness can contribute to generating motivation to adopt technology. Concerning telehealth platforms, the external monitoring of patients by healthcare providers (nurses and physicians) produces a perception of social support and, consequently, can motivate the usage of technology [ 67 ]. In the trial conducted by Doyle et al. on a digital health platform [ 66 ], the triage service implemented by nurses provided reassurance to participants, as they were monitored ‘behind the scenes’ by healthcare providers who could oversee their parameters and suggest interventions. Additionally, participants expressed appreciation for the social interactions established between them and the nurses [ 66 ]. Similarly, digital health platforms, especially those based on telerehabilitation trainings with other participants, can contribute to establishing social interactions with other patients who have the same disease [ 59 , 60 , 66 , 75 ]. The participants in the study by Simmich et al. [ 75 ] considered the enjoyment derived from playing games with others, specifically through active video games, as a significant motivating factor.

What Petrakaki et al. have referred to as ‘technological self-care’ is the unintended consequence of health technologies that strengthen an individual’s ability to take care of their own health [ 38 ]. This includes both personal self-discipline in meeting systemic expectations and the collective encouragement of sharing health knowledge from medical authorities with patients and then disseminating that knowledge to the broader community. This has wider ramifications for the community as a whole [ 38 ].

On the other hand, the ‘surveillance effect’, which involves the continuous monitoring of medical data, can remind patients of the negative aspects of their disease. Some may perceive the effort required by these devices as excessive, negatively affecting their motivation to use the devices [ 66 , 73 ]. Considering the pilot studies, technology appeared to older adults as an added burden to their complicated condition and some participants abandoned the trial due to the onset of health complications [ 58 , 66 , 68 ] or hospitalization [ 63 ].

More generally, program completion and adherence are challenging [ 60 , 64 , 73 ], which is particularly evident in long-term trials (one year or more) focused on activity trackers or platforms for rehabilitation training [ 66 , 68 ]. Adherence seemed high at the beginning of the trial, but over time, there was a decline in compliance for using technology for assisted home exercises [ 60 , 64 ]. Early withdrawal from the trial can be attributed to various factors, including technical difficulties arising from both structural barriers such as poor connectivity [ 59 , 68 ] and false alarms generated by the devices [ 68 ]. Likewise, frustration stemming from a negative experience with the technology, perceived as too complicated to use, contributes to participant dropout [ 58 , 60 , 66 ].

Consistently, a recent investigation into the reasons for the abandonment of wearable activity trackers has identified six factors [ 91 ]. Among these factors, the loss of motivation, which is linked to lower technology acceptance and a negative perception of personal quantification, is one of the most influential [ 91 ]. The positive or negative ‘emotional investment’ [ 92 ] that people activate when using technologies should be considered for successful adoption.

Some trials examined in the present review employed specific motivational techniques to enhance patients’ adherence to the intervention. These techniques included the use of motivational interviews [ 60 ], setting goals and action plans [ 60 , 72 ], employing reminders and motivational messages to enhance disease self-management and promote self-efficacy [ 60 , 66 , 72 , 74 , 75 ], as well as providing information on health consequences and assessing outcome goals [ 60 ]. In addition, some studies have emphasized the role of caregivers. Their involvement during the trial and the support they provided to the participants ensured better adherence [ 58 , 68 ]. However, motivational strategies and behaviour change techniques may not be fully effective if they are not accompanied by increased self-reflexivity and self-knowledge of one’s own self and their illness [ 93 ].

Moreover, the integration of self-management programs in primary care is often motivated by the need to contain financial pressure and costs associated with managing chronic diseases within healthcare systems [ 43 ]. Nevertheless, the implicit transfer of medical responsibilities from healthcare systems to individuals could result in the exclusion of patients who lack access to health technologies or choose not to adopt them [ 44 ]. This could potentially worsen inequalities in healthcare access.

Despite the results achieved, this study has some limitations. First, this integrative systematic review excluded certain publications (such as dissertations, conference proceedings, etc.) due to the adopted search strategy. Yet, we believe that the systematic review process adopted, which involved medical, psychological, and sociological databases, as well as three independent researchers who conducted screening and data extraction, ensured a rigorous approach to identifying papers containing consolidated results and relevant information. Second, the analysis focuses on publications written in English, and this may exclude other empirical evidence. Thirdly, an assessment of inter-rater agreement among the authors who reviewed the records has not been conducted. Fourth, this review could not include all studies published before 2012. It is challenging to determine the exact historical moment when certain digital health technologies began to spread, as their implementation depends on various factors (economic, social, cultural, etc.) and can vary in different world regions [ 94 , 95 ]. Moreover, this review encompasses technologies that have experienced different stages of development. Regardless, to achieve results suitable for current technological developments, it has been decided to establish a timeframe.

Despite these limitations, the selected studies allowed us to conduct an updated analysis of recent literature and identify factors that influence the use of a wide range of technologies by older people with chronic diseases.

The purpose of the present study was to systematically review the recent literature that addresses the factors related to the adoption or refusal of health technology by elderly individuals with chronic diseases. Moreover, the review provides an overview of the current state of health technologies for elderly individuals with chronic diseases, as well as the specific types of chronic diseases that have been targeted. The findings of the study might help to improve healthcare delivery for this specific population as well as delay disease progression and prevent complications. Besides the positive effects at the individual level, considering the barriers and facilitators that promote the use of health technologies for older individuals leads to a significant decrease in public health expenditure.

Future research aiming to promote technology adoption should therefore consider these factors at different levels: the level of the users, the level of the caregivers and the societal level. In addition, the research will need to delve into the actual effect of the Covid-19 pandemic on the use of technologies. Indeed, on one hand, the pandemic could have acted as a catalyst for the accelerated adoption of technology, enhancing both the implementation and utilization of Internet-based services, such as telemedicine. On the other hand, the pandemic did not address the gap in terms of digital skills, health literacy, and technological competencies among the most vulnerable people, as indicated in a recent systematic review by Elbaz et al. [ 18 ].

The study results have practical implications not only for technology developers but also for all the social actors involved in the design and implementation of healthcare technologies, including formal and informal caregivers and policy stakeholders. These actors could use this systematic review to enhance their understanding of the utilisation of technology by the ageing population. This review emphasises the factors that facilitate technology adoption and identifies barriers that impede it, with the ultimate goal of promoting health and independent living.

Data availability

The authors declare that all datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Chronic Obstructive Pulmonary Disease

Information and Communication Technology

United Nations. World Population Prospects 2022, Department of Economic and Social Affairs Population Division. 2022. Available at https://www.un.org/development/desa/pd/ . Accessed 20 Sept 2022.

Wang H, Naghavi M, Allen C. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the global burden of Disease Study 2015. Lancet. 2016;388(10053):1459–544.

Article   Google Scholar  

Eurostat. Ageing Europe. Looking at the lives of older people in the EU. Luxembourg: Publications Office of the European Union. 2020. Available at https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Ageing_Europe_-_looking_at_the_lives_of_older_people_in_the_EU . Accessed 6 Sept 2022.

Boersma P, Black LI, Ward BW. Prevalence of multiple chronic conditions among US adults, 2018. Prev Chronic Dis. 2020;17:E106.

Article   PubMed   PubMed Central   Google Scholar  

Bähler C, Huber CA, Brüngger B. Reich O. Multimorbidity, health care utilization and costs in an elderly community-dwelling population: a claims data based observational study. BMC Health Serv Res. 2015;15:1–12.

Volders E, Bolman CA, de Groot RH, Lechner L. The effect of active plus, a computer-tailored physical activity intervention, on cognitive functioning of elderly people with chronic illness (es)–study protocol for a randomized controlled trial. BMC Public Health. 2019;19(1):1–10.

Fitzpatrick AL, Powe NR, Cooper LS, Ives DG, Robbins JA. Barriers to health care access among the elderly and who perceives them. Am J Public Health. 2004;94(10):1788–94.

Maresova P, Javanmardi E, Barakovic S, Barakovic Husic J, Tomsone S, Krejcar O, et al. Consequences of chronic diseases and other limitations associated with old age. A scoping review. BMC Public Health. 2019;19(1):1431.

Doetsch J, Pilot E, Santana P, Krafft T. Potential barriers in healthcare access of the elderly population influenced by the economic crisis and the troika agreement: a qualitative case study in Lisbon, Portugal. Int J Equity Health. 2017;16(1):184.

OECD. Waiting Times for Health Services: Next in Line. OECD Health Policy Studies, Publishing OECD. Paris. 2020. Available at https://doi.org/10.1787/242e3c8c-en . Accessed 7 Sept 2022.

World Health Organization. Classification of digital health interventions v1. 0: A shared language to describe the uses of digital technology for health. Geneva. 2018.

World Health Organization. Global strategy on digital health 2020–2025. Geneva. 2021.

Lupton D. Digital Sociology. London: Routledge; 2015.

Google Scholar  

Bouma H. Gerontechnology: emerging technologies and their impact on aging in society stud. Health Technol Inf. 1998;48:93–104.

CAS   Google Scholar  

Chen K, Chan AH. Use or non-use of gerontechnology– a qualitative study. Int J Environ Res Public Health. 2013;10(10):4645–66.

Chen LK. Gerontechnology and artificial intelligence: better care for older people. Arch Gerontol Geriatr. 2020;91:104252.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Søraa RA, Fosch-Villaronga E, Quintas J, Dias J, Tøndel G, Serrano A. Mitigating isolation and loneliness in remote areas through social robot emotional care. In: Ray PK, Nakashima N, Ahmed A, Ro S, Soshino Y, editors. Mobile technologies for delivering healthcare in remote, rural, or developing regions. London: Institution of Engineering and Technology; 2020. pp. 255–68.

Elbaz S, Cinalioglu K, Sekhon K, Gruber J, Rigas C, Bodenstein K, et al. A systematic review of telemedicine for older adults with dementia during COVID-19: an alternative to in-person health services? Front Neurol. 2021;12:761965.

Goldberg EM, Lin MP, Burke LG, Jiménez FN, Davoodi NM, Merchant RC. Perspectives on Telehealth for older adults during the COVID-19 pandemic using the quadruple aim: interviews with 48 physicians. BMC Geriatric. 2022;22:188.

Article   CAS   Google Scholar  

Facchinetti G, Petrucci G, Albanesi B, De Marinis MG, Piredda M. Can smart home technologies help older adults manage their chronic condition? A systematic literature review. Int J Environ Res Public Health. 2023;20(2):1205.

Wang D, Liu S, Wu J, Lin Q. Purchase and use of home healthcare devices for the elderly: a pilot study in Shanghai, China. BMC Public Health. 2020;20(1):1–10.

Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res. 2013;15(5):e93.

Nelson LA, Mulvaney SA, Gebretsadik T, Ho YX, Johnson KB, Osborn CY. Disparities in the use of a mHealth medication adherence promotion intervention for low-income adults with type 2 diabetes. J Am Med Inf Assoc. 2016;23(1):12–8.

Showell C, Cummings E, Turner P. The invisibility of disadvantage: why do we not notice? 2017. Stud Health Technol Inform. 2017;235:388–392.

Chen K, Chan AH. Gerontechnology acceptance by elderly Hong Kong Chinese: a senior technology acceptance model (STAM). Ergonomics. 2014;57(5):635–52.

Article   PubMed   Google Scholar  

Astell AJ, McGrath C, Dove E. That’s for old so and so’s!’: does identity influence older adults’ technology adoption decisions? Ageing Soc. 2020;40(7):1550–76.

Lansley P. The promise and challenge of providing assistive technology to older people. Age Ageing. 2001;30(6):439–40.

Article   CAS   PubMed   Google Scholar  

Yusif S, Soar J, Hafeez-Baig A. Older people, assistive technologies, and the barriers to adoption: a systematic review. Int J Med Inf. 2016;94:112–6.

Claes V, Devriendt E, Tournoy J, Milisen K. Attitudes and perceptions of adults of 60 years and older towards in-home monitoring of the activities of daily living with contactless sensors: an explorative study. Int J Nurs Stud. 2015;52(1):134–48.

Londei ST, Rousseau J, Ducharme F, St-Arnaud A, Meunier J, Saint-Arnaud J, et al. An intelligent videomonitoring system for fall detection at home: perceptions of elderly people. J Telemed Telecare. 2009;15(8):383–90.

Merkel S, Kucharski A. Participatory design in gerontechnology: a systematic literature review. Gerontologist. 2019;59(1):e16–e25.

Mort M, Roberts C, Callén B. Ageing with telecare: care or coercion in austerity? Sociol Health Illn. 2013;35(6):799–812.

Gibson G, Dickinson C, Brittain K, Robinson L. Personalisation, customisation and bricolage: how people with dementia and their families make assistive technology work for them. Ageing Soc. 2019;39(11):2502–19.

Pols J, Willems D. Innovation and evaluation: taming and unleashing telecare technology. Sociol Health Illn. 2011;33(3):484–98.

Greenhalgh T, Wherton J, Sugarhood P, Hinder S, Procter R, Stones R. What matters to older people with assisted living needs? A phenomenological analysis of the use and non-use of telehealth and telecare. Soc Sci Med. 2013;93(1):86–94.

Fox NJ, Ward KJ, O’Rourke AJ. The ‘expert patient’: empowerment or medical dominance? The case of weight loss, pharmaceutical drugs and the internet. Soc Sci Med. 2005;60(6):1299–309.

Lupton D. The diverse domains of quantified selves: self-tracking modes and dataveillance. Econ Soc. 2016;45(1):101–22.

Article   MathSciNet   Google Scholar  

Petrakaki D, Hilberg E, Waring J. Between empowerment and self-discipline: governing patients’ conduct through technological self-care. Soc Sci Med. 2018;213:146–53.

Dean DMM. Governmentality: power and rule in modern society. London: Sage; 1999.

Crawshaw P. Governing at a distance: social marketing and the (bio) politics of responsibility. Soc Sci Med. 2012;75(1):200–7.

Introna LD. Algorithms, Governance, and governmentality. Sci Technol Hum Values. 2016;41(1):17–49.

Waring J, Latif A, Boyd M, Barber N, Elliott R. Pastoral power in the community pharmacy: a foucauldian analysis of services to promote patient adherence to new medicine use. Soc Sci Med. 2016;148:123–30.

Ong BN, Rogers A, Kennedy A, Bower P, Sanders T, Morden A, et al. Behaviour change and social blinkers? The role of sociology in trials of self-management behaviour in chronic conditions. Sociol Health Illn. 2014;36(2):226–38.

Øversveen E. Stratified users and technologies of empowerment: theorising social inequalities in the use and perception of diabetes self-management technologies. Sociol Health Illn. 2020;42(4):862–76.

Kallinikos J, Tempini N. Patient data as medical facts: social media practices as a foundation for medical knowledge creation. Inf Syst Res. 2014;25(4):817–33.

Till C. Exercise as labour: quantified self and the transformation of exercise into labour. Societies. 2014;4(3):446–62.

Wilson J, Heinsch M, Betts D, Booth D, Kay-Lambkin F. Barriers and facilitators to the use of e-health by older adults: a scoping review. BMC Public Health. 2021;21:1–12.

Bunn F, Dickinson A, Barnett-Page E, Mcinnes E, Horton K. A systematic review of older people’s perceptions of facilitators and barriers to participation in falls-prevention interventions. Ageing Soc. 2008;28(4):449–72.

Changizi M, Kaveh MH. Effectiveness of the mHealth technology in improvement of healthy behaviors in an elderly population-a systematic review. Mhealth. 2017;3:51.

Chalfont G, Mateus C, Varey S, Milligan C. Self-efficacy of older people using technology to self-manage COPD, hypertension, heart failure, or dementia at home: an overview of systematic reviews. Gerontologist. 2021;61(6):e318–34.

Niazkhani Z, Toni E, Cheshmekaboodi M, Georgiou A, Pirnejad H. Barriers to patient, provider, and caregiver adoption and use of electronic personal health records in chronic care: a systematic review. BMC Med Inf Decis Mak. 2020;20(1):153.

Khosravi P, Ghapanchi AH. Investigating the effectiveness of technologies applied to assist seniors: a systematic literature review. Int J Med Inf. 2016;85(1):17–26.

Leonardsen AL, Hardeland C, Helgesen AK, Grøndahl VA. Patient experiences with technology enabled care across healthcare settings- a systematic review. BMC Health Serv Res. 2020;20(1):779.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ et al. Cochrane handbook for systematic reviews of interventions. The Cochrane Collaboration. 2023. Available at https://training.cochrane.org/handbook . Accessed 29 Aug 2023.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34(4):285–91.

Ondiege B, Clarke M. Investigating user identification in remote patient monitoring devices. Bioengineering. 2017;4(3):76.

Banbury A, Parkinson L, Nancarrow S, Dart J, Gray L, Buckley J. Multi-site videoconferencing for home-based education of older people with chronic conditions: the Telehealth literacy project. J Telemed Telecare. 2014;20(7):353–9.

Herkert C, Graat-Verboom L, Gilsing-Fernhout J, Schols M, Kemps HMC. Home-based exercise program for patients with combined advanced chronic cardiac and pulmonary diseases: exploratory study. JMIR Form Res. 2021;5(11):e28634.

Zulfiqar AA, Vaudelle O, Hajjam M, Geny B, Talha S, Letourneau D, et al. Results of the GER-e-TEC experiment involving the use of an automated platform to detect the exacerbation of geriatric syndromes. J Clin Med. 2020;9(12):3836.

Zulfiqar AA, Massimbo D, Hajjam M, Gény B, Talha S, Hajjam J, et al. Glycemic disorder risk remote monitoring program in the COVID-19 very elderly patients. Front Physiol. 2021;12:749731.

Gellis ZD, Kenaley B, McGinty J, Bardelli E, Davitt J, Ten Have T. Outcomes of a telehealth intervention for homebound older adults with heart or chronic respiratory failure: a randomized controlled trial. Gerontologist. 2012;52(4):541–52.

Smaerup M, Laessoe U, Grönvall E, Henriksen JJ, Damsgaard EM. The use of computer-assisted home exercises to preserve physical function after a vestibular rehabilitation program: a randomized controlled study. Rehabil Res Pract. 2016;2016:7026317.

PubMed   PubMed Central   Google Scholar  

Middlemass JB, Vos J, Siriwardena AN. Perceptions on use of home telemonitoring in patients with long term conditions - concordance with the Health Information Technology Acceptance Model: a qualitative collective case study. BMC Med Inf Decis Mak. 2017;17(1):89.

Doyle J, Murphy E, Gavin S, Pascale A, Deparis S, Tommasi P, et al. A digital platform to support self-management of multiple chronic conditions (ProACT): findings in relation to engagement during a one-year proof-of-concept trial. J Med Internet Res. 2021;23(12):e22672.

Nancarrow S, Banbury A, Buckley J. Evaluation of a national broadband network-enabled telehealth trial for older people with chronic disease. Aust Health Rev. 2016;40(6):641–8.

Mitseva A, Peterson CB, Karamberi C, Oikonomou LC, Ballis AV, Giannakakos C, et al. Gerontechnology: providing a helping hand when caring for cognitively impaired older adults. Intermediate results from a controlled study on the satisfaction and acceptance of informal caregivers. Curr Gerontol Geriatr Res. 2012;2012:401705.

Johnson DO, Cuijpers RH, Juola JF, Torta E, Simonov M, Frisiello A, et al. Socially assistive robots: a comprehensive approach to extending independent living. Int J Soc Robot. 2014;6(2):195–211.

Christiansen L, Sanmartin Berglund J, Anderberg P, Cellek S, Zhang J, Lemmens E, et al. Associations between mobile health technology use and self-rated quality of life: a cross-sectional study on older adults with cognitive impairment. Gerontol Geriatr Med. 2021;7:23337214211018924.

Jiang Y, Sun P, Chen Z, Guo J, Wang S, Liu F, et al. Patients’ and healthcare providers’ perceptions and experiences of telehealth use and online health information use in chronic disease management for older patients with chronic obstructive pulmonary disease: a qualitative study. BMC Geriatr. 2022;22(1):9.

Robinson SA, Wan ES, Shimada SL, Richardson CR, Moy ML. Age and attitudes towards an internet-mediated, pedometer-based physical activity intervention for chronic obstructive pulmonary disease: secondary analysis. JMIR Aging. 2020;3(2):e19527.

Ancker JS, Witteman HO, Hafeez B, Provencher T, Van de Graaf M, Wei E. You get reminded you’re a sick person: personal data tracking and patients with multiple chronic conditions. J Med Internet Res. 2015;17(8):e202.

Cajita MI, Hodgson NA, Lam KW, Yoo S, Han HR. Facilitators of and barriers to mHealth adoption in older adults with heart failure. Comput Inf Nurs. 2018;36(8):376.

Simmich J, Mandrusiak A, Russell T, Smith S, Hartley N. Perspectives of older adults with chronic disease on the use of wearable technology and video games for physical activity. Digit Health. 2021;7:20552076211019900.

Søraa RA, Nyvoll P, Tøndel G, Fosch-Villaronga E, Serrano JA. The social dimension of domesticating technology: interactions between older adults, caregivers, and robots in the home. Technol Forecast Soc Change. 2021;167:120678.

AlMahadin G, Lotfi A, Zysk E, Siena FL, Carthy MM, Breedon P. Parkinson’s disease: current assessment methods and wearable devices for evaluation of movement disorder motor symptoms. A patient and healthcare professional perspective. BMC Neurol. 2020;20(1):419.

Metting E, Schrage AJ, Kocks JW, Sanderman R, van der Molen T. Assessing the needs and perspectives of patients with asthma and chronic obstructive pulmonary disease on patient web portals: focus group study. JMIR Form Res. 2018;2(2):e22.

Portz JD, Bayliss EA, Bull S, Boxer RS, Bekelman DB, Gleason K, et al. Using the technology acceptance model to explore user experience, intent to use, and use behavior of a patient portal among older adults with multiple chronic conditions: descriptive qualitative study. J Med Internet Res. 2019;21(4):e11604.

Jo HS, Hwang YS. Psychological factors that affect the acceptance and need for ICT services for older adults with chronic diseases. Gerontechnology. 2021;20(2):1–11.

Duroseau N, Abramson T, Pergament K, Chan V, Govindavari JP, Ciraco C, et al. Acceptance of technology-based tools in a sample of Parkinson’s patients. Chronic Illn. 2017;13(1):3–13.

Ha J, Park HK. Factors affecting the acceptability of technology in health care among older Korean adults with multiple chronic conditions: a cross-sectional study adopting the Senior Technology Acceptance Model. Clin Interv Aging. 2020;15:1873–81.

Price-Haywood EG, Harden-Barrios J, Ulep R, Luo Q. eHealth literacy: patient engagement in identifying strategies to encourage use of patient portals among older adults. Popul Health Manag. 2017;20(6):486–94.

Reading Turchioe M, Grossman LV, Baik D, Lee C, Maurer MS, Goyal P, et al. Older adults can successfully monitor symptoms using an inclusively designed mobile application. J Am Geriatr Soc. 2020;68(6):1313–8.

Or C, Tao D. Usability study of a computer-based self-management system for older adults with chronic diseases. JMIR Res Protoc. 2012;1(2):e13.

Rodríguez-Fernández JM, Danies E, Hoertel N, Galanter W, Saner H, Franco OH. Telemedicine readiness across medical conditions in a US national representative sample of older adults. J Appl Gerontol. 2022;41(4):982–92.

Mitsuhashi T. Effects of two-week e-learning on eHealth literacy: a randomized controlled trial of Japanese internet users. PeerJ. 2018;6:e5251.

Zaman SB, Khan RK, Evans RG, Thrift AG, Maddison R, Islam SMS. Exploring barriers to and enablers of the adoption of information and Communication Technology for the care of older adults with chronic diseases: scoping review. JMIR Aging. 2022;5(1):e25251.

Fozard J, Wahl H. Age and cohort effects in gerontechnology: a reconsideration. Gerontechnology. 2012;11(1):10–21.

Charness N, Boot WR. A grand challenge for psychology: reducing the age-related digital divide. Curr Dir Psychol Sci. 2022;31(2):187–93.

Attig C, Franke T. Abandonment of personal quantification: a review and empirical study investigating reasons for wearable activity tracking attrition. Comput Hum Behav. 2020;102:223–37.

Lupton D. Critical perspectives on digital health technologies. Sociol Compass. 2014;8(12):1344–59.

Rapp A, Tirassa M. Know thyself: a theory of the self for personal informatics. Hum Comput Interact. 2017;32(5–6):335–80.

Senbekov M, Saliev T, Bukeyeva Z, Almabayeva A, Zhanaliyeva M, Aitenova N, Toishibekov Y, Fakhradiyev I. The recent progress and applications of digital technologies in healthcare: a review. Int J Telemed Appl. 2020;2020:8830200.

European Observatory on Health Systems and Policies, Fahy N, Williams GA. Use of digital health tools in Europe: before, during and after COVID-19. World Health Organization. Regional Office for Europe. 2021 Available at https://apps.who.int/iris/handle/10665/345091 . Accessed 29 Aug 2023.

Download references

Acknowledgements

The Authors thank the chairs and the session participants for their valuable suggestions received during the ESPAnet Conference “Social Policy Change between Path Dependency and Innovation”, held in Vienna in 2022, where the preliminary results of the research have been presented. Moreover, the authors acknowledge Dr Giovanni Lamura, of the Centre for Socio-Economic Research on Ageing (IRCCS-INRCA Institute), for his helpful suggestions. VQ acknowledges her research contract co-funded by the European Union - PON Research and Innovation 2014–2020, in accordance with Article 24, paragraph 3, lett. a), of Law No. 240 of December 30, 2010, as amended, and Ministerial Decree No. 1062 of August 10, 2021.

This work has been funded by the European Union - NextGenerationEU under the Italian Ministry of University and Research (MUR) National Innovation Ecosystem [grant ECS00000041 - VITALITY - CUP E13C22001060006].

Author information

Authors and affiliations.

Department of Political Science, Communication and International Relations, University of Macerata, Don Minzoni street, 22, 62100, Macerata, Italy

Alessia Bertolazzi, Valeria Quaglia & Ramona Bongelli

You can also search for this author in PubMed   Google Scholar

Contributions

All the Authors contributed to the conceptualization of the study, collected the articles, conducted the analysis of the papers, reviewed the results, and approved the final version of the manuscript. The authors confirm their contribution to the writing of the paper as follows: AB: Background, Method, Results, Discussion; VQ: Results, Discussion, Conclusion; RB: Results, Conclusion.

Corresponding author

Correspondence to Alessia Bertolazzi .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1: The search strategy used in the electronic databases

Supplementary material 2: characteristics of included studies, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Bertolazzi, A., Quaglia, V. & Bongelli, R. Barriers and facilitators to health technology adoption by older adults with chronic diseases: an integrative systematic review. BMC Public Health 24 , 506 (2024). https://doi.org/10.1186/s12889-024-18036-5

Download citation

Received : 30 April 2023

Accepted : 07 February 2024

Published : 16 February 2024

DOI : https://doi.org/10.1186/s12889-024-18036-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Older adults
  • Technology adoption
  • Chronic disease
  • Health technology
  • Telemedicine
  • Health information technology

BMC Public Health

ISSN: 1471-2458

what is systematic review research design

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Wiley-Blackwell Online Open

Logo of blackwellopen

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure  1 . A list of excluded studies with reasons can be found in Appendix B .

An external file that holds a picture, illustration, etc.
Object name is JEBM-15-39-g001.jpg

Study selection flowchart

3.1. Characteristics of included reviews

Table  1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n  = 3), literature search ( n  = 3), study selection ( n  = 2), data extraction ( n  = 1), and RoB assessment ( n  = 2) (Table  2 ).

Summary of findings from review evaluating systematic review methods

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n  = 4), Hopewell et al. 20 and Crumley et al. 19 ( n  = 30), and Robson et al. 14 and Morissette et al. 23 ( n  = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table  2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration  or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Study Protocol

Remote measurement based care (RMBC) interventions for mental health—Protocol of a systematic review and meta-analysis

Contributed equally to this work with: Felix Machleid, Twyla Michnevich

Roles Conceptualization, Methodology, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany, Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Berlin, Germany

ORCID logo

Affiliation Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany

Roles Conceptualization, Writing – review & editing

Affiliation Department of Infectious Diseases and Respiratory Medicine, Charité Campus Virchow-Klinikum, Charité – Universitätsmedizin Berlin, Berlin, Germany

Affiliation Clinic for Psychiatry, Psychotherapy and Psychosomatics, Krankenhaus am Urban, Berlin, Germany

Affiliations Clinics for Psychiatry and Psychotherapy, Clinics at the Theodor-Wenzel-Werk, Berlin, Germany, Recovery Cat GmbH, Berlin, Germany

Roles Conceptualization, Writing – original draft

Affiliation Recovery Cat GmbH, Berlin, Germany

Affiliations Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany, Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Berlin, Germany, Recovery Cat GmbH, Berlin, Germany

  • Felix Machleid, 
  • Twyla Michnevich, 
  • Leu Huang, 
  • Louisa Schröder-Frerkes, 
  • Caspar Wiegmann, 
  • Toni Muffel, 
  • Jakob Kaminski

PLOS

  • Published: February 16, 2024
  • https://doi.org/10.1371/journal.pone.0297929
  • Peer Review
  • Reader Comments

Poor management of mental illnesses is associated with lower treatment adherence, chronification, avoidable re-hospitalisations, and high costs. Remote measurement based care (RMBC) interventions have gained increasing relevance due to its potential in providing a comprehensive and patient-centric approach to mental health management.

The systematic review and meta-analysis aims to provide a comprehensive overview and analysis of existing evidence on the use of RMBC for patients with mental illness and to examine the effectiveness of RMBC interventions in alleviating disorder-specific symptoms, reducing relapse and improving recovery-oriented outcomes, global functioning, and quality of life.

Methods and analysis

Our multidisciplinary research team will develop a comprehensive search strategy, adapted to each electronic database (PubMed, Medline, Embase, and PsychINFO) to be examined systematically. Studies with patients formally diagnosed by the International Classification of Diseases or the Diagnostic and Statistical Manual of Mental Disorders which include assessment of self-reported psychiatric symptoms will be included. Publications will be reviewed by teams of independent researchers. Quality of studies will be assessed using the Cochrane Collaboration’s tool for assessing risk of bias. Outcomes cover symptom-focused or disease-specific outcomes, relapse, recovery-focused outcomes, global functioning, quality of life and acceptability of the intervention. Further data that will be extracted includes study characteristics, target population, intervention, and tracking characteristics. Data will be synthesised qualitatively, summarising findings of the systematic review. Randomised controlled trials (RCTs) will be considered for meta-analysis if data is found comparable in terms of mental illness, study design and outcomes. Cumulative evidence will be evaluated according to the Grading of Recommendations Assessment, Development and Evaluation framework.

Trial registration

Trial registration number : PROSPERO CRD42022356176 .

Citation: Machleid F, Michnevich T, Huang L, Schröder-Frerkes L, Wiegmann C, Muffel T, et al. (2024) Remote measurement based care (RMBC) interventions for mental health—Protocol of a systematic review and meta-analysis. PLoS ONE 19(2): e0297929. https://doi.org/10.1371/journal.pone.0297929

Editor: Qin Xiang Ng, Singapore General Hospital, SINGAPORE

Received: January 23, 2023; Accepted: January 14, 2024; Published: February 16, 2024

Copyright: © 2024 Machleid et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

Funding: Felix Machleid is funded by the Junior Digital Clinician Scientist Program of the Berlin Institute of Health. Jakob Kaminski was supported by the digital health accelerator that facilitated the spin-off process of Recovery Cat (a software company dedicated to digital health in psychiatry) from Charité. No additional specific grant from any funding agency in the public, commercial or not-for-profit sectors was obtained. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: Jakob Kaminski is shareholder and managing director of Recovery Cat GmbH. Toni Muffel is an employee at Recovery Cat GmbH, Caspar Wiegmann received honorary from Recovery Cat for consulting. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

Mental illnesses are estimated to be one of the highest global burdens of disease [ 1 ]. Managing these illnesses presents challenges to patients and clinicians alike. Often, subjective symptom reporting [ 2 , 3 ], memory bias [ 4 ], complex treatment interactions, poorly coordinated transitions from inpatient to outpatient settings [ 5 ], and short, infrequent appointments in outpatient and private practices [ 6 ] result in the loss of information about symptom progression and treatment side effects. Poor continuity of care increases the risk of lower treatment adherence [ 7 ], worsening of disease [ 7 , 8 ], avoidable re-hospitalisations [ 5 , 8 ], and associated costs [ 8 ].

To address these challenges, there has been a surge in development of diagnostic and therapeutic mobile mental health (MMH) technologies. One of the leading use cases for MMH is remote measurement based care (RMBC), which involves asynchronous assessment of (electronic) patient reported outcomes (ePROs) outside of clinical encounters and their use for clinical decision-making and scheduling [ 9 , 10 ]. In addition to ’traditional’ retrospective PRO assessment formats (e.g. validated questionnaires), ambulatory and diary methods have gained interest, as they offer unique insights into patients’ experience of symptoms and well-being in their natural surroundings [ 11 ]. Such methods, termed ecological momentary assessment (EMA), have evolved with technological advances that allow self-reporting of symptoms in an internet or mobile context, including web/online, text messaging or phone-call based methods [ 12 ].

RMBC interventions have gained significance in the context of mental health management because of their ability to continuously track PROs. They offer the potential to improve the detection of deterioration, to personalise treatment plans, and improve accessibility to high quality mental health services [ 9 ].

As technology rapidly advances, studies have shown strong evidence in favour of RMBC, which can improve clinical outcomes and increase treatment adherence [ 13 – 15 ]. For example, a study including 6,424 participants with various psychiatric diagnoses found that continuous feedback to therapists on the course of symptoms was associated with a doubling of therapeutic effects related to individual (or symptomatic) functioning, interpersonal relationships, and social role performance [ 16 ].

Moreover, the benefits of measurement-based care (MBC) during in-person clinical encounters have been recognised before the rise of RMBC. MBC has been linked to faster remission [ 17 , 18 ] compared to treatment as usual (TAU) and fewer missed outpatient appointments [ 19 , 20 ]. MBC can also support clinicians in adjusting treatment quickly and effectively [ 10 , 21 ]. Patients consider MBC helpful [ 22 ], and it may improve doctor patient communication or increase treatment motivation [ 23 , 24 ].

Despite these advantages, asynchronous MBC using digital solutions is not yet widespread in clinical practice resulting in various implementation efforts [ 25 ]. Although there is limited robust scientific evidence from randomised controlled trials or longitudinal studies [ 9 ], there is a significant number of pilot and feasibility studies on RMBC systems. However these studies often have the typical limitations of academic research such as insufficient power or limited bias reduction strategies [ 26 , 27 ]. This leads to heterogeneity in available data and necessitates regular systematic evaluations identifying general trends or effects to increase the adoption of MBC and MMH in clinical practice.

In a 2018 systematic review by Goldberg et al. containing 13 RCTs on RMBC the results were promising [ 9 ]. The review highlighted the short-term feasibility and acceptability of RMBC. However, only three studies isolated the effects of RMBC experimentally, with one reporting greater symptom improvement in the RMBC group and two finding no differences between intervention group and controls. In the planned systematic review and meta-analysis, we aim to provide a comprehensive overview of the current evidence on RMBC in psychiatric care and update the above mentioned findings by Goldberg and colleagues [ 9 ]. Specifically, we will focus on interventions that aim to alleviate disorder-specific symptoms, reduce relapse, and improve recovery-oriented outcomes, global functioning, and quality of life. As the data allows, we will also conduct meta-analyses of relevant outcomes.

Protocol and registration

This protocol follows the guidelines of PRISMA-P (preferred reporting items for systematic review and explanation meta-analysis protocols) checklist ( S1 Table , [ 8 ]). The systematic review and meta-analysis were registered at PROSPERO (reference CRD42022356176).

Eligibility criteria

The research question and inclusion and exclusion criteria ( S2 Table ) were formulated using the PICOS (population, intervention, comparison, outcome, study) framework [ 28 ].

The review will include studies published before August 24th, 2022 examining adults (≥18 years) with mental health disorders according to the International Statistical Classification of Diseases and Related Health Problems (ICD) and/ or the Diagnostic and Statistical Manual (DSM). Interventions delivered solely to family members (either targets of the intervention or in addition to patients) will be excluded.

Interventions

We will include interventions that allow the assessment of self-reported symptoms of mental illness and other aspects of well-being. This feature is not required to be the predominant element of the intervention. Thus, studies that include an evidence-based form of therapy (e.g. cognitive-behavioural therapy, psychodynamic therapies, behaviour therapy or behaviour modification, etc.) plus an RMBC element will also be included.

Due to the variety of included study designs, no specific comparison to a control group is intended.

Studies must report quantitative data, such as changes in symptom-related outcomes, recovery-related outcomes (e.g. empowerment, self-efficacy, hope, social connectedness), (global) level of functioning, relapse, or quality of life. Symptom tracking entirely derived from passive sensing or passive monitoring, as well as quantitative data in the form of correlative data analyses and statistical prediction models, will be excluded.

Study designs

We will comprehensively consider randomised studies, including RCTs, cluster RCTs or factorial RCTs, non-randomised studies, including observational, cohort, cross-sectional or case-control studies; mixed methods studies, and feasibility or pilot studies with available full texts written in English or German.

Information sources and search strategy

The multidisciplinary research team developed a comprehensive search strategy by screening reference lists of scientific and grey literature, including MeSH terms related to (1) mental disorders and psychological distress, (2) measurement-based care, and (3) digital technologies. The syntax of each database search will be slightly modified due to database specifications and the searches will be conducted using PubMed, Medline, Embase, and PsychINFO. The search strategy for each database can be found in S3 Table .

Study selection

For all reports included in the synthesis, data will be collected, extracted, and reviewed in a Google Sheets spreadsheet developed by the research team. In an initial screening, titles and abstracts will each be evaluated in batches by three subgroups of independent researchers (TwM&LS, FM&LH, JK&CW&ToM). In a second screening step, full-text articles of the selected records will be retrieved and imported to the Zotero reference management software. The subgroups will assess a different batch of full-text articles than the ones they formerly screened. Disagreements between the researchers will be discussed to reach a consensus. When no consensus can be reached within the pair/trio, the discussion will be conducted within the entire research team. The rationale for the exclusion of each full text will be provided.

Data extraction

We plan to extract various data: study identification (authors, year of publication, doi, URL), population (e.g. number of cases and controls, diagnosis, age, gender, years pre-university education), tracking- (e.g. mode, items, frequency) and study characteristics (e.g. design, hypotheses, study site, duration, randomisation, post-assessment period, follow-up, outcomes, response rate). Outcomes will be categorised into six categories: (1) symptom-focused or disease-specific outcomes, (2) relapse, (3) recovery-focused outcomes, (4) (global) functioning, (5) quality of life and (6) acceptability. We will consider metrics and timing of measurements for each outcome measurement tool and instrument. We will code p-values and standardised effect sizes (e.g. Cohen’s d, hazard ratio, odds ratio) when available. One researcher will extract all study features, and an additional researcher will review the coding. Discrepancies will be resolved through discussions with a third researcher.

Assessment of bias

Risk of bias in randomised studies will be examined using the Cochrane Collaboration’s tool version 2.0 [ 29 ]. We will use Funnel plot methods to examine publication bias in analyses with more than 10 studies included [ 30 ].

Data synthesis

For all studies included, a detailed description of the data items extracted and relevant results will be provided in narrative synthesis and respective tables.

Meta-analysis

Data analysis will be conducted using RStudio [ 31 ]. All types of randomised clinical studies will be considered for meta-analysis. To account for potential variations in true effects in the studies due to differences in study populations, interventions, and target behaviours, all meta-analyses will be conducted as a random-effects analyses [ 32 ].

Different diagnostic groups such as psychosis, depression or mania will be examined separately. If k>2 studies report the same outcome, they will be meta-analytically pooled. When a single study uses multiple measures to report the same outcome, the one defined as the primary outcome of the study will be favoured, or the more common measure in other studies will be chosen. Same outcomes will be examined across all diagnostic groups, with the inclusion of the diagnosis factor as a covariate in statistical analysis. When different measures are used across studies for similar outcomes, effect sizes will be standardised for pooling [ 32 ]. We will analyse studies that include treatment as usual and active control groups together. If reported, we will use intention to treat data for our analyses.

We will express effect sizes for continuous measures as standardised mean differences (SMDs) and their 95% confidence intervals calculated using the pooled standard deviation of the interventions. SMDs will be presented as values of Hedges’ g . For dichotomous measures, we will calculate risk ratios and combine the studies using the Mantel-Haenszel method. The number needed to treat (NNT) will be calculated to further illustrate the clinical relevance of RMBC interventions.

In the case of cluster RCTs, we consider each cluster as a distinct entity, using summary measures from each individual cluster for meta-analysis. In instances where the RCT does not provide adequate details for analysis, we rely on cluster-specific information, such as the intraclass correlation coefficient, to perform an approximate analysis [ 30 ]. In factorial RCTs, data from each treatment arm is extracted separately and treated as an individual study when relevant to meta-analysis. All findings will be reported transparently [ 30 ].

Sensitivity analyses will be carried out to examine the effect of a specific study on the pooled outcomes. Subgroup analyses separating population characteristics (e.g. mean age, gender, years of pre-university education, severity of illness), study characteristics or RMBC intervention type will be conducted to reduce heterogeneity of pooled estimates.

Heterogeneity

We will assess heterogeneity using the I 2 statistic, p -value of χ 2 test and visual inspection of forest plots.

Confidence in cumulative evidence

Cumulative evidence will be evaluated according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.

Dissemination

We will publish the results of our study in a peer-reviewed journal in the field of psychiatry, digital health or telemedicine, and present them at international conferences and workshops.

Our study will summarise existing evidence on the use of RMBC interventions in mental health care and offer insights into their impact on clinical-, health service utilisation-, recovery- and quality of life related outcomes.

Supporting information

S1 table. prisma-p 2015 checklist..

https://doi.org/10.1371/journal.pone.0297929.s001

S2 Table. Inclusion and exclusion criteria related to the PICOS design.

https://doi.org/10.1371/journal.pone.0297929.s002

S3 Table. Search syntax, date of searches, and number of results returned for each database, number of references for review.

https://doi.org/10.1371/journal.pone.0297929.s003

Acknowledgments

We want to thank our colleagues at Recovery Cat for their support and thoughtful input.

  • View Article
  • PubMed/NCBI
  • Google Scholar
  • 30. Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions. Cochrane Collaboration; 2008.
  • 31. RStudio Team. In: RStudio: Integrated Development for R [Internet]. 2022 [cited 24 Dec 2022]. http://www.rstudio.com/ .
  • 32. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis. John Wiley & Sons; 2021.

IMAGES

  1. Introduction to systematic reviews

    what is systematic review research design

  2. Systematic reviews

    what is systematic review research design

  3. The Systematic Review Process

    what is systematic review research design

  4. How to Conduct a Systematic Review

    what is systematic review research design

  5. A Step by Step Guide for Conducting a Systematic Review

    what is systematic review research design

  6. Levels of evidence and study design

    what is systematic review research design

VIDEO

  1. What is Research??

  2. Statistical Procedure in Meta-Essentials

  3. Research Methodology Part I simple concepts

  4. تحديد موضوع ومجال البحث بطريقة ممنهجة |Determine the topic and field of research systematically

  5. The Research Design

  6. Research Methodology

COMMENTS

  1. Study designs: Part 7

    In this article, we describe the systematic review, a type of secondary research design that is used to summarize the results of prior primary research studies. Systematic reviews are considered the highest level of evidence for a particular research question. [ 1] Go to: SYSTEMATIC REVIEWS

  2. Systematic Review

    A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review

  3. Introduction to systematic review and meta-analysis

    Introduction A systematic review collects all possible studies related to a given topic and design, and reviews and analyzes their results [ 1 ]. During the systematic review process, the quality of studies is evaluated, and a statistical meta-analysis of the study results is conducted on the basis of their quality.

  4. How to Do a Systematic Review: A Best Practice Guide for Conducting and

    Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of e …

  5. Systematic Review

    A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review

  6. Easy guide to conducting a systematic review

    A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology.

  7. How to Write a Systematic Review: A Narrative Review

    A systematic review, as its name suggests, is a systematic way of collecting, evaluating, integrating, and presenting findings from several studies on a specific question or topic. [ 1] A systematic review is a research that, by identifying and combining evidence, is tailored to and answers the research question, based on an assessment of all ...

  8. How to do a systematic review

    What is a systematic review? A systematic review aims to bring evidence together to answer a pre-defined research question. This involves the identification of all primary research relevant to the defined review question, the critical appraisal of this research, and the synthesis of the findings.13 Systematic reviews may combine data from different

  9. Systematic reviews: Structure, form and content

    A systematic review collects secondary data, and is a synthesis of all available, relevant evidence which brings together all existing primary studies for review ( Cochrane 2016 ). A systematic review differs from other types of literature review in several major ways.

  10. Research Guides: Study Design 101: Systematic Review

    The systematic review is created after reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings. Advantages Exhaustive review of the current literature and other sources (unpublished studies, ongoing research)

  11. PDF Systematic Literature Reviews: an Introduction

    Systematic literature reviews (SRs) are a way of synthesising scientific evidence to answer a particular research question in a way that is transparent and reproducible, while seeking to include all published evidence on the topic and appraising the quality of this evidence.

  12. A step by step guide for conducting a systematic review and meta

    Systematic review/meta-analysis steps include development of research question and its validation, forming criteria, search strategy, searching databases, importing all results to a library and exporting to an excel sheet, protocol writing and registration, title and abstract screening, full-text screening, manual searching, extracting data and ...

  13. Introduction to Systematic Reviews

    Research design and evidence, by CFCF, CC BY-SA 4.0, via Wikimedia Commons. Things to know about systematic reviews: Systematic reviews are a type of research study; Systematic reviews aim to provide a comprehensive and unbiased summary of the existing evidence on a particular research question

  14. Steps of a Systematic Review

    Quick overview of the process: Steps and resources from the UMB HSHSL Guide. YouTube video (26 min) Another detailed guide on how to conduct and write a systematic review from RMIT University A roadmap for searching literature in PubMed from the VU Amsterdam Alexander, P. A. (2020).

  15. How-to conduct a systematic literature review: A quick guide for

    Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...

  16. Study designs: Part 7

    In this segment, we discuss systematic review, which is a study design used to summarize the results of several primary research studies. System …

  17. Five steps to conducting a systematic review

    A review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries.

  18. Systematic Reviews: What is a systematic review?

    Literature reviews, also known as narrative reviews, attempt to find all published materials on a subject, whereas systematic reviews try to find everything that focuses on answering a specific question. Since systematic reviews are generally associated with health related fields, their main objective is to ensure the results of the review ...

  19. Types of Reviews

    Not all research questions are well-suited for systematic reviews. Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice ...

  20. Guides: Systematic Reviews: Should I do a systematic review?

    A systematic literature review is a research methodology designed to answer a focused research question. Authors conduct a methodical and comprehensive literature synthesis focused on a well-formulated research question. Its aim is to identify and synthesize all of the scholarly research on a particular topic, including both published and ...

  21. Overview

    Systematic Review: Comprehensive literature synthesis on a specific research question, typically requires a team: Systematic; exhaustive and comprehensive; search of all available evidence: Yes: Yes: Narrative and tables, describes what is known and unknown, recommendations for future research, limitations of findings: Rapid Review

  22. Literature review as a research methodology: An ...

    A systematic review can be explained as a research method and process for identifying and critically appraising relevant research, as well as for collecting and analyzing data from said research (Liberati et al., 2009). The aim of a systematic review is to identify all empirical evidence that fits the pre-specified inclusion criteria to answer ...

  23. (PDF) Research Design and Methods: A Systematic Review of Research

    Research Design and Methods: A Systematic Review of Research Paradigms, Sampling Issues and Instruments Development International Journal of Economics & Management Sciences 6 (2) DOI:...

  24. Barriers and facilitators to health technology adoption by older adults

    This study aimed to conduct an integrative systematic review of the research evidence on the factors that facilitate or hinder the adoption of different types of technology by older individuals with chronic diseases. ... Regarding the research design of the included studies, fifteen were based on research trials, presenting different lengths of ...

  25. Systematic Review

    The systematic review is created after reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings. Advantages Exhaustive review of the current literature and other sources (unpublished studies, ongoing research)

  26. Effect of exercise for depression: systematic review and network meta

    Objective To identify the optimal dose and modality of exercise for treating major depressive disorder, compared with psychotherapy, antidepressants, and control conditions. Design Systematic review and network meta-analysis. Methods Screening, data extraction, coding, and risk of bias assessment were performed independently and in duplicate. Bayesian arm based, multilevel network meta ...

  27. An overview of methodological approaches in systematic reviews

    Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses ... SR design (type of review; no. of studies included) Topic; subject area ... To review systematically research studies, which have investigated the impact of gray literature in meta‐analyses ...

  28. Remote measurement based care (RMBC) interventions for mental health

    Background Poor management of mental illnesses is associated with lower treatment adherence, chronification, avoidable re-hospitalisations, and high costs. Remote measurement based care (RMBC) interventions have gained increasing relevance due to its potential in providing a comprehensive and patient-centric approach to mental health management. Objectives The systematic review and meta ...

  29. The role of culture in student discipline of secondary schools in cross

    The present study is a systematic review that identifies future research avenues on culture and discipline in secondary schools in a cross-cultural context. Design/methodology/approach The literature, as published in top management, education and psychology journals, was reviewed around culture and discipline in secondary schools.