How to undertake a literature search: a step-by-step guide

Affiliation.

  • 1 Literature Search Specialist, Library and Archive Service, Royal College of Nursing, London.
  • PMID: 32279549
  • DOI: 10.12968/bjon.2020.29.7.431

Undertaking a literature search can be a daunting prospect. Breaking the exercise down into smaller steps will make the process more manageable. This article suggests 10 steps that will help readers complete this task, from identifying key concepts to choosing databases for the search and saving the results and search strategy. It discusses each of the steps in a little more detail, with examples and suggestions on where to get help. This structured approach will help readers obtain a more focused set of results and, ultimately, save time and effort.

Keywords: Databases; Literature review; Literature search; Reference management software; Research questions; Search strategy.

  • Databases, Bibliographic*
  • Information Storage and Retrieval / methods*
  • Nursing Research
  • Review Literature as Topic*
  • Open access
  • Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

  • Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
  • Andrew Booth 2 ,
  • Jo Varley-Campbell 1 ,
  • Nicky Britten 3 &
  • Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

195k Accesses

193 Citations

122 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

figure 1

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Literature Search Process
  • Citation Chasing
  • Tacit Models
  • Unique Guidance
  • Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

methods of literature search

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • Current issue
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Volume 28, Issue 6
  • Rapid reviews methods series: Guidance on literature search
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0001-6644-9845 Irma Klerings 1 ,
  • Shannon Robalino 2 ,
  • http://orcid.org/0000-0003-4808-3880 Andrew Booth 3 ,
  • http://orcid.org/0000-0002-2903-6870 Camila Micaela Escobar-Liquitay 4 ,
  • Isolde Sommer 1 ,
  • http://orcid.org/0000-0001-5531-3678 Gerald Gartlehner 1 , 5 ,
  • Declan Devane 6 , 7 ,
  • Siw Waffenschmidt 8
  • On behalf of the Cochrane Rapid Reviews Methods Group
  • 1 Department for Evidence-Based Medicine and Evaluation , University of Krems (Danube University Krems) , Krems , Niederösterreich , Austria
  • 2 Center for Evidence-based Policy , Oregon Health & Science University , Portland , Oregon , USA
  • 3 School of Health and Related Research (ScHARR) , The University of Sheffield , Sheffield , UK
  • 4 Research Department, Associate Cochrane Centre , Instituto Universitario Escuela de Medicina del Hospital Italiano de Buenos Aires , Buenos Aires , Argentina
  • 5 RTI-UNC Evidence-based Practice Center , RTI International , Research Triangle Park , North Carolina , USA
  • 6 School of Nursing & Midwifery, HRB TMRN , National University of Ireland Galway , Galway , Ireland
  • 7 Evidence Synthesis Ireland & Cochrane Ireland , University of Galway , Galway , Ireland
  • 8 Information Management Department , Institute for Quality and Efficiency in Healthcare , Cologne , Germany
  • Correspondence to Irma Klerings, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems, Niederösterreich, Austria; irma.klerings{at}donau-uni.ac.at

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

  • Evidence-Based Practice
  • Systematic Reviews as Topic
  • Information Science

Data availability statement

No data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjebm-2022-112079

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Compared with systematic reviews, rapid reviews (RR) often abbreviate or limit the literature search in some way to accelerate review production. However, RR guidance rarely specifies how to select topic-appropriate search approaches.

WHAT THIS STUDY ADDS

This paper presents an overview of considerations and recommendations for RR searching, covering the complete search process from the planning stage to record management. We also provide extensive appendices with practical examples, useful sources and a glossary of terms.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

There is no one-size-fits-all solution for RR literature searching: review teams should consider what search approaches best fit their RR project.

Introduction

This paper is part of a series from the Cochrane Rapid Reviews Methods Group (RRMG) providing methodological guidance for rapid reviews (RRs). 1–3 While the RRMG’s guidance 4 5 on Cochrane RR production includes brief advice on literature searching, we aim to provide in-depth recommendations for the entire search process.

Literature searching is the foundation for all reviews; therefore, it is important to understand the goals of a specific RR. The scope of RRs varies considerably (from focused questions to overviews of broad topics). 6 As with conventional systematic reviews (SRs), there is not a one-size-fits-all approach for RR literature searches. We aim to support RR teams in choosing methods that best fit their project while understanding the limitations of modified search methods. Our recommendations derive from current systematic search guidance, evidence on modified search methods and practical experience conducting RRs.

This paper presents considerations and recommendations, described briefly in table 1 . The table also includes a comparison to the SR search process based on common recommendations. 7–10 We provide supplemental materials, including a list of additional resources, further details of our recommendations, practical examples, and a glossary (explaining the terms written in italics) in online supplemental appendices A–C .

Supplemental material

  • View inline

Recommendations for rapid review literature searching

Preparation and planning

Given that the results of systematic literature searches underpin a review, planning the searches is integral to the overall RR preparation. The RR search process follows the same steps as an SR search; therefore, RR teams must be familiar with the general standards of systematic searching . Templates (see online supplemental appendix B ) and reporting guidance 11 for SR searches can also be adapted to structure the RR search process.

Developing a plan for the literature search forms part of protocol development and should involve an information specialist (eg, librarian). Information specialists can assist in refining the research question, selecting appropriate search methods and resources, designing and executing search strategies, and reporting the search methods. At minimum, specialist input should include assessing information sources and methods and providing feedback on the primary database search strategy.

Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches (eg, using automation tools, reusing existing search strategies, omitting planning or quality assurance steps) and (2) reducing the size of the search result (eg, limiting the number of information sources, increasing the precision of search strategies, using study design filters). Study selection (ie, screening search results) is usually more resource-intensive than searching, 12 particularly for topics with complex or broad concepts or diffuse terminology; thus, the second option may be more efficient for the entire RR. Investing time upfront in optimising search sensitivity (ie, completeness) and precision (ie, positive predictive value) can save time in the long run by reducing the screening and selection workload.

Preliminary or scoping searches are critical to this process. They inform the choice of search methods and identify potentially relevant literature. Texts identified through preliminary searching serve as known relevant records that can be used throughout the search development process (see sections on database selection, development and validation of search strategies).

In addition to planning the search itself, the review team should factor in time for quality assurance steps (eg, search strategy peer review) and the management of search results (eg, deduplication, full-text retrieval).

Information sources and methods

To optimise the balance of search sensitivity and precision, RR teams should prioritise the most relevant information sources for the topic and the type of evidence required. These can include bibliographic databases (eg, MEDLINE/PubMed), grey literature sources and targeted supplementary search methods. Note that this approach differs from the Methodological Expectations of Cochrane Intervention Reviews Standards 9 where the same core set of information sources is required for every review and further supplemented by additional topic-specific and evidence-specific sources.

Choosing bibliographic databases

For many review topics, most evidence is found in peer-reviewed journal articles, making bibliographic databases the main resource of systematic searching. Limiting the number of databases searched can be a viable option in RRs, but it is important to prioritise topic-appropriate databases.

MEDLINE has been found to have high coverage for studies included in SRs 13 14 and is an appealing database choice because access is free via PubMed. However, coverage varies depending on topics and relevant study designs. 15 16 Additionally, even if all eligible studies for a topic were available in MEDLINE, search strategies will usually miss some eligible studies because search sensitivity is lower than database coverage. 13 17 This means searching MEDLINE alone is not a viable option, and additional information sources or search methods are required. Known relevant records can be used to help assess the coverage of selected databases (see also online supplemental appendix C ).

Further information sources and search techniques

Supplementary systematic search methods have three main goals, to identify (1) grey literature, (2) published literature not covered by the selected bibliographic databases and (3) database-covered literature that was not retrieved by the database searches.

When RRs search only a small number of databases, supplementary searches can be particularly important to pick up eligible studies not identified via database searching. While supplementary methods might increase the time spent on searching, they sometimes better optimise search sensitivity and precision, saving time in the long run. 18 Depending on the topic and relevant evidence, such methods can offer an alternative to adding additional specialised database searches. To decide if and what supplementary searches are helpful, it is important to evaluate what literature might be missed by the database searches and how this might affect the specific RR.

Study registries and other grey literature

Some studies indicate that the omission of grey literature searches rarely affects review conclusions. 17 19 However, the relevance of study registries and other grey literature sources is topic-dependent. 16 19–21 For example, randomised controlled trials (RCTs) on newly approved drugs are typically identified in ClinicalTrials.gov. 20 For rapidly evolving topics such as COVID-19, preprints are an important source. 21 For public health interventions, various types of grey literature may be important (eg, evaluations conducted by local public health agencies). 22

Further supplementary search methods

Other supplementary techniques (eg, checking reference lists, reviewing specific websites or electronic table of contents, contacting experts) may identify additional studies not retrieved by database searches. 23 One of the most common approaches involves checking reference lists of included studies and relevant reviews. This method may identify studies missed by limited database searches. 12 Another promising citation-based approach is using the ‘similar articles’ option in PubMed, although research has focused on updating existing SRs. 24 25

Considerations for RRs of RCTs

Databases and search methods to identify RCTs have been particularly well researched. 17 20 24 26 27 For this reason, it is possible to give more precise recommendations for RRs based on RCTs than for other types of review. Table 2 provides an overview of the most important considerations; additional information can be found in online supplemental appendix C .

Information sources for identification of randomised controlled trials (RCTs)

Search strategies

We define ‘search strategy’ as a Boolean search query in a specific database (eg, MEDLINE) using a specific interface (eg, Ovid). When several databases are searched, this query is usually developed in a primary database and interface (eg, Ovid MEDLINE) and translated to other databases.

Developing search strategies

Optimising search strategy precision while aiming for high sensitivity is critical in reducing the number of records retrieved. Preliminary searches provide crucial information to aid efficient search strategy development. Reviewing the abstracts and subject headings used in known relevant records will assist in identifying appropriate search terms. Text analysis tools can also be used to support this process, 28 29 for example, to develop ‘objectively derived’ search strategies. 30

Reusing or adapting complete search strategies (eg, from SRs identified by the preliminary searches) or selecting elements of search strategies for reuse can accelerate search strategy development. Additionally, validated search filters (eg, for study design) can be used to reduce the size of the search result without compromising the sensitivity of a search strategy. 31 However, quality assurance measures are necessary whether the search strategy is purpose-built, reused or adapted (see the ‘Quality assurance’ section.)

Database-specific and interface-specific functionalities can also be used to improve searches’ precision and reduce the search result’s size. Some options are: restricting to records where subject terms have been assigned as the major focus of an article (eg, major descriptors in MeSH), using proximity operators (ie, terms adjacent or within a set number of words), frequency operators (ie, terms have to appear a minimum number of times in an abstract) or restricting search terms to the article title. 32–34

Automated syntax translation can save time and reduce errors when translating a primary search strategy to different databases. 35 36 However, manual adjustments will usually be necessary.

The time taken to learn how to use supporting technologies (eg, text analysis, syntax translation) proficiently should not be underestimated. The time investment is most likely to pay off for frequent searchers. A later paper in this series will address supporting software for the entire review process.

Limits and restrictions

Limits and restrictions (eg, publication dates, language) are another way to reduce the number of records retrieved but should be tailored to the topic and applied with caution. For example, if most studies about an intervention were published 10 years ago, then an arbitrary cut-off of ‘the last 5 years’ will miss many relevant studies. 37 Similarly, limiting to ‘English only’ is acceptable for most cases, but early in the COVID-19 pandemic, a quarter of available research articles were written in Chinese. 38 Depending on the RR topic, certain document types (eg, conference abstracts, dissertations) might be excluded if not considered relevant to the research question.

Note also that preset limiting functions in search interfaces (eg, limit to humans) often rely on subject headings (eg, MeSH) alone. They will miss eligible studies that lack or have incomplete subject indexing. Using (validated) search filters 31 is preferable.

Updating existing reviews

One approach to RR production involves updating an existing SR. In this case, preliminary searches should be used to check if new evidence is available. If the review team decide to update the review, they should assess the original search methods and adapt these as necessary.

One option is to identify the minimum set of databases required to retrieve all the original included studies. 39 Any reused search strategies should be validated and peer-reviewed (see below) and optimised for precision and/or sensitivity.

Additionally, it is important to assess whether the topic terminology or the relevant databases have changed since the original SR search.

In some cases, designing a new search process may be more efficient than reproducing the original search.

Quality assurance and search strategy peer review

Errors in search strategies are common and can impact the sensitivity and comprehensiveness of the search result. 40 If an RR search uses a small number of information sources, such errors could affect the identification of relevant studies.

Validation of search strategies

The primary database search strategy should be validated using known relevant records (if available). This means testing if the primary search strategy retrieves eligible studies found through preliminary searching. If some known studies are not identified, the searcher assesses the reasons and decides if revisions are necessary. Even a precision-focused systematic search should identify the majority—we suggest at least 80%–90%—of known studies. This is based on benchmarks for sensitivity-precision-maximising search filters 41 and assumes that the set of known studies is representative of the whole of relevant studies.

Peer review of search strategies

Ideally, an information specialist should review the planned information sources and search methods and use the PRESS (Peer Review of Electronic Search Strategies) checklist 42 to assess the primary search strategy. Turnaround time has to be factored into the process from the outset (eg, waiting for feedback, revising the search strategy). PRESS recommends a maximum turnaround time of five working days for feedback, but in-house peer review often takes only a few hours.

If the overall RR time plan does not allow for a full peer review of the search strategy, a review team member with search experience should check the search strategy for spelling errors and correct use of Boolean operators (AND, OR, NOT) at a minimum.

Reporting and record management

Record management requirements of RRs are largely identical to SRs and have to be factored into the time plan. Teams should develop a data management plan and review the relevant reporting standards at the project’s outset. PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension) 11 is a reporting standard for SR searches that can be adapted for RRs.

Reference management software (eg, EndNote, 43 Zotero 44 ) should be used to track search results, including deduplication. Note that record management for database searches is less time-consuming than for many supplementary or grey literature searches, which often require manual entry into reference management software. 12

Additionally, software platforms for SR production (eg, Covidence, 45 EPPI-Reviewer, 46 Systematic Review Data Repository Plus 47 ) can provide a unified way to keep track of records throughout the whole review process, which can improve management and save time. These platforms and other dedicated tools (eg, SRA Deduplicator) 48 also offer automated deduplication. However, the time and cost investment necessary to appropriately use these tools have to be considered.

Decisions about search methods for an RR need to consider where time can be most usefully invested and processes accelerated. The literature search should be considered in the context of the entire review process, for example, protocol development and literature screening: Findings of preliminary searches often affect the development and refinement of the research question and the review’s eligibility criteria . In turn, they affect the number of records retrieved by the searches and therefore the time needed for literature selection.

For this reason, focusing only on reducing time spent on designing and conducting searches can be a false economy when seeking to speed up review production. While some approaches (eg, text analysis, automated syntax translation) may save time without negatively affecting search validity, others (eg, skipping quality assurance steps, using convenient information sources without considering their topic appropriateness) may harm the entire review. Information specialists can provide crucial aid concerning the appropriate design of search strategies, choice of methods and information sources.

For this reason, we consider that investing time at the outset of the review to carefully choose a small number of highly appropriate search methods and optimise search sensitivity and precision likely leads to better and more manageable results.

Ethics statements

Patient consent for publication.

Not applicable.

  • Gartlehner G ,
  • Nussbaumer-Streit B ,
  • Nussbaumer Streit B ,
  • Garritty C ,
  • Tricco AC ,
  • Nussbaumer-Streit B , et al
  • Trivella M ,
  • Hamel C , et al
  • Hartling L ,
  • Guise J-M ,
  • Kato E , et al
  • Lefebvre C ,
  • Glanville J ,
  • Briscoe S , et al
  • Higgins JPT ,
  • Lasserson T ,
  • Chandler J , et al
  • European network for Health Technology Assessment (EUnetHTA)
  • Rethlefsen ML ,
  • Kirtley S ,
  • Waffenschmidt S , et al
  • Klerings I , et al
  • Bramer WM ,
  • Giustini D ,
  • Halladay CW ,
  • Trikalinos TA ,
  • Schmid IT , et al
  • Frandsen TF ,
  • Eriksen MB ,
  • Hammer DMG , et al
  • Klerings I ,
  • Wagner G , et al
  • Husk K , et al
  • Featherstone R ,
  • Nuspl M , et al
  • Knelangen M ,
  • Hausner E ,
  • Metzendorf M-I , et al
  • Gianola S ,
  • Bargeri S , et al
  • Hillier-Brown FC ,
  • Moore HJ , et al
  • Varley-Campbell J , et al
  • Sampson M ,
  • de Bruijn B ,
  • Urquhart C , et al
  • Fitzpatrick-Lewis D , et al
  • Affengruber L ,
  • Waffenschmidt S ,
  • Kaiser T , et al
  • The InterTASC Information Specialists’ Sub-Group
  • Kleijnen J , et al
  • Jacob C , et al
  • Kaunelis D ,
  • Mensinkai S , et al
  • Mast F , et al
  • Sanders S ,
  • Carter M , et al
  • Marshall IJ ,
  • Marshall R ,
  • Wallace BC , et al
  • Fidahic M ,
  • Runjic R , et al
  • Hopewell S ,
  • Salvador-Oliván JA ,
  • Marco-Cuenca G ,
  • Arquero-Avilés R
  • Navarro-Ruan T ,
  • Hobson N , et al
  • McGowan J ,
  • Salzwedel DM , et al
  • Clarivate Analytics
  • Corporation for Digital Scholarship
  • Veritas Health Innovation Ltd
  • Graziosi S ,
  • Brunton J , et al
  • Agency for Healthcare Research and Quality
  • Institute for Evidence-Based Healthcare

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

  • Data supplement 1

Twitter @micaelaescb

Collaborators On behalf of the Cochrane Rapid Reviews Methods Group: Declan Devane, Gerald Gartlehner, Isolde Sommer.

Contributors IK, SR, AB, CME-L and SW contributed to the conceptualisation of this paper. IK, AB and CME-L wrote the first draft of the manuscript. All authors critically reviewed and revised the manuscript. IK is responsible for the overall content.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests AB is co-convenor of the Cochrane Qualitative and Implementation Methods Group. In the last 36 months, he received royalties from Systematic Approaches To a Successful Literature Review (Sage 3rd edn), payment or honoraria form the Agency for Healthcare Research and Quality, and travel support from the WHO. DD works part time for Cochrane Ireland and Evidence Synthesis Ireland, which are funded within the University of Ireland Galway (Ireland) by the Health Research Board (HRB) and the Health and Social Care, Research and Development (HSC R&D) Division of the Public Health Agency in Northern Ireland.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles

  • Research methods and reporting Rapid reviews methods series: Guidance on team considerations, study selection, data extraction and risk of bias assessment Barbara Nussbaumer-Streit Isolde Sommer Candyce Hamel Declan Devane Anna Noel-Storr Livia Puljak Marialena Trivella Gerald Gartlehner BMJ Evidence-Based Medicine 2023; 28 418-423 Published Online First: 19 Apr 2023. doi: 10.1136/bmjebm-2022-112185
  • Research methods and reporting Rapid reviews methods series: Guidance on assessing the certainty of evidence Gerald Gartlehner Barbara Nussbaumer-Streit Declan Devane Leila Kahwati Meera Viswanathan Valerie J King Amir Qaseem Elie Akl Holger J Schuenemann BMJ Evidence-Based Medicine 2023; 29 50-54 Published Online First: 19 Apr 2023. doi: 10.1136/bmjebm-2022-112111
  • Research methods and reporting Rapid Reviews Methods Series: Involving patient and public partners, healthcare providers and policymakers as knowledge users Chantelle Garritty Andrea C Tricco Maureen Smith Danielle Pollock Chris Kamel Valerie J King BMJ Evidence-Based Medicine 2023; 29 55-61 Published Online First: 19 Apr 2023. doi: 10.1136/bmjebm-2022-112070

Read the full text or download the PDF:

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

methods of literature search

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved February 22, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Book cover

Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy pp 1–15 Cite as

Methodological Approaches to Literature Review

  • Dennis Thomas 2 ,
  • Elida Zairina 3 &
  • Johnson George 4  
  • Living reference work entry
  • First Online: 09 May 2023

316 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

  • Literature review
  • Systematic review
  • Meta-analysis
  • Scoping review
  • Research methodology

This is a preview of subscription content, log in via an institution .

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research

Banner

The Literature Review: 3. Methods for Searching the Literature

  • 1. Introduction
  • 2. Why Do a Literature Review?
  • 3. Methods for Searching the Literature
  • 4. Analysing the Literature
  • 5. Organizing the Literature Review
  • 6. Writing the Review

1. Tasks Involved in a Literature Review

There are two major tasks involved in a literature review:

  • Identifying and selecting literature
  • Writing about the literature

2. Skills Required for Conducting a Literature Search

  • Information seeking skills
  • Ability to use manual and electronic methods to identify useful resources
  • Ability to conduct extensive bibliographic searches
  • Critical appraisal skills
  • Ability to describe, critique, and relate each source to the topic
  • Ability to identify areas of controversy in the literature
  • Organizational skills
  • Ability to organize the literature collected around your topic
  • Ability to present the review logically

3. Searching Techniques

Scan the literature for various types of content, including:

  • theoretical foundations and definitions
  • discussion and debate
  • current issues

Skim potential works to select materials for inclusion

  • decide whether to include or exclude a work from the review

4. Sorting the Literature

For each article identified for possible inclusion in the literature review, you need to:

1. read the abstract

  • decide whether to read the entire article

2. read the introduction

  • explains why the study is important
  • provides a review and evalution of relevant literature

3. read Methods section critically

  • focus on participants and methodology

4. evaluate results

  • are the conclusions logical?
  • is there evidence of bias?

5. Notetaking

  • Take notes as you read through each paper that you will include in the review
  • Purpose of study - research aims or hypotheses
  • Research design and methodology
  • Data analysis
  • Summary of findings

Part of the task in taking notes is to begin the process of sifting and arranging ideas

6. Questions to Keep in Mind

  • What are the key sources of information on this topic?
  • What are the major issues and debates on this topic?
  • What are the key theories, concepts, and ideas on this topic?
  • What are the main questions and problems that have been addressed so far?
  • What are the strengths and weaknesses of the various arguments on the topic?
  • Who are the significant research personalities in this area?
  • << Previous: 2. Why Do a Literature Review?
  • Next: 4. Analysing the Literature >>
  • Last Updated: Feb 8, 2022 5:25 PM
  • URL: https://libguides.uwi.edu/litreviewsoe

Systematic Reviews: Types of literature review, methods, & resources

  • Types of literature review, methods, & resources
  • Protocol and registration
  • Search strategy
  • Medical Literature Databases to search
  • Study selection and appraisal
  • Data Extraction/Coding/Study characteristics/Results
  • Reporting the quality/risk of bias
  • Manage citations using RefWorks This link opens in a new window
  • GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

  • Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
  • NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
  • RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
  • Systematic Review Toolbox Select the Healthcare discipline and features you want a tool to support.
  • SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
  • RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
  • Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
  • GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
  • DistillerSR Needs subscription. Create coded data extraction forms from templates.
  • EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
  • SUMARI Needs subscription. Qualitative data analysis.
  • Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
  • Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
  • PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

  • RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
  • Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
  • Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
  • For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
  • << Previous: Home
  • Next: Protocol and registration >>

Creative Commons License

  • Last Updated: Nov 16, 2023 9:12 AM
  • URL: https://guides.himmelfarb.gwu.edu/systematic_review

GW logo

  • Himmelfarb Intranet
  • Privacy Notice
  • Terms of Use
  • GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
  • Himmelfarb Health Sciences Library
  • 2300 Eye St., NW, Washington, DC 20037
  • Phone: (202) 994-2850
  • [email protected]
  • https://himmelfarb.gwu.edu

Banner

Best Practice for Literature Searching

  • Literature Search Best Practice

What is literature searching?

  • What are literature reviews?
  • Hierarchies of evidence
  • 1. Managing references
  • 2. Defining your research question
  • 3. Where to search
  • 4. Search strategy
  • 5. Screening results
  • 6. Paper acquisition
  • 7. Critical appraisal
  • Further resources
  • Training opportunities and videos
  • Join FSTA student advisory board This link opens in a new window
  • Chinese This link opens in a new window
  • Italian This link opens in a new window
  • Persian This link opens in a new window
  • Portuguese This link opens in a new window
  • Spanish This link opens in a new window

Literature searching is the task of finding relevant information on a topic from the available research literature. Literature searches range from short fact-finding missions to comprehensive and lengthy funded systematic reviews. Or, you may want to establish through a literature review that no one has already done the research you are conducting. If so, a comprehensive search is essential to be sure that this is true.

Whatever the scale, the aim of literature searches is to gain knowledge and aid decision-making.  They are embedded in the scientific discovery process. Literature searching is a vital component of what is called "evidence-based practice", where decisions are based on the best available evidence.

What is "literature"?

Research literature writes up research that has been done in order to share it with others around the world. Far more people can read a research article than could ever visit a particular lab, so the article is the vehicle for disseminating the research.  A research article describes in detail the research that's been done, and what the researchers think can be concluded from it.   

It is important, in literature searching, that you search for  research literature .  Scientific information is published in different formats for different purposes: in  textbooks  to teach students; in  opinion  pieces, sometimes called  editorials  or  commentaries , to persuade peers; in  review articles  to survey the state of knowledge.  An abundance of other literature is available online, but not actually published (by an academic publisher)--this includes things like  conference proceedings ,  working papers, reports  and  preprints .  This type of material is called grey (or gray) literature . 

Most of the time what you are looking for for your literature review is research literature (and not opinion pieces, grey literature, or textbook material) that has been published in  scholarly peer reviewed journals .

As expertise builds, using a greater diversity of literature becomes more appropriate.  For instance, advanced students might use conference proceedings in a literature review to map the direction of new and forthcoming research. The most advanced literature reviews, systematic reviews, need to try to track down unpublished studies to be comprehensive, and a great challenge can be locating not only relevant grey literature, but studies that have been conducted but not published anywhere.  If in doubt, always check with a teacher or supervisor about what type of literature you should be including in your search.   

Why undertake literature searches?

By undertaking regular literature searches in your area of expertise, or undertaking complex literature reviews, you are:

  • Able to provide context for and justify your research
  • Exploring new research methods
  • Highlighting gaps in existing research
  • Checking if research has been done before
  • Showing how your research fits with existing evidence
  • Identifying flaws and bias in existing research
  • Learning about terminology and different concepts related to your field
  • Able to track larger trends
  • Understanding what the majority of researchers have found on certain questions.
  • << Previous: Literature Search Best Practice
  • Next: What are literature reviews? >>
  • Last Updated: Sep 15, 2023 2:17 PM
  • URL: https://ifis.libguides.com/literature_search_best_practice
  • UNC Libraries
  • HSL Academic Process
  • Systematic Reviews
  • Step 3: Conduct Literature Searches

Systematic Reviews: Step 3: Conduct Literature Searches

Created by health science librarians.

HSL Logo

  • Step 1: Complete Pre-Review Tasks
  • Step 2: Develop a Protocol

About Step 3: Conduct Literature Searches

Partner with a librarian, systematic searching process, choose a few databases, search with controlled vocabulary and keywords, acknowledge outdated or offensive terminology, helpful tip - building your search, use nesting, boolean operators, and field tags, build your search, translate to other databases and other searching methods, document the search, updating your review.

  • Searching FAQs
  • Step 4: Manage Citations
  • Step 5: Screen Citations
  • Step 6: Assess Quality of Included Studies
  • Step 7: Extract Data from Included Studies
  • Step 8: Write the Review

  Check our FAQ's

   Email us

  Chat with us (during business hours)

   Call (919) 962-0800

   Make an appointment with a librarian

  Request a systematic or scoping review consultation

Search the FAQs

In Step 3, you will design a search strategy to find all of the articles related to your research question. You will:

  • Define the main concepts of your topic
  • Choose which databases you want to search
  • List terms to describe each concept
  • Add terms from controlled vocabulary like MeSH
  • Use field tags to tell the database where to search for terms
  • Combine terms and concepts with Boolean operators AND and OR
  • Translate your search strategy to match the format standards for each database
  • Save a copy of your search strategy and details about your search

There are many factors to think about when building a strong search strategy for systematic reviews. Librarians are available to provide support with this step of the process.

Click an item below to see how it applies to Step 3: Conduct Literature Searches.

Reporting your review with PRISMA

For PRISMA, there are specific items you will want to report from your search.  For this step, review the PRISMA-S checklist.

  • PRISMA-S for Searching
  • Specify all databases, registers, websites, organizations, reference lists, and other sources searched or consulted to identify studies. Specify the date when each source was last searched or consulted. Present the full search strategies for all databases, registers and websites, including any filters and limits used.
  • For information on how to document database searches and other search methods on your PRISMA flow diagram, visit our FAQs "How do I document database searches on my PRISMA flow diagram?" and "How do I document a grey literature search for my PRISMA flow diagram?"

Managing your review with Covidence

For this step of the review, in Covidence you can:

  • Document searches in Covidence review settings so all team members can view
  • Add keywords from your search to be highlighted in green or red while your team screens articles in your review settings

How a librarian can help with Step 3

When designing and conducting literature searches, a librarian can advise you on :

  • How to create a search strategy with Boolean operators, database-specific syntax, subject headings, and appropriate keywords 
  • How to apply previously published systematic review search strategies to your current search
  • How to test your search strategy's performance 
  • How to translate a search strategy from one database's preferred structure to another

The goal of a systematic retrieve is to find all results that are relevant to your topic. Because systematic review searches can be quite extensive and retrieve large numbers of results, an important aspect of systematic searching is limiting the number of irrelevant results that need to be screened. Librarians are experts trained in literature searching and systematic review methodology. Ask us a question or partner with a librarian to save time and improve the quality of your review. Our comparison chart detailing two tiers of partnership provides more information on how librarians can collaborate with and contribute to systematic review teams.

Magnifying glass looking at city lights

Search Process

  • Use controlled vocabulary, if applicable
  • Include synonyms/keyword terms
  • Choose databases, websites, and/or registries to search
  • Translate to other databases
  • Search using other methods (e.g. hand searching)
  • Validate and peer review the search

Databases can be multidisciplinary or subject specific. Choose the best databases for your research question. Databases index various journals, so in order to be comprehensive, it is important to search multiple databases when conducting a systematic review. Consider searching databases with more diverse or global coverage (i.e., Global Index Medicus) when appropriate. A list of frequently used databases is provided below. You can access UNC Libraries' full listing of databases on the HSL website (arranged alphabetically or by subject ).

Generally speaking, when literature searching, you are not searching the full-text article. Instead, you are searching certain citation data fields, like title, abstract, keyword, controlled vocabulary terms, and more. When developing a literature search, a good place to start is to identify searchable concepts of the research question, and then expand by adding other terms to describe those concepts. Read below for more information and examples on how to develop a literature search, as well as find tips and tricks for developing more comprehensive searches.

Identify search concepts and terms for each

Start by identifying the main concepts of your research question. If unsure, try using a question framework to help identify the main searchable concepts. PICO is one example of a question framework and is used specifically for clinical questions. If your research question doesn't fit into the PICO model well, view other examples of question frameworks and try another!

Click to view our example in PICO format

Question: for patients 65 years and older, does an influenza vaccine reduce the future risk of pneumonia, controlled vocabulary.

Controlled vocabulary is a set of terminology assigned to citations to describe the content of each reference. Searching with controlled vocabulary can improve the relevancy of search results. Many databases assign controlled vocabulary terms to citations, but their naming schema is often specific to each database. For example, the controlled vocabulary system searchable via PubMed is MeSH, or Medical Subject Headings. More information on searching MeSH can be found here .

Note: Controlled vocabulary may be outdated, and some databases allow users to submit requests to update terminology.

Click to view Controlled Vocabulary for our example PICO

As mentioned above, databases with controlled vocabulary often use their own unique system. A listing of controlled vocabulary systems by database is shown below.

Keyword Terms

Not all citations are indexed with controlled vocabulary terms, however, so it is important to combine controlled vocabulary searches with keyword, or text word, searches. 

Authors often write about the same topic in varied ways and it is important to add these terms to your search in order to capture most of the literature. For example, consider these elements when developing a list of keyword terms for each concept:

  • American versus British spelling
  • hyphenated terms
  • quality of life
  • satisfaction
  • vaccination
  • influenza vaccination

There are several resources to consider when searching for synonyms. Scan the results of preliminary searches to identify additional terms. Look for synonyms, word variations, and other possibilities in Wikipedia, other encyclopedias or dictionaries, and databases. For example, PubChem lists additional drug names and chemical compounds.

Click to display Controlled Vocabulary and Keywords for our example PICO

Combining controlled vocabulary and text words in PubMed would look like this:

"Influenza Vaccines"[Mesh] OR "influenza vaccine" OR "influenza vaccines" OR "flu vaccine" OR "flu vaccines" OR "flu shot" OR "flu shots" OR "influenza virus vaccine" OR "influenza virus vaccines"

Social and cultural norms have been rapidly changing around the world. This has led to changes in the vocabulary used, such as when describing people or populations. Library and research terminology changes more slowly, and therefore can be considered outdated, unacceptable, or overly clinical for use in conversation or writing.

For our example with people 65 years and older, APA Style Guidelines recommend that researchers use terms like “older adults” and “older persons” and forgo terms like “senior citizens” and “elderly” that connote stereotypes. While these are current recommendations, researchers will recognize that terms like “elderly” have previously been used in the literature. Therefore, removing these terms from the search strategy may result in missed relevant articles. 

Research teams need to discuss current and outdated terminology and decide which terms to include in the search to be as comprehensive as possible. The research team or a librarian can search for currently preferred terms in glossaries, dictionaries, published guidelines, and governmental or organizational websites. The University of Michigan Library provides suggested wording to use in the methods section when antiquated, non-standard, exclusionary, or potentially offensive terms are included in the search.

Check the methods sections or supplementary materials of published systematic reviews for search strategies to see what terminology they used. This can help inform your search strategy by using MeSH terms or keywords you may not have thought of. However, be aware that search strategies will differ in their comprehensiveness.

You can also run a preliminary search for your topic, sort the results by Relevance or Best Match, and skim through titles and abstracts to identify terminology from relevant articles that you should include in your search strategy.

Nesting is a term that describes organizing search terms inside parentheses. This is important because, just like their function in math, commands inside a set of parentheses occur first. Parentheses let the database know in which order terms should be combined. 

Always combine terms for a single concept inside a parentheses set. For example: 

( "Influenza Vaccines"[Mesh] OR "influenza vaccine" OR "influenza vaccines" OR "flu vaccine" OR "flu vaccines" OR "flu shot" OR "flu shots" OR "influenza virus vaccine" OR "influenza virus vaccines" )

Additionally, you may nest a subset of terms for a concept inside a larger parentheses set, as seen below. Pay careful attention to the number of parenthesis sets and ensure they are matched, meaning for every open parentheses you also have a closed one.

( "Influenza Vaccines"[Mesh] OR "influenza vaccine" OR "influenza vaccines" OR "flu vaccine" OR "flu vaccines" OR "flu shot" OR "flu shots" OR "influenza virus vaccine" OR "influenza virus vaccines" OR   (( flu OR influenza ) AND ( vaccine OR vaccines OR vaccination OR immunization )))

Boolean operators

Boolean operators are used to combine terms in literature searches. Searches are typically organized using the Boolean operators OR or AND. OR is used to combine search terms for the same concept (i.e., influenza vaccine). AND is used to combine different concepts (i.e., influenza vaccine AND older adults AND pneumonia). An example of how Boolean operators can affect search retrieval is shown below. Using AND to combine the three concepts will only retrieve results where all are present. Using OR to combine the concepts will retrieve results that use all separately or together. It is important to note that, generally speaking, when you are performing a literature search you are only searching the title, abstract, keywords and other citation data. You are not searching the full-text of the articles.

boolean venn diagram example

The last major element to consider when building systematic literature searches are field tags. Field tags tell the database exactly where to search. For example, you can use a field tag to tell a database to search for a term in just the title, the title and abstract, and more. Just like with controlled vocabulary, field tag commands are different for every database.

If you do not manually apply field tags to your search, most databases will automatically search in a set of citation data points. Databases may also overwrite your search with algorithms if you do not apply field tags. For systematic review searching, best practice is to apply field tags to each term for reproducibility.

For example:

("Influenza Vaccines"[Mesh] OR "influenza vaccine"[tw] OR "influenza vaccines"[tw] OR "flu vaccine"[tw] OR "flu vaccines"[tw] OR "flu shot"[tw] OR "flu shots"[tw] OR "influenza virus vaccine"[tw] OR "influenza virus vaccines"[tw] OR ((flu[tw] OR influenza[tw]) AND (vaccine[tw] OR vaccines[tw] OR vaccination[tw] OR immunization[tw])))

Click to view field tags for several health databases

For more information about how to use a variety of databases, check out our guides on searching.

  • Searching PubMed guide Guide to searching Medline via the PubMed database
  • Searching Embase guide Guide to searching Embase via embase.com
  • Searching Scopus guide Guide to searching Scopus via scopus.com
  • Searching EBSCO Databases guide Guide to searching CINAHL, PsycInfo, Global Health, & other databases via EBSCO

Combining search elements together

Organizational structure of literature searches is very important. Specifically, how terms are grouped (or nested) and combined with Boolean operators will drastically impact search results. These commands tell databases exactly how to combine terms together, and if done incorrectly or inefficiently, search results returned may be too broad or irrelevant.

For example, in PubMed:

(influenza OR flu) AND vaccine is a properly combined search and it produces around 50,000 results.

influenza OR flu AND vaccine is not properly combined.  Databases may read it as everything about influenza OR everything about (flu AND vaccine), which would produce more results than needed.

We recommend one or more of the following:

  • put all your synonyms together inside a set of parentheses, then put AND between the closing parenthesis of one set and the opening parenthesis of the next set
  • use a separate search box for each set of synonyms
  • run each set of synonyms as a separate search, and then combine all your searches
  • ask a librarian if your search produces too many or too few results

Click to view the proper way to combine MeSH terms and Keywords for our example PICO

Question: for patients 65 years and older, does an influenza vaccine reduce the future risk of pneumonia , translating search strategies to other databases.

Databases often use their own set of terminology and syntax. When searching multiple databases, you need to adjust the search slightly to retrieve comparable results. Our sections on Controlled Vocabulary and Field Tags have information on how to build searches in different databases.  Resources to help with this process are listed below.

  • Polyglot search A tool to translate a PubMed or Ovid search to other databases
  • Search Translation Resources (Cornell) A listing of resources for search translation from Cornell University
  • Advanced Searching Techniques (King's College London) A collection of advanced searching techniques from King's College London

Other searching methods

Hand searching.

Literature searches can be supplemented by hand searching. One of the most popular ways this is done with systematic reviews is by searching the reference list and citing articles of studies included in the review. Another method is manually browsing key journals in your field to make sure no relevant articles were missed. Other sources that may be considered for hand searching include: clinical trial registries, white papers and other reports, pharmaceutical or other corporate reports, conference proceedings, theses and dissertations, or professional association guidelines.

Searching grey literature

Grey literature typically refers to literature not published in a traditional manner and often not retrievable through large databases and other popular resources. Grey literature should be searched for inclusion in systematic reviews in order to reduce bias and increase thoroughness. There are several databases specific to grey literature that can be searched.

  • Open Grey Grey literature for Europe
  • OAIster A union catalog of millions of records representing open access resources from collections worldwide
  • Grey Matters: a practical tool for searching health-related grey literature (CADTH) From CADTH, the Canadian Agency for Drugs and Technologies in Health, Grey Matters is a practical tool for searching health-related grey literature. The MS Word document covers a grey literature checklist, including national and international health technology assessment (HTA) web sites, drug and device regulatory agencies, clinical trial registries, health economics resources, Canadian health prevalence or incidence databases, and drug formulary web sites.
  • Duke Medical Center Library: Searching for Grey Literature A good online compilation of resources by the Duke Medical Center Library.

Systematic review quality is highly dependent on the literature search(es) used to identify studies. To follow best practices for reporting search strategies, as well as increase reproducibility and transparency, document various elements of the literature search for your review. To make this process more clear, a statement and checklist for reporting literature searches has been developed and and can be found below.

  • PRISMA-S: Reporting Literature Searches in Systematic Reviews
  • Section 4.5 Cochrane Handbook - Documenting and reporting the search process

At a minimum, document and report certain elements, such as databases searched, including name (i.e., Scopus) and platform (i.e. Elsevier), websites, registries, and grey literature searched. In addition, this also may include citation searching and reaching out to experts in the field. Search strategies used in each database or source should be documented, along with any filters or limits, and dates searched. If a search has been updated or was built upon previous work, that should be noted as well. It is also helpful to document which search terms have been tested and decisions made for term inclusion or exclusion by the team. Last, any peer review process should be stated as well as the total number of records identified from each source and how deduplication was handled. 

If you have a librarian on your team who is creating and running the searches, they will handle the search documentation.

You can document search strategies in word processing software you are familiar with like Microsoft Word or Excel, or Google Docs or Sheets. A template, and separate example file, is provided below for convenience. 

  • Search Strategy Documentation Template
  • Search Strategy Documentation Example

*Some databases like PubMed are being continually updated with new technology and algorithms. This means that searches may retrieve different results than when originally run, even with the same filters, date limits, etc.

When you decide to update a systematic review search, there are two ways of identifying new articles:  

1. rerun the original search strategy without any changes. .

Rerun the original search strategy without making any changes.  Import the results into your citation manager, and remove all articles duplicated from the original set of search results.

2. Rerun the original search strategy and add an entry date filter.

Rerun the original search strategy and add a date filter for when the article was added to the database ( not the publication date).  An entry date filter will find any articles added to the results since you last ran the search, unlike a publication date filter, which would only find more recent articles.

Some examples of entry date filters for articles entered since December 31, 2021 are:

  • PubMed:   AND ("2021/12/31"[EDAT] : "3000"[EDAT])
  • Embase: AND [31-12-2021]/sd
  • CINAHL:   AND EM 20211231-20231231
  • PsycInfo: AND RD 20211231-20231231
  • Scopus:   AND LOAD-DATE AFT 20211231  

Your PRISMA flow diagram

For more information about updating the PRISMA flow diagram for your systematic review, see the information on filling out a PRISMA flow diagram for review updates on the Step 8: Write the Review page of the guide.

  • << Previous: Step 2: Develop a Protocol
  • Next: Step 4: Manage Citations >>
  • Last Updated: Feb 8, 2024 9:22 AM
  • URL: https://guides.lib.unc.edu/systematic-reviews

Search & Find

  • E-Research by Discipline
  • More Search & Find

Places & Spaces

  • Places to Study
  • Book a Study Room
  • Printers, Scanners, & Computers
  • More Places & Spaces
  • Borrowing & Circulation
  • Request a Title for Purchase
  • Schedule Instruction Session
  • More Services

Support & Guides

  • Course Reserves
  • Research Guides
  • Citing & Writing
  • More Support & Guides
  • Mission Statement
  • Diversity Statement
  • Staff Directory
  • Job Opportunities
  • Give to the Libraries
  • News & Exhibits
  • Reckoning Initiative
  • More About Us

UNC University Libraries Logo

  • Search This Site
  • Privacy Policy
  • Accessibility
  • Give Us Your Feedback
  • 208 Raleigh Street CB #3916
  • Chapel Hill, NC 27515-8890
  • 919-962-1053
  • Submission Guidelines

safety learning system header

safety learning system header

Learning Objectives

(1) Explain steps in conducting a literature search

(2) Identify resources to utilize in a literature search

(3) Perform an online literature search using U of U Health resources

Valentina is a third year pediatric resident who notices that many of the teenagers she sees in clinic use their phones to play games and connect with friends and family members. She wonders if there could be an app for teenagers to manage their chronic diseases, specifically type 1 diabetes. But where does she begin? 

What is a literature search?

iterature search is a comprehensive exploration of published literature with the purpose of finding scholarly articles on a specific topic . Managing and organizing selected scholarly works can also be useful.

Why do a literature search?

Literature search is a critical component for any evidence-based project. It helps you to understand the complexity of a clinical issue, gives you insight into the scope of a problem, and provides you with best treatment approaches and the best available evidence on the topic. Without this step, your evidence-based practice project cannot move forward.

Five steps for literature search success

There are several steps involved in conducting a literature search. You may discover more along the way, but these steps will provide a good foundation. 

Plan using PICO(T) to develop your clinical question and formulate a search strategy.

Identify a database to search.

Conduct your search in one or more databases.

Select relevant articles .

Organize your results . Remember that searching the literature is a process.

#1: Plan using PICO(T)

The PICO(T) question framework is a formula for developing answerable, researchable questions. Using PICO(T) guides you in your search for evidence and may even help you be more efficient in the process ( Click here to learn all about PICO(T) ). 

Once you have your PICO(T) question you can formulate a search strategy by identifying key words, synonyms and subject headings. These can help you determine which databases to use. 

#2: Identify a database

For your search, you will need to consult a variety of resources to find information on your topic. While some of these resources will overlap, each also contains unique information that you won’t find in other databases.  

The "Big 3" databases: Embase, PubMed, and Scopus are always important to search because they contain large numbers of citations and have a fairly broad scope. ( Click here to access these databases and others in the library's A to Z database.) 

In addition to searching these expansive databases, try one that is more topic specific.

We are here to help.

If you are conducting a literature search and are not certain of the details, don't panic! U of U Health has a wealth of resources, including experienced librarians, to help you through the process. Learn more here. 

Utah’s Epic-embedded librarian support

Did you know you can request evidence-based information from the library directly through Epic?  Contact us through Epic’s Message Basket.

Eccles Health Sciences medical librarians are able to provide expertise in articulating the clinical question, identifying appropriate data sources, and locating the best evidence in the shortest amount of time. You can also send a message to ASK EHSL .

#3: Conduct your search

Now that you have identified pertinent databases, it is time to begin the search!

Use the key words that you’ve identified from your PICO(T) question to start searching. You might start your search broadly, with just a few key words, and then add more once you see the scope of the literature. If the initial search doesn't produce many results, you can play with removing some key words and adding more granular detail.

In our intro case study, Valentina’s population is teenagers with type 1 diabetes and her intervention is a mobile app. Watch the video below to see how Valentina uses the powerful Embase PICO search feature to identify synonyms for type 1 diabetes, mobile apps, and teenagers.

Example of   Embase using PICO Why use Embase? This search casts a wider net than most databases for more results.

Common Search Terms and Symbols

AND Includes both keywords Narrows search OR Either keyword/concept Combine synonyms and similar concepts Expands search "Double quotes"  Specific phrase Wildcard* Any word ending variants (singular, plural, etc.) Example: nurs* = nurse, nurses, nursing, etc.

Controlled Vocabulary

Want to help make your search more accurate? Try using the controlled vocabulary, or main words or phrases that describe the main themes in an article, within databases. Controlled vocabulary is a standardized hierarchical system. For example, PubMed uses Medical Subject Headings or MeSH terms to “map” keywords to the controlled vocabulary. Not all databases use a controlled vocabulary, but many do. Embase’s controlled vocabulary is called Emtree, and CINAHL’s  controlled vocabulary is called CINAHL Headings. Consider focusing the controlled vocabulary as the major topic when using MeSH, Emtree, or CINAHL Headings. 

For Valentina’s question, there are MeSH terms for Adolescent, Diabetes Mellitus, Type 1, and Mobile Applications.

Example of  PubMed using MeSH MeSH helps focus your PubMed search

Talk with your librarians for more help with searching with controlled vocabularies. 

Every database uses filters to help you narrow your search. There are different filters in each database, but they tend to work in similar ways. Use filters to help you refine your search, rather than adding those keywords to the search. Filters include article/publication type, age, language, publication years, and species.

Using filters can help return the most accurate results for your search.

Article/publication types, such as randomized controlled trial, systematic reviews, can be used as filters.

Use an Age Filter, rather than adding “pediatric” or “geriatric” to your search.

Valentina uses the age filter for her question rather than as a keyword in the video below.

Example of a PubMed keyword search using filters PubMed is the most common search because it is the most widely available.

#4: Select relevant articles

Once you have completed your search, you’ll select articles that are relevant to your question. Some databases also include a “similar articles” feature which recommends other articles similar to the article you’re reviewing—this can also be a helpful tool.

When you’ve identified an article that appears relevant to your topic, use the “Snowballing” technique to find additional articles. Snowballing involves reviewing the reference lists of articles from your search. 

In other words, look at your key articles and review their reference list for additional key or seminal articles to aid in your search.

#5: Organize your results

As you begin to collect articles during your literature search, it is important to store them in an organized fashion. Most research databases include personalized accounts for storing selected references and search strategies. 

Reference managers are a great way to not only keep articles organized, but they also generate in-text citations and bibliographies when writing manuscripts, and provide a platform for sharing references with others working on your project.

A number of reference managers—such as Zotero , EndNote , RefWorks, Mendeley , and Papers are available. EndNote Basic (web-based) is freely available to U of U faculty, staff and students. If you need help with this process, contact a librarian to help you select the reference manager  that will best suit your needs.

Using these steps, you’re ready to start your literature search. It is important to remember that there is not a right or wrong way to do the search. Literature searches are an iterative process—it will take some time and negotiation to find what you are looking for. You can always change your approach, or the information resource you are using. The important thing is to just keep trying. And before you get frustrated or give up, contact a librarian . They are here to help!

This article originally appeared May 12, 2020. It was updated to reflect current practice on March 14, 2021.

Tallie Casucci

Barbara wilson.

You have a good idea about what you want to study, compare, understand or change. But where do you go from there? First, you need to be clear about exactly what it is you want to find out. In other words, what question are you attempting to answer? Librarian Tallie Casucci and nursing leaders Gigi Austria and Barb Wilson help us understand how to formulate searchable, answerable questions using the PICO(T) framework.

EBP, or evidence-based practice, is a term we encounter frequently in today’s health care environment. But what does it really mean for the health care provider? College of Nursing interim dean Barbara Wilson and Nurse manager Gigi Austria explain how to integrate EBP into all aspects of patient care.

Frequent and deliberate practice is critical to attaining procedural competency. Cheryl Yang, pediatric emergency medicine fellow, shares a framework for providing trainees with opportunities to learn, practice, and maintain procedural skills, while ensuring high standards for patient safety.

Subscribe to our newsletter

Receive the latest insights in health care equity, improvement, leadership, resilience, and more..

methods of literature search

Contact the Accelerate Team

50 North Medical Drive   |   Salt Lake City, Utah 84132   |   801-587-2157

Duke University Libraries

Literature Reviews

  • Types of reviews
  • Getting started

Types of reviews and examples

Choosing a review type.

  • 1. Define your research question
  • 2. Plan your search
  • 3. Search the literature
  • 4. Organize your results
  • 5. Synthesize your findings
  • 6. Write the review
  • Thompson Writing Studio This link opens in a new window
  • Need to write a systematic review? This link opens in a new window

methods of literature search

Contact a Librarian

Ask a Librarian

Overview of types of literature reviews

Made with  Visme Infographic Maker

  • Literature (narrative)
  • Scoping / Evidence map
  • Meta-analysis

Characteristics:

  • Provides examination of recent or current literature on a wide range of subjects
  • Varying levels of completeness / comprehensiveness, non-standardized methodology
  • May or may not include comprehensive searching, quality assessment or critical appraisal

Mitchell, L. E., & Zajchowski, C. A. (2022). The history of air quality in Utah: A narrative review.  Sustainability ,  14 (15), 9653.  doi.org/10.3390/su14159653

  • Assessment of what is already known about an issue
  • Similar to a systematic review but within a time-constrained setting
  • Typically employs methodological shortcuts, increasing risk of introducing bias, includes basic level of quality assessment
  • Best suited for issues needing quick decisions and solutions (i.e., policy recommendations)

Learn more about the method:

Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach.  Systematic reviews, 1 (1), 1-9.  https://doi.org/10.1186/2046-4053-1-10

Virginia Commonwealth University Libraries. (2021). Rapid Review Protocol .

Quarmby, S., Santos, G., & Mathias, M. (2019). Air quality strategies and technologies: A rapid review of the international evidence.  Sustainability, 11 (10), 2757.  https://doi.org/10.3390/su11102757

  • Compiles evidence from multiple reviews into one document
  • Often defines a broader question than is typical of a traditional systematic review.

Choi, G. J., & Kang, H. (2022). The umbrella review: a useful strategy in the rain of evidence.  The Korean Journal of Pain ,  35 (2), 127–128.  https://doi.org/10.3344/kjp.2022.35.2.127

Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13(3), 132–140. https://doi.org/10.1097/XEB.0000000000000055

Rojas-Rueda, D., Morales-Zamora, E., Alsufyani, W. A., Herbst, C. H., Al Balawi, S. M., Alsukait, R., & Alomran, M. (2021). Environmental risk factors and health: An umbrella review of meta-analyses.  International Journal of Environmental Research and Public Dealth ,  18 (2), 704.  https://doi.org/10.3390/ijerph18020704

  • Main purpose is to map out and categorize existing literature, identify gaps in literature
  • Search comprehensiveness determined by time/scope constraints, could take longer than a systematic review
  • No formal quality assessment or critical appraisal

Learn more about the methods :

Arksey, H., & O'Malley, L. (2005) Scoping studies: towards a methodological framework.  International Journal of Social Research Methodology ,  8 (1), 19-32.  https://doi.org/10.1080/1364557032000119616

Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science: IS, 5, 69. https://doi.org/10.1186/1748-5908-5-69

Miake-Lye, I. M., Hempel, S., Shanman, R., & Shekelle, P. G. (2016). What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products.  Systematic reviews, 5 (1), 1-21.  https://doi.org/10.1186/s13643-016-0204-x

Example : 

Rahman, A., Sarkar, A., Yadav, O. P., Achari, G., & Slobodnik, J. (2021). Potential human health risks due to environmental exposure to nano-and microplastics and knowledge gaps: A scoping review.  Science of the Total Environment, 757 , 143872.  https://doi.org/10.1016/j.scitotenv.2020.143872

  • Seeks to systematically search for, appraise, and synthesize research evidence
  • Adheres to strict guidelines, protocols, and frameworks
  • Time-intensive and often take months to a year or more to complete. 
  • The most commonly referred to type of evidence synthesis. Sometimes confused as a blanket term for other types of reviews.

Gascon, M., Triguero-Mas, M., Martínez, D., Dadvand, P., Forns, J., Plasència, A., & Nieuwenhuijsen, M. J. (2015). Mental health benefits of long-term exposure to residential green and blue spaces: a systematic review.  International Journal of Environmental Research and Public Health ,  12 (4), 4354–4379.  https://doi.org/10.3390/ijerph120404354

  • Statistical technique for combining results of quantitative studies to provide more precise effect of results
  • Aims for exhaustive, comprehensive searching
  • Quality assessment may determine inclusion/exclusion criteria
  • May be conducted independently or as part of a systematic review

Berman, N. G., & Parker, R. A. (2002). Meta-analysis: Neither quick nor easy. BMC Medical Research Methodology , 2(1), 10. https://doi.org/10.1186/1471-2288-2-10

Hites R. A. (2004). Polybrominated diphenyl ethers in the environment and in people: a meta-analysis of concentrations.  Environmental Science & Technology ,  38 (4), 945–956.  https://doi.org/10.1021/es035082g

Flowchart of review types

  • Review Decision Tree - Cornell University For more information, check out Cornell's review methodology decision tree.
  • LitR-Ex.com - Eight literature review methodologies Learn more about 8 different review types (incl. Systematic Reviews and Scoping Reviews) with practical tips about strengths and weaknesses of different methods.
  • << Previous: Getting started
  • Next: 1. Define your research question >>
  • Last Updated: Feb 15, 2024 1:45 PM
  • URL: https://guides.library.duke.edu/lit-reviews

Duke University Libraries

Services for...

  • Faculty & Instructors
  • Graduate Students
  • Undergraduate Students
  • International Students
  • Patrons with Disabilities

Twitter

  • Harmful Language Statement
  • Re-use & Attribution / Privacy
  • Support the Libraries

Creative Commons License

  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

990 Accesses

55 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

methods of literature search

  • Open access
  • Published: 09 February 2024

Key influences on university students’ physical activity: a systematic review using the Theoretical Domains Framework and the COM-B model of human behaviour

  • Catherine E. B. Brown 1 ,
  • Karyn Richardson 1 ,
  • Bengianni Halil-Pizzirani 1 ,
  • Lou Atkins 2 ,
  • Murat Yücel 3   na1 &
  • Rebecca A. Segrave 1   na1  

BMC Public Health volume  24 , Article number:  418 ( 2024 ) Cite this article

697 Accesses

Metrics details

Physical activity is important for all aspects of health, yet most university students are not active enough to reap these benefits. Understanding the factors that influence physical activity in the context of behaviour change theory is valuable to inform the development of effective evidence-based interventions to increase university students’ physical activity. The current systematic review a) identified barriers and facilitators to university students’ physical activity, b) mapped these factors to the Theoretical Domains Framework (TDF) and COM-B model, and c) ranked the relative importance of TDF domains.

Data synthesis included qualitative, quantitative, and mixed-methods research published between 01.01.2010—15.03.2023. Four databases (MEDLINE, PsycINFO, SPORTDiscus, and Scopus) were searched to identify publications on the barriers/facilitators to university students' physical activity. Data regarding study design and key findings (i.e., participant quotes, qualitative theme descriptions, and survey results) were extracted. Framework analysis was used to code barriers/facilitators to the TDF and COM-B model. Within each TDF domain, thematic analysis was used to group similar barriers/facilitators into descriptive theme labels. TDF domains were ranked by relative importance based on frequency, elaboration, and evidence of mixed barriers/facilitators.

Thirty-nine studies involving 17,771 participants met the inclusion criteria. Fifty-six barriers and facilitators mapping to twelve TDF domains and the COM-B model were identified as relevant to students’ physical activity. Three TDF domains, environmental context and resources (e.g., time constraints), social influences (e.g., exercising with others), and goals (e.g., prioritisation of physical activity) were judged to be of greatest relative importance (identified in > 50% of studies). TDF domains of lower relative importance were intentions, reinforcement, emotion, beliefs about consequences, knowledge, physical skills, beliefs about capabilities, cognitive and interpersonal skills, social/professional role and identity, and behavioural regulation. No barriers/facilitators relating to the TDF domains of memory, attention and decision process, or optimism were identified.

Conclusions

The current findings provide a foundation to enhance the development of theory and evidence informed interventions to support university students’ engagement in physical activity. Interventions that include a focus on the TDF domains 'environmental context and resources,' 'social influences,' and 'goals,' hold particular promise for promoting active student lifestyles.

Trial registration

Prospero ID—CRD42021242170.

Peer Review reports

Physical activity (PA) has a powerful positive impact on all aspects of health. Regular PA can prevent and treat noncommunicable diseases [ 1 , 2 ], build resilience against the development of mental illness [ 3 ], and attenuate cognitive decline [ 4 ]. Given these pervasive health benefits, increasing participation in PA is recognised as a global priority by international public health organisations. Indeed, a core aspect of the World Health Organisation’s action plan for a “healthier world” is to achieve a 15% reduction in the global prevalence of physical inactivity by 2030 [ 5 ].

Despite international efforts to reduce physical inactivity, university students frequently do not meet the recommended level of PA required to attain its health benefits. Approximately 40–50% of university students are physically inactive [ 6 ], many of whom attribute their inactivity to unique challenges associated with university life. For many students, the transition to university coincides with new academic, social, financial, and personal responsibilities [ 7 ], disrupting established routines and imposing additional barriers to the initiation or maintenance of healthy lifestyle habits such as regular PA [ 8 ]. Students’ PA tends to decline further during periods of high stress and academic pressure, such as exams and assignment deadlines [ 9 ]. This pattern has been observed across diverse university populations and cultural contexts [ 10 , 11 , 12 ], highlighting the importance of understanding the factors that contribute to physical inactivity among this cohort globally.

Understanding the barriers and facilitators to PA in the context of the university setting is an important step in developing effective, targeted interventions to promote active lifestyles among university students. A recently published systematic review found that lack of time, motivation, access to places to practice PA, and financial resources were primary barriers to PA for undergraduate university students [ 13 ]. A corresponding and complementary synthesis of the facilitators of PA, however, has not yet been conducted. Such a synthesis would be valuable in enabling a comprehensive understanding of the factors that influence students' PA and identifying facilitators that could be leveraged in intervention design. Furthermore, applying theoretical frameworks to understand barriers and facilitators to PA can guide the development of theory-informed, evidence-based interventions for university students that purposely and effectively target factors that influence their participation in PA.

The Theoretical Domains Framework (TDF) [ 14 , 15 , 16 ] and the COM-B model of behaviour [ 17 ] are two robust, gold-standard frameworks frequently used to examine the determinants of human behaviour. The TDF is an integrated framework of 14 theoretical domains (see Additional file 1 for domains, definitions, and constructs) which provide a comprehensive understanding of the key factors driving behaviour. The TDF was developed through expert consensus, synthesising 33 psychological theories (such as social cognitive theory [ 18 , 19 ] and the theory of planned behaviour [ 20 , 21 ] and 128 theoretical constructs (such as ‘competence’, ‘goal priority’, etc.) across disciplines identified as most relevant to the implementation of behaviour change interventions. Identifying the relative importance of theoretical domains allows intervention designers to triage which behaviour change strategies should be prioritised in intervention development [ 22 , 23 ]. The TDF has been widely applied by researchers and practitioners to systematically identify which theoretical domains are most relevant for understanding health behaviour change and policy implementation across a range of contexts, including education [ 24 ], healthcare [ 25 ], and workplace environments [ 26 ].

The 14 TDF domains map onto the COM-B model (Fig.  1 ), which is a broader framework for understanding behaviour and provides a direct link to intervention development frameworks. The COM-B model posits that no behaviour will occur without sufficient capability, opportunity, and motivation. Where any of these are lacking, they can be strategically targeted to support increased engagement in a desired behaviour, including participation in PA. Within the COM-B model, capability can be psychological (e.g., knowledge to engage in the necessary processes) or physical (e.g., physical skills); opportunity can be social (e.g., interpersonal influences) or physical (e.g., environmental resources); and motivation can be automatic (e.g., emotional reactions, habits) or reflective (e.g., intentions, beliefs). The COM-B model was developed through a process of theoretical analysis, empirical evidence, and expert consensus as a central part of a broader framework for developing behaviour change interventions known as the Behaviour Change Wheel (BCW) [ 17 ].

figure 1

The TDF domains linked to the COM-B model subcomponents

Note. Reproduced from Atkins, L., Francis, J., Islam, R., et al. (2017) A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Science 12, 77.  https://doi.org/10.1186/s13012-017-0605-9

Using the TDF and COM-B model to understand the barriers and facilitators to university students’ participation in PA is valuable to inform the development of effective evidence-based interventions that are tailored to address the most influential determinants of behaviour change. As such, this systematic review aimed to: a) identify barriers and facilitators to university students’ participation in PA; b) map these factors using the TDF and COM-B model; and c) determine the relative importance of each TDF domain.

Study design

The systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 27 ]. The review protocol was registered on PROSPERO (CRD42021242170).

Search strategy

Search terms and parameters were developed in collaboration with a Monash University librarian with expertise in systematic review methodology. The following databases were searched on 15.03.2023 to identify relevant literature: MEDLINE, PsycINFO, and SPORTDiscus. Key articles were also selected for citation searching via Scopus. In consultation with a librarian, these databases were selected due to their unique scope, relevance, broad coverage, and utility. This process ensured the identified literature aligned with the aim and research topic of our systematic review. A 01.01.2010—15.03.2023 publication period was purposefully specified to account for the significant advancements in digital fitness support and tracking tools within the past decade [ 28 ], All available records were searched using the following combination of concepts in the title or abstract of the article: 1) barriers, facilitators, or intervention, Footnote 1 2) physical activity, 3) university, and 4) students. Each search concept was created by first developing a list of search terms relevant to each concept (e.g., for the ‘physical activity’ concept search terms included ‘physical exercise’, ‘physical fitness’, ‘sports’, ‘inactive’, ‘sedentary’, etc.). To create each concept, search terms were then searched collectively using the operator ‘OR’. Each search concept was then combined into the final search by using the operator ‘AND’. Search terms related to concepts 1, 2 and 3 included indexed terms unique and relevant to each database (i.e., Medical Subject Heading Terms for MEDLINE, Index Terms for PsycINFO, and Thesaurus terms for SPORTDiscus). The search was performed according to Boolean operators (e.g., AND, OR) (see Additional file 2 for the complete search syntax for MEDLINE). Unpublished studies were not sought.

Selection criteria

Articles were included if they: (a) reported university students’ self-reported barriers and/or facilitators to physical activity or exercise Footnote 2 ; (b) were written in English; and (c) were peer-reviewed journal articles. Articles encompassed studies directly investigating barriers and/or facilitators to students’ participation in PA and physical exercise intervention studies, where the latter reported participants’ self-reported barriers and/or facilitators to intervention adherence (see Table  1 below for full criteria).

Study selection

Identified articles were uploaded to EndNote X9 software [ 30 ]. A duplication detection tool was used to detect duplicates, which were then screened for accuracy by CB prior to removal. The remaining articles were uploaded to Covidence to enable blind screening and conflict resolution. Articles were screened at the title and abstract level against the inclusion and exclusion criteria by author CB, and 25% were independently screened by BP. The full text of studies meeting the inclusion criteria was then screened against the same criteria by CB, and 25% were again independently screened by BP. Differences were resolved by an independent author (KR). Inter-rater agreement in screening between CB and BP was high (0.96 for title and abstract screening, 0.83 for full-text screening). The decision to dual-screen 25% of studies was strategically chosen to balance thoroughness with efficiency, ensuring both the validity of the screening criteria and the reliability of the primary screener’s decisions. This approach aligns with the protocols used in similar systematic reviews in the field (e.g., [ 31 , 32 ]).

Data extraction

Key article characteristics were extracted, including the author/s, year of publication, country of origin, participant characteristics (e.g., enrolment status, exercise engagement [if reported]), sample size, research design, methods, and analytical approach. Barriers and facilitators were also extracted for each article and subsequently coded according to the 14 domains of the TDF and six subcomponents of the COM-B model. Quantitative data were only extracted if ≥ 50% of students endorsed a factor as a barrier or facilitator. This cut-off criterion was applied to maintain focus on the most common variables of influence and aligns with other reviews synthesising common barriers and facilitators to behaviour change (e.g., [ 26 , 33 ]).

A coding manual was developed to guide the process of mapping barriers and facilitators to the TDF and COM-B. All articles were independently coded by at least two authors (CB and BS, BP or KR). The first version of the manual was developed a priori, based on established guides for applying the TDF and COM-B model to investigate barriers and facilitators to behaviour [ 14 , 34 ], and updated as needed via regular consultation with a co-author and TDF/COM-B designer LA to ensure the accuracy of the data extraction. Barriers and facilitators were only coded to multiple TDF domains if deemed essential to accurately contextualise the core elements of the barrier/facilitator, and when the data in individual papers was described in sufficient detail to indicate that more than one domain was relevant. For example, if ‘lack of time due to competing priorities’ was reported as a barrier to PA, this encompassed both the ‘environmental context and resources’ (i.e., time) and ‘goals’ (i.e., competing priorities) domains of the TDF. Coding conflicts were resolved via discussion with LA.

Data analysis

The following three-step method was utilised to synthesise quantitative and qualitative data:

Framework analysis [ 35 ] was conducted to deductively code barriers and facilitators onto TDF domains and COM-B subcomponents. This involved identifying barriers and facilitators in each article, extracting and labelling them, and determining their relevance against the definitions of the TDF domains and COM-B subcomponents. This process involved creating tables to assist in the systematic categorisation of barriers and facilitators into relevant TDF domains and COM-B subcomponents.

Within each TDF domain, thematic analysis [ 36 ] was conducted to group similar barriers and facilitators together and inductively generate summary theme labels.

The relative importance of each TDF domain was calculated according to frequency (number of studies), elaboration (number of themes) and the identification of mixed barriers/facilitators regarding whether a theme was a barrier or facilitator within each domain (e.g., if some participants reported that receiving encouragement from their family to exercise was a facilitator, and others reported that lack of encouragement from their family to exercise was a barrier). The rank order was determined first by frequency, then elaboration, and finally by mixed barriers/facilitators.

This methodology follows previous studies using the TDF and COM-B to characterise barriers and facilitators to behaviour change and rank their relative importance [ 22 , 23 ].

Study characteristics

Following the removal of duplicates, 6,152 articles met the search criteria and were screened based on title and abstract. A total of 5,995 articles were excluded because they did not meet the inclusion criteria (see Fig.  2 below for the PRISMA flowchart). After the title and abstract screening, 157 full-text articles were retrieved and assessed for eligibility. One additional article was identified and included following citation searching of selected key articles. Thirty-nine articles met the inclusion criteria (see Additional file 3 for a summary of these studies). Eight studies were conducted in the USA, seven in Canada, three in Germany, two each in Qatar, Spain, the United Arab Emirates, and the United Kingdom, and one each in Australia, Belgium, Columbia, Egypt, Ireland, Japan, Kuwait, Malaysia, New Zealand, Saudi Arabia, South Africa, Sri Lanka, and Uganda.

figure 2

PRISMA flowchart illustrating the article selection process

Relative importance of TDF domains and COM-B components

Twelve of the 14 TDF domains and all six subcomponents of the COM-B model were identified as relevant to university students' PA. The rank order of relative importance of TDF domains and associated COM-B subcomponents are presented in Table  2 . The three most important domains were identified in at least 54% of studies.

Barriers and facilitators to student’s physical activity

Within the TDF domains, 56 total themes were identified, including 26 mixed barriers/facilitators, 18 facilitators and 12 barriers (Table  3 ). The barriers and facilitators identified within each TDF domain are summarised below (with associated COM-B subcomponent presented in parentheses), in order of relative importance:

1. Environmental context and resources (Physical Opportunity) ( n  = 90% studies)

The most frequent barrier to PA across all TDF domains was ‘lack of time’, most often in the context of study demands. Time constraints were exacerbated by long commutes to university, family responsibilities, involvement in co-curricular activities, and employment commitments. Students’ need for ‘easily accessible exercise options, facilities and equipment’ was a recurring theme. PA was deemed inaccessible if exercise facilities and other infrastructure to support PA, such as bike paths and running trails, were situated too far from the university campus or students’ residences, or if fitness classes were scheduled at inconvenient times. ‘Financial costs’ emerged as a theme. The costs associated with accessing exercise facilities, equipment and programs consistently deterred students from engaging in PA. The desire for ‘safe and enjoyable’, ‘weather appropriate’ environments for PA were frequently reported. Participating in outdoor PA in green spaces or near water increased enjoyment, provided the environment felt safe and weather conditions were suitable for PA. Factors related to students’ home, work, and university environment impacted their participation in ‘incidental PA’. Incidental PA was influenced by whether students engaged in domestic house chores, and manual work, and actively commuted to university and between classes on-campus. Students’ ‘access to a variety of physical activities’ and ‘information provision regarding on-campus exercise options’ impacted their PA. Students most often had access to a wide variety of physical activities, however, it could be difficult to access information about what types of activities were available on-campus and how to sign up to participate. The ‘lack of personalised physical activities to cater to individual fitness needs’ was a barrier, particularly for students with low levels of PA who required beginner-oriented programs. Another barrier was the ‘lack of university policy and promotion to encourage PA’, which led students to perceive that there was no obligation to participate in PA and that the university did not value it. ‘Health-concerning behaviours associated with university’, including poor diet, increased alcohol intake and sedentary behaviour, negatively impacted students’ PA. ‘Listening to music while exercising’ was a facilitator.

2. Social influences (Social Opportunity) ( n  = 72% studies)

Within social influences, ‘exercising with others’ emerged as the most frequent theme. Doing so increased students’ accountability, enjoyment and motivation, and helped them to overcome feelings of intimidation when exercising alone. Having a lack of friends to exercise with was a particular concern for students who were new to exercise or infrequently participated in PA. Receiving ‘encouragement from others to be physically active’, such as family members, friends, peers, and fitness instructors, shaped students’ values toward PA and enhanced their motivation and self-efficacy. Students’ family members, friends and teachers discouraged PA if it was not valued, or in favour of other priorities, such as academic commitments. Another recurrent theme was ‘competition or relative comparison to others’. While most students were motivated by competition, a minority felt demotivated if they compared themselves to others with higher PA standards, especially if they failed to achieve similar PA goals. Sociocultural norms influenced barriers/facilitators to PA across different cultures, and between various groups, such as international versus domestic students, and women versus men. Students from Japan and Hawaii viewed PA as an important part of their culture, in contrast to students from the Philippines who described the opposite. Participation in PA enabled international students to integrate with domestic students and learn about the local culture, however cultural segregation was a barrier to participation in university team sports. For female students from some middle-eastern countries, including Saudi Arabia, the UAE and Qatar, cultural norms made it impermissible for women to engage in PA, particularly compared to men. Religion also differentially impacted barriers/facilitators between women and men. Muslim women reported that Islamic practices, such as needing to engage in PA separately from men, be accompanied by a male family member while going outdoors, or dress modestly, posed additional barriers to PA. However, one study reported that Islamic teachings generally encouraged PA for both women and men by emphasising the importance of maintaining good health. Other gender-specific barriers were identified. Women often felt unwelcome or intimidated by men in exercise facilities, partly due to the perception that these facilities were tailored toward “masculine” sports and/or dominated by men. ‘Being stared at while engaging in PA’ was another barrier, impacting both women and students with a disability. A less common facilitator was the influence of both positive and negative ‘exercise role models’. For example, students practiced PA because they aspired to be like someone who was physically active, or because they did not want to be like someone who was not physically active.

3. Goals (Reflective Motivation) ( n  = 54%)

‘Prioritisation of PA compared to other activities’ was the most common theme within goals. Students frequently prioritised other activities, such as study, social activities, or work, over PA. However, those who played team sports or regularly practiced PA were more inclined to prioritise it for its recognised health benefits (i.e., stress management), and its role in enhancing confidence. Additional facilitators included ‘engaging in PA to achieve an external goal’, such as improving one’s appearance, and ‘setting specific PA-related goals’ as a means to enhance accountability.

4. Intentions (Reflective Motivation) ( n  = 44%)

Within intentions, ‘motivation to engage in PA’ was the most common theme. Students most often noted a lack of self-motivation for PA. Less frequent barriers included perceiving PA as an obligatory or necessary "chore", and ‘failing to follow through on intentions to engage in PA’. Conversely, ‘self-discipline to engage in PA’ emerged as a facilitator that assisted students in maintaining a regular PA routine.

5. Reinforcement (Automatic Motivation) ( n  = 38%)

The most frequent facilitator within reinforcement was ‘experiencing the positive effects of PA’ on their health and wellbeing. These included physical health benefits (i.e., maintaining fitness), psychological benefits (i.e., stress reduction), and cognitive health benefits (i.e., enhanced academic performance). Conversely, barriers arose from ‘experiencing discomfort during or after PA’ due to pain, muscle soreness or fatigue. ‘Past and current habits and routines’ was a theme. Students were more likely to participate in PA if they had established regular exercise routines, and that forming these habits at an early age made it easier to maintain them later in life. However, maintaining a regular PA routine was difficult in the context of inflexible university schedules. Students’ ‘sense of accomplishment in relation to PA’ was a theme. Students were less likely to feel a sense of accomplishment after participating in PA if it was not physically challenging. Consistent facilitators were ‘receiving positive feedback from others’ after engaging in PA, such as compliments, and ‘receiving incentives’, such as reducing the cost of gym memberships if students participated in more PA. ‘Experiencing a sense of achievement’ after reaching a PA-related goal or winning a sports match also served as a facilitator.

6. Emotion (Automatic Motivation) ( n  = 38%)

‘Enjoyment’ was the most frequently cited emotional theme. Most students reported that PA was fun and/or associated with positive feelings, however, a minority described PA as unenjoyable, boring, and repetitive. Students’ ‘poor mental health and negative affectivity’ (such as feeling sad, stressed or self-conscious, as well as fear of injury and pain), adversely impacted their motivation to be physically active.

7. Beliefs about consequences (Reflective Motivation) ( n  = 31%)

‘Beliefs about the physical health consequences of PA’ was the most recurrent barrier/facilitator. Most students understood that PA was essential for maintaining good health and preventing illness. However, some students who rarely or never engaged in PA believed they could delay pursuing an active lifestyle until they were older without compromising their health. Participating in PA to ‘maintain or improve one’s physical appearance’ acted as a facilitator. This motivation was most often cited in contexts such as increasing or decreasing weight, changing body shape or enhancing muscle tone. Beliefs about the positive environmental, occupational and psychological impacts of PA also served as facilitators. Students were motivated to participate in PA due to the environmental benefits of using active transport. They also acknowledged the importance of being physically fit for work and believed that being active was beneficial for mental health. ‘Receiving advice to participate in PA from a credible source’, such as a health professional, further facilitated students’ motivation to be active.

8. Knowledge (Psychological Capability) ( n  = 28%)

'Knowledge about the benefits of PA’, encompassing an understanding of the various types of benefits (i.e., physical, mental, or cognitive) and the biological mechanisms by which PA brings about these changes was identified as the most common knowledge theme. Being aware of these benefits positively influenced students’ motivation to be physically active. Conversely, students’ lack of knowledge about the gym environment and the programs available were barriers to PA. Regarding the gym environment, students’ ‘lack of knowledge about how to navigate through the gym, what exercises to do, and how to use exercise equipment’ amplified feelings of intimidation. Likewise, ‘lack of knowledge about the types of exercise programs and activities that were available on-campus, and how to sign up to participate’ were all barriers. A unique theme emerged concerning ‘knowledge about how to adapt physical activities for students with a disability’. Students with a disability described how fitness instructors often had a limited understanding of how to modify activities to enable them to participate. However, students with a disability were able to overcome this barrier if they possessed their own knowledge about how to tailor physical activities to meet their specific needs.

9. Physical skills (Physical Capability) ( n  = 21%)

The most prevalent theme within physical skills was ‘having the physical skills and fitness to participate in PA’. A lack of physical skills was most frequently a hindrance to PA. Additional obstacles to PA included being physically inhibited due to a ‘lack of energy’ or ‘physical injury’.

10. Beliefs about capabilities (Reflective Motivation) ( n  = 18%)

Within beliefs about capabilities, ‘self-efficacy to participate in PA’ was the most recurrent theme. Students who doubted their success in becoming physically active or who lacked confidence in their ability to initiate PA or participate in sport were less motivated to take part. A less frequent facilitator was students’ ‘self-affirmation to participate in PA’, often referring to positive cognitions about one’s own physical abilities.

11. Cognitive and interpersonal skills (Psychological Capability) ( n  = 15%)

‘Time-management’ was the only theme identified within cognitive and interpersonal skills. Students who struggled to manage their time effectively found it difficult to incorporate regular PA into their daily routine.

12. Social/professional role and identity (Reflective Motivation) ( n  = 8%)

The most frequent theme within social/professional role and identity was ‘perceiving PA as a part of one’s self-identity’. Students who engaged regularly in PA often considered it integral to their identity. Conversely, students who perceived they did not align with the aesthetic and superficial stereotypes commonly associated with the fitness industry felt less motivated to be active. A specific facilitator emerged among physiotherapy students, who were motivated to be active due to the emphasis on PA within their profession.

13. Behavioural regulation (Psychological Capability) ( n  = 3%)

Within the domain of behavioural regulation, two facilitators were equally prevalent: ‘self-monitoring of PA’ and ‘feedback on progress towards a PA-related goal’. By keeping track of their step count and receiving feedback on walking goals, students were motivated to exceed the average number of daily steps or achieve their personal PA targets.

14. Memory, attention, and decision process (Psychological Capability); Optimism (Reflective Motivation) ( n  = 0%)

No barriers or facilitators relating to the TDF domains of memory, attention and decision process, or optimism were identified.

This systematic review used the TDF and COM-B model to identify barriers and facilitators to PA among university students and rank the relative importance of each TDF domain. It is the first review to apply these frameworks in the context of increasing university students’ participation in PA. Twelve TDF domains across all six sub-components of the COM-B model were identified. The three most important TDF domains were ‘environmental context and resources’, ‘social influences’, and ‘goals’. The most common barriers and facilitators were ‘lack of time’, ‘easily accessible exercise options, facilities and equipment’, ‘exercising with others’, and ‘prioritisation of PA compared to other activities’.

The most common barrier to PA was perceived lack of time. This is consistent with previous findings among university students [ 13 , 74 ] and across other populations [ 24 ], For students, lack of time was frequently attributed to a combination of competing priorities and underdeveloped time management skills. Students predominantly prioritised study over PA, as performing well at university is a valued goal and there is a common perception that spending time exercising (at the expense of study) will impede their academic success [ 53 , 58 ]. Evidence from cognitive neuroscience research, however, suggests that this is a mistaken belief. In addition to its broad physical and mental health benefits, a growing body of evidence demonstrates regular PA can change the structure and function of the brain.

These changes can, in turn, enhance numerous aspects of cognition, including memory, attention, and processing speed [ 4 , 75 , 76 , 77 ], and buffer the negative impact of stress on cognition [ 78 ], all of which are important for academic success. However, students are typically unaware of the brain and cognitive health benefits of PA and its potential to improve academic performance, particularly compared to the physical health benefits [ 37 , 40 , 64 ]. Interventions that position participating in PA as a conduit for helping, rather than hindering, academic goals could increase the relative importance of PA to students and therefore increase their motivation to regularly engage in it. The impact that interventions of this nature have on students’ PA is yet to be empirically assessed.

Ineffective time management also contributed to students’ perceived lack of time for PA. Students reported tendencies to procrastinate in the face of overwhelming academic workloads, which left limited time for PA [ 53 ]. Additionally, students lacked an understanding of how to organise time for PA around academic timetables, social and family responsibilities, co-curricular activities, and employment commitments [ 9 , 44 , 53 , 59 ]. To address these challenges, efforts to develop students’ time management skills will be useful for enabling students to regularly participate in PA. Goal-setting and action planning are two specific examples of such skills that can be integrated into interventions to help students initiate and maintain a PA routine [ 79 ]. For example, goal-setting could involve setting a daily PA goal, and action planning could involve planning to engage in a particular PA at a particular time on certain days.

While the most common determinants of university students’ PA levels were not influenced by specific demographic characteristics, several barriers disproportionately impacted women and students with a disability. These findings are in keeping with evidence that PA is lower among these equity-deserving groups compared with the general population [ 68 , 80 ]. For women, particularly those from Middle Eastern cultures, restrictions were often tied to religious practices and sociocultural norms that limited their opportunities to engage in PA [ 45 , 48 , 66 ]. Additionally, a substantial number of women felt intimidated or self-conscious when exercising in front of others, especially men [ 48 , 49 ]. They also felt that exercise facilities were more often tailored towards the needs of men, leading to a perception that they were unwelcome in exercise communities [ 45 , 48 ]. Consequently, women expressed a desire for women-only spaces to exercise to help them overcome these gender-specific barriers to PA [ 47 , 48 , 66 ]. Furthermore, students with a disability faced physical accessibility barriers and perceived stigmatisation that deterred them from PA [ 50 , 52 ]. The lack of accessible exercise facilities and suitable equipment, programs, and education regarding how to adapt physical activities to accommodate their needs limited their opportunity and ability to participate [ 52 ]. Moreover, students with a disability felt stigmatised by others for not fitting into public perceptions of ‘normality’ or the aesthetic values and beauty standards often portrayed by the fitness industry [ 50 ]. These barriers for both equity-deserving groups of students are deeply rooted in historical stereotypes that have traditionally excluded women and people with a disability from engaging in various types of PA [ 81 , 82 ]. Despite growing awareness of these issues, PA inequalities persist due to narrow sociocultural norms, and a lack of diverse representation and inclusion in the fitness industry and associated marketing campaigns [ 83 , 84 ]. A concerted effort to address PA inequalities across the university sector and fitness industry more broadly is needed. One approach for achieving this is to develop interventions that are tailored to the unique needs of equity-deserving groups, emphasise inclusivity, diversity, and empowerment, and feature women and people with a disability being active.

The “This Girl Can” [ 85 ] and “Everyone Can” [ 86 ] multimedia campaigns are two examples of health behaviour interventions that were co-developed with key stakeholders (i.e., women and people with a disability, respectively) to tackle PA inequalities. The “This Girl Can” campaign has reached over 3 million women and girls, projecting inclusive and positive messages that aim to empower them to be physically active. Following the widespread reach of the “This Girl Can” campaign, the “Everybody Can” campaign was launched to support the inclusion of people with a disability in the PA sector. Although not tailored for university students, these campaigns provide a useful example for developing interventions that are specifically designed to address key barriers preventing women and people with a disability from participating in PA.

Across the tertiary education sector globally, efforts to elevate opportunities and motivation to include PA as a core part of the student experience will be beneficial for promoting students’ PA at scale. Two intervention approaches that can be implemented to facilitate such an endeavour are environmental restructuring and enablement [ 17 ]. These intervention approaches should involve the provision of accessible low-cost exercise options, facilities, and programs, integrating PA into the university curriculum, and mobilising student and staff leadership to encourage students’ participation in PA [ 9 ]. Although there is evidence that these approaches can be effective in promoting sustained PA throughout students’ university years and beyond [ 87 ], implementation measures such as these are complex. Implementation requires aligning student activity levels with broader university goals and is further complicated by having to compete with other funding priorities and resource allocations. Notably, due to the negative impact of the COVID-19 pandemic on university students’ physical and mental health [ 88 , 89 ], the post-pandemic era has seen many universities prioritise enhancing student health and wellbeing alongside more traditional strategic goals like academic excellence and workforce readiness. Despite the potential for PA to be used as a vehicle for supporting these strategic goals there is an absence of data on the extent to which this is occurring in the university sector. The limited evidence in this area suggests that some universities have made efforts to support students’ mental health by referring students who access on-campus counselling services to PA programs [ 90 ]. However, the uptake and efficacy of such initiatives is rarely assessed, and even less is known about whether PA is being used to support other strategic goals, such as academic success. Therefore, while the potential is there for the university sector to use PA to support students’ mental health and academic performance, to be successful this needs to become a strategic university priority. Given that these strategic priorities are set at the senior leadership level, engaging senior university staff in intervention design and promotion efforts is important to enhance the value of PA in the tertiary education sector.

Implications for intervention development

The current findings provide a high-level synthesis of the most common barriers and facilitators to university students’ physical activity. These findings can be leveraged with behavioural intervention development tools and frameworks (e.g., the BCW [ 17 ], Obesity-Related Behavioural Intervention Trials model [ 91 ], Intervention Mapping [ 92 ], and the Medical Research Council guidelines for developing complex interventions [ 93 , 94 ]) to develop evidence-based interventions and policies to promote PA. Given that the TDF and COM-B model are directly linked to the BCW framework, applying this process may be particularly useful to translate the current findings into an intervention.

Additionally, current findings can be triangulated with data directly collected from key stakeholders to assist in the development of context-specific interventions. Best practice principles for developing behavioural interventions recommend this approach to ensure a deep understanding of the barriers and facilitators that need to be targeted to increase the likelihood of behaviour change [ 17 ]. Consulting stakeholders directly (i.e., university students and staff) to understand their perspectives on the barriers and facilitators to students’ PA also enables an intervention to be appropriately tailored to the target population’s needs and implementation setting. Studies continue to demonstrate the effectiveness of this approach, especially when framed within the context of frameworks directly linked to intervention development frameworks, such as the TDF [ 95 ].

Strengths and limitations

The findings of this review should be considered with respect to its methodological strengths and limitations. The credibility and reliability of the research findings are supported by a systematic approach to screening and analysing the empirical data, along with the use of gold-standard behavioural science frameworks to classify barriers and facilitators to PA. The inclusion of qualitative, quantitative, and mixed-methods studies of both barriers and facilitators to students’ PA allowed for a comprehensive understanding of the factors that influence students’ PA that have not previously been captured.

While the present review elucidates students’ own perspectives of the factors that influence their activity levels, other stakeholders such as university staff, will also influence the adoption, operationalisation, and scale of PA interventions in a university setting. It will be important for future research to explore factors that influence university decision-makers in these roles to inform large-scale strategies for promoting students' PA.

Additionally, only one study included in the review used the TDF to explore barriers and facilitators to PA [ 47 ]. Therefore, it is possible that certain TDF domains may not have been identified because students were not asked relevant questions to assess the influence of those domains on their PA. For instance, domains such as ‘memory, attention, and decision process’, and ‘optimism’ are likely to play a role in understanding the barriers and facilitators to PA despite not being identified in this review.

Moreover, quantitative data were only extracted if ≥ 50% of students endorsed the factor as a barrier or facilitator to PA. This threshold was purposefully applied to maintain a focus on the TDF domains most universally relevant to the broad student population in the context of understanding their barriers and facilitators to PA. It is possible that less frequently reported barriers and facilitators, which may not be as prominently featured in the results, could be relevant to specific groups of students, such as those identified as equity-deserving.

Lastly, a quality appraisal of the included studies was not undertaken. This decision was informed by the aim of the review, which was to describe and synthesise the literature to subsequently map data to the TDF and COM-B rather than assess the effectiveness of interventions or determine the strength of evidence. However, this decision, combined with dual screening 25% of the studies and excluding unpublished studies and grey literature, may introduce sources of error and bias, which should be considered when interpreting the results presented.

PA is an effective, scalable, and empowering means of enhancing physical, mental, and cognitive health. This approach could help students reach their academic potential and cope with the many stressors that accompany student life, in addition to setting a strong foundation for healthy exercise habits for a lifetime. As such, understanding the barriers and facilitators to an active student lifestyle is beneficial. This systematic review applied the TDF and COM-B model to identify and map students’ barriers and facilitators to PA and, in doing so, provides a pragmatic, theory-informed, and evidence-based foundation for designing future context-specific PA interventions. The findings from this review highlight the importance of developing PA interventions that focus on the TDF domains ‘environmental context and resources’, ‘social influences’, and ‘goals’, for which intervention approaches could involve environmental restructuring, education, and enablement. If successful, such strategies could make a significant contribution to improving the overall health and academic performance of university students.

Availability of data and materials

The review protocol is available on PROSPERO. The datasets used and/or analysed during the current study and materials used are available from the corresponding author on reasonable request.

The term ‘intervention’ was included to identify student barriers and facilitators to engaging in implemented physical activity interventions.

Physical exercise is defined as “a subset of physical activity that is planned, structured, and repetitive”, and purposefully focused on the improvement or maintenance of physical fitness, whereas physical activity is defined as “any bodily movement produced by skeletal muscles that results in energy expenditure” [ 96 ].

Abbreviations

Behaviour Change Wheel

Capability, Opportunity, Model-Behaviour

  • Physical activity

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

International Prospective Register of Systematic Reviews

Theoretical Domains Framework

Naci H, Ioannidis JPA. Comparative effectiveness of exercise and drug interventions on mortality outcomes: metaepidemiological study. BMJ. 2013;347:f5577.

Article   PubMed   PubMed Central   Google Scholar  

Stensel D, Hardman A, Gill J. Physical Activity and Health: The Evidence Explained. 2021.

Book   Google Scholar  

Stubbs B, Vancampfort D, Hallgren M, Firth J, Veronese N, Solmi M, et al. EPA guidance on physical activity as a treatment for severe mental illness: a meta-review of the evidence and Position Statement from the European Psychiatric Association (EPA), supported by the International Organization of Physical Therapists in Mental Health (IOPTMH). Eur Psychiatry. 2018;54:124–44.

Article   ADS   PubMed   Google Scholar  

Hotting K, Roder B. Beneficial effects of physical exercise on neuroplasticity and cognition. Neurosci Biobehav Rev. 2013;37(9):2243–57.

Article   PubMed   Google Scholar  

World Health Organization. Global action plan on physical activity 2018–2030: more active people for a healthier world. Geneva: World Health Organization; 2018.

Keating XD, Guan J, Piñero JC, Bridges DM. A Meta-Analysis of College Students’ Physical Activity Behaviors. J Am Coll Health. 2005;54(2):116–26.

Worsley JD, Harrison P, Corcoran R. Bridging the Gap: Exploring the Unique Transition From Home, School or College Into University. Frontiers in Public Health. 2021;9. Available from: https://doi.org/10.3389/fpubh.2021.634285

Romaguera D, Tauler P, Bennasar M, Pericas J, Moreno C, Martinez S, et al. Determinants and patterns of physical activity practice among Spanish university students. J Sports Sci. 2011;29(9):989–97.

Deliens T, Deforche B, De Bourdeaudhuij I, Clarys P. Determinants of physical activity and sedentary behaviour in university students: a qualitative study using focus group discussions. BMC Public Health. 2015;15(1):201.

Buckworth J, Nigg C. Physical activity, exercise, and sedentary behavior in college students. J Am Coll Health. 2004;53:28–34.

Nelson MC, Kocos R, Lytle LA, Perry CL. Understanding the perceived determinants of weight-related behaviors in late adolescence: a qualitative analysis among college youth. J Nutr Educ Behav. 2009;41(4):287–92.

Rouse PC, Biddle SJH. An ecological momentary assessment of the physical activity and sedentary behaviour patterns of university students. Health Educ J. 2010;69(1):116–25.

Article   Google Scholar  

Ferreira Silva RM, Mendonça CR, Azevedo VD, Raoof Memon A, Noll P, Noll M. Barriers to high school and university students’ physical activity: A systematic review. Huertas-Delgado FJ, editor. PLoS ONE. 2022;17(4):e0265913.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, et al. A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12(1):77.

Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26.

Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37.

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(1):42.

Bandura A. Social foundations of thought and action: A social cognitive theory. Social foundations of thought and action: A social cognitive theory. 1986. xiii, 617–xiii, 617.

Google Scholar  

Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31(2):143–64.

Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179–211.

Ajzen I. The theory of planned behavior. Handbook Theor Soc Psychol. 2012;1:438–59.

Atkins L, Sallis A, Chadborn T, Shaw K, Schneider A, Hopkins S, et al. Reducing catheter-associated urinary tract infections: a systematic review of barriers and facilitators and strategic behavioural analysis of interventions. Implement Sci. 2020;15:1–22.

Chung OS, Dowling NL, Brown C, Robinson T, Johnson AM, Ng CH, et al. Using the theoretical domains framework to inform the implementation of therapeutic virtual reality into mental healthcare. Administration and Policy in Mental Health and Mental Health Services Research. 2023;50(2):237–68.

Weatherson KA, McKay R, Gainforth HL, Jung ME. Barriers and facilitators to the implementation of a school-based physical activity policy in Canada: application of the theoretical domains framework. BMC Public Health. 2017;17(1):835.

Taylor N, Lawton R, Slater B, Foy R. The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013;8(1):123.

Garne-Dalgaard A, Mann S, Bredahl TVG, Stochkendahl MJ. Implementation strategies, and barriers and facilitators for implementation of physical activity at work: a scoping review. Chiropractic Manual Therapies. 2019;27(1):48.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

Fritz T, Huang E, Murphy G, Zimmermann T. Persuasive technology in the real world: a study of long-term use of activity sensing devices for fitness. In: CHI '14 proceedings of the SIGCHI conference on human factors in computing systems: April 26–may 01, 2014; Toronto. Ontario: ACM; 2014. p. 487–96.

Tacconelli E. Systematic reviews: CRD’s guidance for undertaking reviews in health care. Lancet Infect Dis. 2010;10(4):226.

The EndNote Team. EndNote. Philadelphia, PA: Clarivate; 2013.

O’Mahony B, Kerins C, Murrin C, Kelly C. Barriers and facilitators to the implementation of nutrition standards for school food: a mixed methods systematic review protocol. HRB Open Res. 2020;3:20.

Stuart G, D’Lima D. Perceived barriers and facilitators to attendance for cervical cancer screening in EU member states: a systematic review and synthesis using the Theoretical Domains Framework. Psychol Health. 2022;37(3):279–330.

Bouma SE, van Beek JFE, Diercks RL, van der Woude LHV, Stevens M, van den Akker-Scheek I. Barriers and facilitators perceived by healthcare professionals for implementing lifestyle interventions in patients with osteoarthritis: a scoping review. BMJ Open. 2022;12(2):e056831–e056831.

Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide to Designing Interventions. London: Silverback Publishing; 2014. Available from: www.behaviourchangewheel.com .

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Diehl K, Fuchs AK, Rathmann K, Hilger-Kolb J. Students’ Motivation for Sport Activity and Participation in University Sports: a Mixed-Methods Study. Biomed Res Int. 2018;2018:1–7.

Bellows-Riecken K, Mark R, Rhodes RE. Qualitative elicitation of affective beliefs related to physical activity. Psychol Sport Exerc. 2013;14(5):786–92.

Ramírez-Vélez R. Prevalencia De Barreras Para La Práctica De Actividad Física En. Nutr Hosp. 2015;2:858–65.

Lerner J, Burns C, De Róiste Á. Correlates of physical activity among college students. Recreational Sports Journal. 2011;35(2):95–106.

Snyder K, Lee JM, Bjornsen A, Dinkel D. What gets them moving? College students’ motivation for exercise: an exploratory study. Recreational Sports Journal. 2017;41(2):111–24.

Walsh A, Taylor C, Brennick D. Factors that influence campus dwelling university students’ facility to practice healthy living guidelines. Can J Nurs Res. 2018;50(2):57–63.

Forrest CK, Bruner MW. Evaluating social media as a platform for delivering a team-building exercise intervention: a pilot study. Int J Sport Exercise Psychol. 2017;15(2):190–206.

Ranasinghe C, Sigera C, Ranasinghe P, Jayawardena R, Ranasinghe ACR, Hills AP, et al. Physical inactivity among physiotherapy undergraduates: exploring the knowledge-practice gap. BMC Sports Sci Med Rehabil. 2016;8(1):39.

Othman MS, Mat Ludin AF, Chen LL, Hossain H, Abdul Halim II, Sameeha MJ, et al. Motivations, barriers and exercise preferences among female undergraduates: a need assessment analysis. Muazu Musa R, editor. PLoS ONE. 2022;17(2):e0264158.

Leinberger-Jabari A, Al-Ajlouni Y, Ieriti M, Cannie S, Mladenovic M, Ali R. Assessing motivators and barriers to active healthy living among a multicultural college student body: a qualitative inquiry. J Am Coll Health. 2023;71(2):338–42.

Burton NW, Barber BL, Khan A. A qualitative study of barriers and enablers of physical activity among female Emirati university students. IJERPH. 2021;18(7):3380.

silver mp, easty lk, sewell km, georges r, behman a. perspectives on exercise participation among canadian university students. health educ j. 2019;78(7):851–65.

lacaille lj, dauner kn, krambeer rj, pedersen j. psychosocial and environmental determinants of eating behaviors, physical activity, and weight change among college students: a qualitative analysis. J Am Coll Health. 2011;59(6):531–8.

Monforte J, Úbeda-Colomer J, Pans M, Pérez-Samaniego V, Devís-Devís J. Environmental barriers and facilitators to physical activity among university students with physical disability—a qualitative study in Spain. IJERPH. 2021;18(2):464.

Brunette MK, Lariviere M, Schinke RJ, Xing X, Pickard P. Fit to belong: activity and acculturation of Chinese students. J Sport Behav. 2011;34(3):207.

Devine MA. Leisure-time physical activity: experiences of college students with disabilities. Adapt Phys Activ Q. 2016;33(2):176–94.

Article   MathSciNet   PubMed   Google Scholar  

Kwan MYW, Faulkner GEJ. Perceptions and barriers to physical activity during the transition to university. 2011.

Pan J, Nigg C. Motivation for physical activity among hawaiian, japanese, and filipino university students in Hawaii. J Appl Sport Psychol. 2011;23(1):1–15.

Wilson OWA, Walters SR, Naylor ME, Clarke JC. University students’ negotiation of physical activity and sport participation constraints. Recreational Sports Journal. 2019;43(2):84–92.

Tong HL, Coiera E, Laranjo L. Using a mobile social networking app to promote physical activity: a qualitative study of users’ perspectives. J Med Internet Res. 2018;20(12):e11439.

Marmo J. Applying SOCIAL COGNITIVE THEORY TO DEVELOP TARGETED MESSAGES: COLLEGE STUDENTS AND PHYSICAL ACTIVITY. West J Commun. 2013;77(4):444–65.

Hilger-Kolb J, Loerbroks A, Diehl K. ‘When I have time pressure, sport is the first thing that is cancelled’: a mixed-methods study on barriers to physical activity among university students in Germany. J Sports Sci. 2020;38(21):2479–88.

Nannyonjo J, Nsibambi C, Goon D, Amusa L. Physical activity patterns of female students of Kyambogo University, Uganda. African Journal for Physical, Health Education, Recreation and Dance. 2013;19(4:1):865–73.

El Gilany AH, Badawi K, El Khawaga G, Awadalla N. Physical activity profile of students in Mansoura University. Egypt East Mediterr Health J. 2011;17(08):694–702.

Goldstein SP, Forman EM, Butryn ML, Herbert JD. Differential programming needs of college students preferring web-based versus in-person physical activity programs. Health Commun. 2018;33(12):1509–15.

Griffiths K, Moore R, Brunton J. Sport and physical activity habits, behaviours and barriers to participation in university students: an exploration by socio-economic group. Sport Educ Soc. 2022;27(3):332–46.

King KA, Vidourek RA, English L, Merianos AL. Vigorous physical activity among college students: using the health belief model to assess involvement and social support. AEHD. 2014;4(2):267–79.

Miyawaki C, Ohara K, Mase T, Kouda K, Fujitani T, Momoi K, et al. The purpose and the motivation for future practice of physical activity and related factors in Japanese university students. jhse. 2019;14(1). Available from: http://hdl.handle.net/10045/77873 .

Pellerine LP, Bray NW, Fowles JR, Furlano JA, Morava A, Nagpal TS, et al. The influence of motivators and barriers to exercise on attaining physical activity and sedentary time guidelines among Canadian undergraduate students. IJERPH. 2022;19(19):12225.

Aljayyousi GF, Abu Munshar M, Al-Salim F, Osman ER. Addressing context to understand physical activity among Muslim university students: the role of gender, family, and culture. BMC Public Health. 2019;19(1):1452.

Nolan VT, Sandada M, Surujlal J. Perceived benefits and barriers to physical exercise participation of first year university students. 2011.

Chaabna K, Mamtani R, Abraham A, Maisonneuve P, Lowenfels AB, Cheema S. Physical activity and Its barriers and facilitators among university students in Qatar: a cross-sectional study. IJERPH. 2022;19(12):7369.

Awadalla NJ, Aboelyazed AE, Hassanein MA, Khalil SN, Aftab R, Gaballa II, et al. Assessment of physical inactivity and perceived barriers to physical activity among health college students, south-western Saudi Arabia. East Mediterr Health J. 2014;20(10):596–604.

Article   CAS   PubMed   Google Scholar  

Cooke PA, Tully MA, Cupples ME, Gilliland AE, Gormley GJ. A randomised control trial of experiential learning to promote physical activity. Educ Prim Care. 2013;24(6):427–35.

Musaiger AO, Al-Kandari FI, Al-Mannai M, Al-Faraj AM, Bouriki FA, Shehab FS, et al. Perceived barriers to weight maintenance among university students in Kuwait: the role of gender and obesity. Environ Health Prev Med. 2014;19(3):207–14.

Quintiliani LM, Bishop HL, Greaney ML, Whiteley JA. Factors across home, work, and school domains influence nutrition and physical activity behaviors of nontraditional college students. Nutr Res. 2012;32(10):757–63.

Von Sommoggy J, Rueter J, Curbach J, Helten J, Tittlbach S, Loss J. How does the campus environment influence everyday physical activity? A photovoice study among students of two German universities. Front Public Health. 2020;8:561175.

Pan M, Ying B, Lai Y, Kuan G. Status and Influencing Factors of Physical Exercise among College Students in China: A Systematic Review. Int J Environ Res Public Health. 2022;19(20):13465.

Den Ouden L, Kandola A, Suo C, Hendrikse J, Costa RJS, Watt MJ, et al. The influence of aerobic exercise on hippocampal integrity and function: preliminary findings of a multi-modal imaging analysis. Brain Plast. 2018;4(2):211–6.

Maleki S, Hendrikse J, Chye Y, Caeyenberghs K, Coxon JP, Oldham S, et al. Associations of cardiorespiratory fitness and exercise with brain white matter in healthy adults: a systematic review and meta-analysis. Brain Imaging Behav. 2022;16(5):2402–25.

Müller P, Duderstadt Y, Lessmann V, Müller NG. Lactate and BDNF: key mediators of exercise induced neuroplasticity? J Clin Med. 2020;9(4):1136.

Nicastro TM, Greenwood BN. Central monoaminergic systems are a site of convergence of signals conveying the experience of exercise to brain circuits involved in cognition and emotional behavior. Curr Zool. 2016;62(3):293–306.

Fleig L, Pomp S, Parschau L, Barz M, Lange D, Schwarzer R, et al. From intentions via planning and behavior to physical exercise habits. Psychol Sport Exerc. 2013;14(5):632–9.

Carroll DD, Courtney-Long EA, Stevens AC, Sloan ML, Lullo C, Visser SN, et al. Vital signs: disability and physical activity—United States, 2009–2012. Morb Mortal Wkly Rep. 2014;63(18):407.

Ferez S. From women’s exclusion to gender institution: a brief history of the sexual categorisation process within sport. Int J History Sport. 2012;29(2):272–85.

Ferez S, Ruffié S, Joncheray H, Marcellini A, Pappous S, Richard R. Inclusion through sport: a critical view on paralympic legacy from a historical perspective. Social Inclusion. 2020;8(3):224–35.

Rasmussen K, Dufur MJ, Cope MR, Pierce H. Gender marginalization in sports participation through advertising: the case of Nike. Int J Environ Res Public Health. 2021;18(15):7759.

Richardson EV, Smith B, Papathomas A. Disability and the gym: Experiences, barriers and facilitators of gym use for individuals with physical disabilities. Disabil Rehabil. 2017;39(19):1950–7.

Sport England. This Girl Can | This Girl Can | This girl can. 2015 [cited 2023 Aug 21]. Available from: https://www.thisgirlcan.co.uk/ .

ukactive. Everyone Can. 2017 [cited 2023 Aug 21]. Available from: https://everyonecan.ukactive.com/ .

Kim M, Cardinal BJ. A Review of How Physical Activity Education Policies in Higher Education Affect College Students’ Physical Activity Behavior and Motivation. In 2016. Available from: https://api.semanticscholar.org/CorpusID:210543192 .

Ro A, Rodriguez VE, Enriquez LE. Physical and mental health impacts of the COVID-19 pandemic among college students who are undocumented or have undocumented parents. BMC Public Health. 2021;21(1):1580.

Wang X, Hegde S, Son C, Keller B, Smith A, Sasangohar F. Investigating mental health of US college students during the COVID-19 pandemic: cross-sectional survey study. J Med Internet Res. 2020;22(9):e22817.

Brunton JA, Mackintosh CI. Interpreting university sport policy in England: seeking a purpose in turbulent times? Int J Sport Policy Politics. 2017;9(3):377–95.

Czajkowski SM, Powell LH, Adler N, Naar-King S, Reynolds KD, Hunter CM, et al. From ideas to efficacy: the ORBIT model for developing behavioral treatments for chronic diseases. Health Psychol. 2015;34(10):971.

Eldredge LKB, Markham CM, Ruiter RA, Fernández ME, Kok G, Parcel GS. Planning health promotion programmes: an intervention mapping approach. 4th ed. San Francisco (USA): Wiley; 2016.

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337(7676):979–83.

Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.

Ndupu LB, Staples V, Lipka S, Faghy M, Bessadet N, Bussell C. Application of theoretical domains framework to explore the enablers and barriers to physical activity among university staff and students: a qualitative study. BMC Public Health. 2023;23(1):670–670.

Caspersen CJ, Powell KE, Christenson GM. Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep. 1985;100(2):126–31.

CAS   PubMed   PubMed Central   Google Scholar  

Download references

Acknowledgements

The authors extend their gratitude to the funder, the nib foundation, for its financial support, which was instrumental in facilitating this research. We are also indebted to the Wilson Foundation and the David Winston Turner Endowment Fund for their generous philanthropic contributions, which have supported the BrainPark research team and facility where this research was conducted. Special thanks are owed to the library staff at Monash University for their expertise in conducting systematic reviews, which helped inform the selection of databases and the development of the search strategy.

This research was supported by nib foundation. The nib foundation had no role in the design of the study and collection, analysis, and interpretation of data, and in writing the manuscript. The views expressed are those of the authors and not necessarily those of the nib foundation.

Author information

Murat Yücel and Rebecca A. Segrave share senior authorship.

Authors and Affiliations

BrainPark, Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia

Catherine E. B. Brown, Karyn Richardson, Bengianni Halil-Pizzirani & Rebecca A. Segrave

Centre for Behaviour Change, University College London, London, UK

QIMR Berghofer Medical Research Institute, Herston, Brisbane, QLD, Australia

Murat Yücel

You can also search for this author in PubMed   Google Scholar

Contributions

CB, KR, BP, LA and RS developed the review protocol. CB and BP conducted the search and screened articles, and KR resolved conflicts. CB, KR, BP, LA and RS extracted the barriers and facilitators, mapped barriers and facilitators to the TDF and COM-B model, and interpreted the results. CB drafted the paper. All authors read, revised, and approved the submitted version.

Corresponding author

Correspondence to Catherine E. B. Brown .

Ethics declarations

Ethics approval and consent to participate.

The current review contained no participants and did not require ethical approval.

Consent for publication

No individual person’s data in any form is contained in the current article.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. .

Theoretical Domains Framework domains, definitions, and constructs.

Additional file 2. 

Search syntax for Ovid MEDLINE.

Additional file 3. 

Summary of study characteristics.

Additional file 4. 

PRISMA Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Brown, C.E.B., Richardson, K., Halil-Pizzirani, B. et al. Key influences on university students’ physical activity: a systematic review using the Theoretical Domains Framework and the COM-B model of human behaviour. BMC Public Health 24 , 418 (2024). https://doi.org/10.1186/s12889-023-17621-4

Download citation

Received : 26 August 2023

Accepted : 30 December 2023

Published : 09 February 2024

DOI : https://doi.org/10.1186/s12889-023-17621-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • University students
  • Physical exercise
  • Behaviour change
  • Theoretical domains framework
  • Facilitators

BMC Public Health

ISSN: 1471-2458

methods of literature search

  • Open access
  • Published: 12 February 2024

The negative pressure wound therapy for prevention of sternal wound infection: Can we reduce infection rate after the use of bilateral internal thoracic arteries? A systematic literature review and meta-analysis

  • Hind Elhassan 1 ,
  • Ridha Amjad 2 ,
  • Unna Palaniappan 2 ,
  • Mahmoud Loubani 1 &
  • David Rose 3  

Journal of Cardiothoracic Surgery volume  19 , Article number:  87 ( 2024 ) Cite this article

215 Accesses

Metrics details

Negative pressure wound therapy (NPWT) is traditionally used to treat postoperative wound infections. However, its use in closed wound sternotomy post cardiac surgery in high-risk patients has become increasingly popular. The potential preventive benefit of reducing sternal wound infections has been recently acknowledged. Bilateral internal mammary artery (BIMA) grafts are used in coronary artery bypass grafting but have been associated with an increased risk of sternal wound infections (SWIs).

This systematic analysis examines whether NPWT can reduce the incidence of SWI following BIMA grafts, leading to more patients benefiting from the better survival outcome associated with BIMA grafting.

A comprehensive systematic search and meta-analysis were performed to identify studies on the use of NPWT in closed wound sternotomy. Ovid MEDLINE (in-process and other nonindexed citations and Ovid MEDLINE 1990 to present), Ovid EMBASE (1990 to present), and The Cochrane Library (Wiley), PubMed, and Google Scholar databases were searched from their inception to May 2022 using keywords and MeSH terms. Thirty-four articles from 1991 to May 2022 were selected.

Three studies reported on the outcome of NPWT following BIMA grafting. The pooled analysis did not show any significant difference in the incidence of sternal wound infection between NPWT and standard dressing (RR 0.48 95% CI 0.17–1.37; P  = 0.17) with substantial heterogeneity (I 2  65%). Another seven studies were found comparing the outcome of SWI incidence of negative pressure closed wound therapy with conventional wound therapy in patients undergoing adult cardiac surgery. The pooled analysis showed that NPWT was associated with a low risk of SWIs compared to conventional dressing (RR 0.47 95% CI 0.36–0.59; P  < 0.00001), with low heterogeneity (I 2  1%).

The literature identified that NPWT significantly decreased the incidence of sternal wound complications when applied to sutured sternotomy incisions in high-risk patients, and in some cases, it eliminated the risk. However, the inadequate number of randomized controlled trials assessing the effectiveness of NPWT in BIMA grafting emphasizes the need for further, robust studies.

Peer Review reports

Introduction

Sternal wound infections (SWIs) can cause extensive complications following median sternotomy in cardiac operations. Studies have shown that the incidence of SWI after cardiac surgery ranges from 0.9% to 20%. 1.6% of these patients develop deep sternal wound infections (DSWIs), which can affect muscle tissue, sternum, sub sternum, and mediastinum [ 1 , 2 , 3 ]. According to Perezgrovas-Olaria et al. [ 1 ] on their recent meta-analysis deep sternal wound infection found to be associated with higher mortality, longer postoperative hospitalization, stroke, myocardial infarction and respiratory and renal failure. Superficial sternal wound infection, which involves the skin, subcutaneous tissue, and deep fascia [ 3 ]. Coronary artery bypass grafting (CABG) patients are susceptible to developing DSWIs and have been found to have a 2.5 times greater long-term mortality than those without any infection [ 4 ]. The literature has described numerous risk factors for SWIs [ 5 ]. The use of the bilateral internal mammary artery (BIMA) for coronary artery grafting is the most consistently reported independent risk factor for SWI, together with insulin-dependent diabetes and obesity BIMA use alone can increase the risk by 4.23 times [ 1 , 5 , 6 , 7 , 8 , 9 , 10 ]. This has resulted in the use of BIMA in CABG, despite better long-term prognosis, which has been restricted to low-risk patients [ 1 , 6 , 11 ].

Traditionally, the use of NPWT has been restricted to treating wound complications with open post-sternotomy incisions [ 2 , 11 , 12 , 13 ]. Further investigations into the use of NPWT have promoted its applicability as a prophylactic measure to reduce the incidence of SWI [ 2 , 5 ]. Applying negative pressure to closed incisions promptly after surgery can prevent complications such as SWIs in high-risk individuals [ 2 , 11 ].

NPWT arose in 1997 and has significantly benefited the outcomes of sternal wound dehiscence (SWD). The suction of excess tissue fluid facilitates the prevention of haematoma or seroma formation [ 14 ]. Negative pressure stimulates perfusion, resulting in an accelerated healing process in addition to a reduction in ischaemic wound necrosis, thereby preventing wound breakdown (Fig.  1 ) and helping in primary wound healing, especially in watershed regions [ 15 ].

figure 1

Illustration of mechanism through which NPWT prevent wound infection

If NPWT is proven effective, it can promote the practice of BIMA grafting, allowing long-term prognostic benefits [ 9 , 10 ]. Therefore, this systematic literature review aims to examine the literature on the efficiency of NPWT in reducing the risk of SWI following BIMA grafts.

A systematic literature review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was performed.

Search strategy

A comprehensive systematic search was performed to identify studies that evaluated the usefulness of NPWT in preventing the risk of SWI post cardiac surgery. Ovid MEDLINE (in-process and other nonindexed citations and Ovid MEDLINE 1990 to present), Ovid EMBASE (1990 to present), and The Cochrane Library (Wiley), PubMed, and Google Scholar databases were searched from their inception to May 2022 using key words and the MeSH terms ‘cardiac surgery’, ‘negative pressure wound therapy’, ‘closed incision management’, ‘prevention’, ‘bilateral internal mammary arteries’, ‘BIMA’, and ‘sternal wound infection’.

Study selection and inclusion criteria

Inclusion criteria were full text English studies where NPWT was used as prevention rather than as a treatment of postsurgical infections. Outcome measurements of sternal wound complications after application of the NPWT system in which BIMA had been used in coronary artery grafting. Studies that measure the outcome of NPWT in other cardiac surgeries have also been investigated. The reference lists were also searched for any relevant articles that met the criteria.

Data extraction

Two authors independently extracted data (Elhassan H, Amjad R). Quality assessment was carried out by Elhassan (H) using the Newcastle‒Ottawa Scale (NOS) to assess the quality of nonrandomized studies in the meta-analysis (attached in supplementary material) [ 16 ].

Statistical analysis

The primary endpoint was the incidence of sternal wound infection in NPWT compared to conventional therapy in patients who had bilateral mammary grafting. The secondary endpoint was the incidence of sternal wound infection in NPWT compared to conventional treatment in the general cardiac population. Categorical variables are expressed as percentages, and continuous variables are expressed as the mean and standard deviations. Random-effects models were used to consider pooled effects, heterogeneity and absolute values. The risk ratio (RR) with a 95% confidence interval (CI) was extracted from the incidence of SWIs. Data are summarized in tables and forest plots. Forest plots were used to determine the effect size. The I 2  test was used to evaluate heterogeneity (unimportant ( I 2  = 0–40%), moderate ( I 2  = 40–60%), substantial ( I 2  = 60–75%), or considerable ( I 2  > 75–100%). Statistical analysis was performed using RevMan (Review Manager (RevMan) [Computer program]. Version 5.4. The Cochrane Collaboration, 2020).

From 38 abstracts, 17 studies met the inclusion criteria for full-text review, out of which 15 studies were selected for systematic review and 10 studies for meta-analysis (Fig.  2 ).

figure 2

Flow diagram showing literature search methods for the systematic review

Six out of the 15 studies addressed the outcome of NPWT in patients who underwent BIMA use for grafting. Nearly all the studies have reported favourable clinical outcomes apart from one that showed no difference in SWI incidence between NPWT and conventional therapy. We separated the studies’ results into two groups: studies addressing the outcome of NPWT following the use of BIMA for grafting and the remaining studies that investigated the outcome of NPWT in all other cardiac procedures without direct emphasis on BIMA in their analysis.

We identified a total of 7,996 patients, of whom 2,514 received NPWT post cardiac surgery on a closed incision; the included studies are summarized in Table  1 .

Only 285 patients who had BIMA received NPWT, and the included studies are summarized in Tables  1 and 2 .

Primary endpoint: outcome of NPWT following BIMA grafting

A total of six studies reported on the outcome of NPWT following BIMA grafting. Three studies investigated BIMA grafting as their primary endpoint, while the remaining studies had a subgroup analysis of patients with BIMA grafts from their study group. Three studies compared this outcome with the control group receiving conventional treatment. The quality of these studies was poor, and only one study used propensity-score matching. The pooled analysis did not show a significant difference between NPWT and standard dressing (RR 0.48 95% CI 0.17–1.37; P  = 0.17), with substantial heterogeneity (I 2  65%) Fig.  3 .

figure 3

Sternal wound infection following bilateral mammary artery grafting

Outcome of SWIs following NPWT in cardiac surgery

Seven further studies were found in the literature comparing the outcome of SWI incidence with negative pressure closed wound therapy with conventional wound therapy in patients undergoing adult cardiac surgery. We also examined these studies to explore the effect of NPWT in reducing the incidence of sternal wound infection in other high-risk cardiac patients.

Meta-analysis was conducted for all the studies with a comparison group (a total of 10 studies, including studies in the BIMA group). Two studies were randomized [ 19 , 25 ], and one used propensity-score matching [ 9 ]. The pooled analysis showed that NPWT was associated with a low risk of SWIs compared to conventional dressing (RR 0.47 95% CI = 0.36–0.59; P  < 0.00001), with low heterogeneity (I 2  6%) (Fig.  4 ).

figure 4

Sternal wound infection following general cardiac operations

We have run further subgroup analysis to investigate if NPWT have a preventive benefit on superficial and deep sternal wound infection (Figs.  5 , 6 ).

figure 5

Superficial sternal wound infection (SSWI) post cardiac surgery

figure 6

Deep sternal wound infection (DSWI) post cardiac surgery

Three studies were not included in the analysis of SSWI, as either did not report the number of SSWI separately in their studies [ 18 , 20 ] or the study only focus on DSWI [ 25 ]. NPWT remains to have law risk of SSWI in comparsion with conventional dressing (RR 0.56 95% CI 0.41, 0.77; P  = 0.0004) with unimportant heterogenicity I 2  = 28%).

One study did not provide breakdown on number of patients with DSWI [ 18 ], and one study had no deep sternal wound infection in both arms of the study [ 19 ]. The RR 0.35 95% CI 0.23, 0.55; P  < 0.00001 and no heterogenicity.

Fifteen studies met the inclusion criteria and were included in this review. Nine studies were included in the meta-analysis. The majority of the studies (12 articles) demonstrated a reduction in sternal wound complications in high-risk cardiac surgery associated with the administration of negative pressure on sutured incisions. Two studies reported no significant difference in the rate of SWI between patients receiving NPWT and those receiving conventional treatment. Six studies highlighted the outcome of NPWT within BIMA patients, whether it was part of a subgroup analysis or the primary treatment. Four studies reported positive patient outcomes associated with NPWT, while two articles with control patient groups showed no significant difference. However, pooled analysis was nonsignificant (RR 0.48 95% CI 0.17–1.37; P  = 0.17), with substantial heterogeneity (I 2  65%). However, the general trend was the beneficial effect of NPWT across the different studies.

Atkins, Colli, Gatti, Reddy and Philip [ 5 , 8 , 21 , 22 ] all demonstrated the potency of NPWT post-surgery, as patient reports were entirely clear of infection after prophylactic management. However, the absence of control groups and the small sample size makes it difficult to obtain significant results to draw a clear conclusion and limits the generalizability of these trials to the wider population. As these studies were not randomized controlled studies, it may be argued that the promising results discovered are not the sole by product of the negative pressure wound therapy system; other factors might be involved [ 11 , 17 ]. The literature has shown that female sex increases the risk of postoperative complications [ 5 ]. Despite this, Atkins and colleagues failed to include sufficient females in their selected participants, reinforcing the reduced infection rate [ 5 ]. Thus, trials that consider the relative contributions of other risk factors in postoperative infection, with sufficient patients, are needed to fully understand the protective effect of negative pressure wound therapy on closed chest incisions.

BIMA use has been associated with increased graft patency and positive long-term clinical outcomes in many studies and literature review [ 28 , 29 ]. Although this is the case, there has been some reluctance to use BIMA for coronary grafting, given the risk of sternal wound infection associated with its use. The Arterial Revascularisation Trial (ART), a multicentre, randomized controlled trial, did not show superiority of BIMA grafting compared to single internal mammary artery (SIMA) grafting with vein grafts after 10 years of follow-up [ 30 ]. However, this finding contradicted the largeevidence in the literature. Multiple factors may contribute to this finding: first, the study design was unblinded, surgeons’ experience was not taken into consideration, there was a lack of angiographic studies to assess graft patency, and there was nonadherence to randomization. The SIMA group received radial artery grafting as well, which is known to have a better outcome than vein graft [ 31 ]. Several studies reported higher survival rates and less reintervention compared to SIMA. Tatoulis et al. [ 29 ] conducted a large observational study evaluating the patency of the right internal mammary artery (RIMA) by performing coronary angiography for 991 patients out of 5,766 patients who underwent BIMA grafting in their institution. The ten-year overall patency was 90% for grafts to the left anterior descending artery (LAD) 95%, graft to the circumflex artery (Cx) 91%, graft to the right coronary artery (RCA) 84% and graft to the posterior descending artery (PDA) 86%. This was identical to the 10-year patency as LIMA to LAD and LIMA to Cx. They also demonstrate better RIMA patency than radial artery and vein grafts. There was an 89% survival rate for patients who had BIMA grafting. A meta-analysis of 8 propensity-matched studies by Gaudino et al. [ 28 ] showed that BIMA grafting is associated with superior long-term survival irrespective of sex and diabetes. Davierwala et al. [ 32 ] recently published their outcome of using BIMA in minimally invasive off-pump coronary artery grafts; early outcomes showed 96.8% graft patency on angiographic studies prior to discharge.

Research evidence has shown that using IMA grafts (or ITA grafts) correlates with an elevated risk of SWI, especially when using BIMA [ 6 ]. Research by Grossi et al. [ 33 ] demonstrated that having one or more IMA grafts increased the prevalence of sternal infection by 1.37% compared to patients who had no IMA graft in CABG surgery. A BIMA graft was associated with a 1.7% higher prevalence of SWI than a single IMA graft (SIMA), and the difference in prevalence increased to 8.8% when combined with diabetes. TheART-reported incidence of sternal wound complications was almost double in the BIMA group than in the SIMA group (54 cases vs. 30 cases) after a 10-year follow-up, and there was a significantly increased rate of sternal reconstruction in the BIMA group (31 cases vs. 10 cases) [ 30 ]. Ståhle et al. [ 34 ] investigated sternal wound problems after cardiac surgery in a large cohort of 13,285 patients over fifteen years. Of these, 155 patients had BIMA grafting, with a sternal wound infection rate of 3.9%; this was more than 2 times the rate in other cases (1.7%) [ 34 ]. The results from a prospective study using patients from ten surgical institutions in Paris (n = 1830) showed that the incidence of DSWI in BIMA grafting patients was 6% higher than that in SIMA patients (8.7% vs. 2.7%) [ 35 ]. The robust association between BIMA use and the incidence of SWIs has been determined by large randomized clinical trials, with infection rates ranging from 2.1 to 8.7% [ 8 , 10 , 33 , 34 , 35 , 36 , 37 ]. As a result, it is critical to acknowledge other risk factors (e.g., obesity, diabetes) when using bilateral internal mammary arteries so that strategies to adjust for these risk factors can be implemented in vulnerable patients before surgery [ 38 ]. The use of negative pressure wound therapy might be particularly beneficial in this patient category.

The NPWT system uses special dressings and a negative pressure-creating device to provide a proportional distribution of negative pressure over closed surgical sites [ 2 , 5 , 11 ]. The wound site and surrounding skin are coveredwith an adherent sterile cover, and a vacuum pump is joined to the dressing by a suction tube [ 11 ]. This gives negative pressures ranging from − 75 to − 125 mmHg, enabling wound fluid to be drained into an aseptic canister, as in the case of Prevena™ dressing [ 2 , 5 , 7 , 11 , 17 ]. There are several mechanisms through which NPWT devices exert their effects. It has been suggested that introducing a closed wound medium keeps the sealed incision margins composed, stimulates cells to proliferate and triggers angiogenesis [ 2 , 5 , 7 , 11 , 13 , 39 ]. The extracellular fluid is drained, allowing the removal of exudative content and tissue oedema. Blood flow to the wound area is increased, thus enhancing tissue perfusion and improving the circulation of immune cells and antibiotics [ 2 , 7 , 11 , 13 , 39 ]. These factors prevent the progression of infection and the incidence of sternal wound complications by deterring colony-forming bacteria and increasing the production of granulation tissue [ 2 , 7 , 11 , 13 , 39 ]. There have been few NPWT systems on the market, with Prevena™ currently being the most researched system [ 7 , 9 , 13 , 17 , 18 , 19 , 20 , 21 , 22 , 27 ]. Witt-Majchrzak et al. [ 19 ] reported in their study the NPWT has reduced the risk of overall sternal wound complications including sternal wound instabilities and abnormal healing.

In this meta-analysis, NPWT was shown to have a preventive benefit for wound infection post BIMA use in patients who underwent coronary artery bypass graft and in all other adult cardiac surgeries.

The pooled analysis for studies in general adult cardiac surgery showed a significant reduction in wound infection in patients who received negative pressure wound therapy compared to those who received conventional wound dressing (RR 0.47 95% CI 0.36–0.59; P  < 0.00001), with low heterogeneity (I 2  6).

Negative pressure-generating interventions usually involve greater costs than conventional treatment [ 9 ]. Research by Tabley et al. showed that the total cost of buying PICO™ systems and the reduced costs of treating complications approximately saved £1188.79 “per patient” [ 24 , 40 ]. Concerning the broader employment of NPWT in cardiac surgery, it is crucial to consider the economic effects of preventing SWIs versus treating the added complications [ 9 , 23 ]. If additional analyses of the saved expenses are produced in the literature, they can further support the worthiness of NPWT in surgery.

Study limitations

The majority of the studies were observational studies, apart from one randomized control trial and one propensity-score matching analysis to account for selection bias. The lack of robust studies affects the quality of the review due to literature biases. The small number of studies (n = 3) focusing specifically on the efficacy of prophylactic NPWT after BIMA grafting included in the final analyses (with high heterogeneity) makes it difficult to conclude the suitability of NPWT. We did not acknowledge the presence of treatment allocation bias. The use of NPWT is usually reserved for patients who are perceived to have a high surgical risk of developing SWI. Nonetheless, pooled analysis from studies covering all cardiac procedures showed a potential benefit of NPWT in preventing SWI. Finally, there was a language bias, as we only included studies in English.

Based on findings from this systematic literature review and meta-analysis, closed incision management after cardiac surgery can reduce the incidence of sternal wound infections in cardiac patients. However, the potential usefulness of NPWT after bilateral internal mammary artery grafting needs to be further investigated. The inconsistent study findings in these patients and the flawed study designs leave the question regarding the clinical efficacy of prophylactic use of closed incisional negative pressure therapy in BIMA grafting not conclusively answered. Large-scale, randomized controlled studies that specifically measure postsurgical outcomes in BIMA patients following negative pressure wound therapy are needed.

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article (and its Additional file 1 ).

Abbreviations

Arterial revascularization trial

  • Bilateral internal mammary artery
  • Coronary artery bypass grafting

Circumflex artery

Deep Sternal Wound Infection

Internal mammary artery

Internal thoracic artery

Left anterior descending artery

Left internal mammary artery

Newcastle‒Ottawa Scale

Negative Pressure Wound Therapy

Posterior descending artery

Right coronary artery

Right internal mammary artery

Superficial sternal wound infection

Sternal Wound Infections

Sternal wound dehiscence

Systematic Reviews and Meta-Analyses

Single internal mammary artery

Perezgrovas-Olaria R, Audisio K, Cancelli G, Rahouma M, Ibrahim M, Soletti GJ, Chadow D, Demetres M, Girardi LN, Gaudino M. Deep sternal wound infection and mortality in cardiac surgery: a meta-analysis. Ann Thorac Surg. 2023;115(1):272–80. https://doi.org/10.1016/j.athoracsur.2022.04.054 .

Article   PubMed   Google Scholar  

Dohmen PM, Markou T, Ingemansson R, Rotering H, Hartman JM, van Valen R, et al. Use of incisional negative pressure wound therapy on closed median sternal incisions after cardiothoracic surgery: clinical evidence and consensus recommendations. Med Sci Monit. 2014;20:1814–25.

Article   PubMed   PubMed Central   Google Scholar  

Song Y, Chu W, Sun J, et al. Review on risk factors, classification, and treatment of sternal wound infection. J Cardiothorac Surg. 2023;18:184. https://doi.org/10.1186/s13019-023-02228-y .

Toumpoulis IK, Anagnostopoulos CE, DeRose JJ, Swistel DG. The impact of deep sternal wound infection on long-term survival after coronary artery bypass grafting. Chest. 2005;127(2):464–71.

Atkins BZ, Wooten MK, Kistler J, Hurley K, Hughes CG, Wolfe WG. Does negative pressure wound therapy have a role in preventing poststernotomy wound complications? Surg Innov. 2009;16(2):140–6.

Bayer N, Hart WM, Arulampalam T, Hamilton C, Schmoeckel M. Is the use of BIMA in CABG sub-optimal? A review of the current clinical and economic evidence including innovative approaches to the management of mediastinitis. Ann Thorac Cardiovasc Surg. 2020;26(5):229–39.

Gatti G, Ledwon M, Gazdag L, Cuomo F, Pappalardo A, Fischlein T, et al. Management of closed sternal incision after bilateral internal thoracic artery grafting with a single-use negative pressure system. Updates Surg. 2018;70(4):545–52.

Gatti G, Benussi B, Brunetti D, Ceschia A, Porcari A, Biondi F, et al. The fate of patients having deep sternal infection after bilateral internal thoracic artery grafting in the negative pressure wound therapy era. Int J Cardiol. 2018;269:67–74.

Ruggieri VG, Olivier ME, Aludaat C, Rosato S, Marticho P, Saade YA, et al. Negative pressure versus conventional sternal wound dressing in coronary surgery using bilateral internal mammary artery grafts. Heart Surg Forum. 2019;22(2):E92–6.

Article   Google Scholar  

Borger MA, Rao V, Weisel RD, Ivanov J, Cohen G, Scully HE, et al. Deep sternal wound infection: risk factors and outcomes. Ann Thorac Surg. 1998;65(4):1050–6.

Article   PubMed   CAS   Google Scholar  

Scalise A, Calamita R, Tartaglione C, Pierangeli M, Bolletta E, Gioacchini M, et al. Improving wound healing and preventing surgical site complications of closed surgical incisions: a possible role of Incisional Negative Pressure Wound Therapy. A systematic review of the literature. Int Wound J. 2016;13(6):1260–81.

Huang C, Leavitt T, Bayer LR, Orgill DP. Effect of negative pressure wound therapy on wound healing. Curr Probl Surg. 2014;51(7):301–31.

Grauhan O, Navasardyan A, Hofmann M, Müller P, Stein J, Hetzer R. Prevention of poststernotomy wound infections in obese patients by negative pressure wound therapy. J Thorac Cardiovasc Surg. 2013;145(5):1387–92.

Dohmen PM, Misfeld M, Borger MA, Mohr FW. Closed incision management with negative pressure wound therapy. Expert Rev Med Devices. 2014;11(4):395–402.

Atkins BZ, Tetterton JK, Petersen RP, Hurley K, Wolfe WG. Laser Doppler flowmetry assessment of peristernal perfusion after cardiac surgery: beneficial effect of negative pressure therapy. Int Wound J. 2011;8(1):56–62.

Ottawa Hospital Research Institute. [cited 2022 Oct 24]. Available from: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp

Colli A. First experience with a new negative pressure incision management system on surgical incisions after cardiac surgery in high risk patients. J Cardiothorac Surg. 2011;6:1.

Grauhan O, Navasardyan A, Tutkun B, Hennig F, Müller P, Hummel M, et al. Effect of surgical incision management on wound infections in a poststernotomy patient population. Int Wound J. 2014;11(Suppl 1):6–9.

Witt-Majchrzak A, Zelazny P, Snarska J. Preliminary outcome of treatment of postoperative primarily closed sternotomy wounds treated using negative pressure wound therapy. Pol Prz Chir Polish J Surg. 2015;86(10):456–65.

Google Scholar  

A Retrospective Study to Evaluate Use of Negative Pressure Wound Therapy in Patients Undergoing Bilateral Internal Thoracic Artery Grafting.

Reddy VS. Use of closed incision management with negative pressure therapy for complex cardiac patients. Cureus. 2016;8:2.

Philip B, McCluskey P, Hinchion J. Experience using closed incision negative pressure wound therapy in sternotomy patients. J Wound Care. 2017;26(8):491–5.

Suelo-Calanao RL, Thomson R, Read M, Matheson E, Isaac E, Chaudhry M, et al. The impact of closed incision negative pressure therapy on prevention of median sternotomy infection for high risk cases: A single centre retrospective study. J Cardiothorac Surg. 2020;15(1):1–9.

Tabley A, Aludaat C, le Guillou V, Gay A, Nafeh-Bizet C, Scherrer V, et al. A survey of cardiac surgery infections with PICO negative pressure therapy in high-risk patients. Ann Thorac Surg. 2020;110(6):2034–40.

Rashed A, Csiszar M, Beledi A, Gombocz K. The impact of incisional negative pressure wound therapy on the wound healing process after midline sternotomy. Int Wound J. 2021;18(1):95.

Brega C, Calvi S, Albertini A. Use of a negative pressure wound therapy system over closed incisions option in preventing poststernotomy wound complications. Wound Repair Regener. 2021;29(5):848–52.

Nguyen KA, Taylor GA, Webster TK, Jenkins RA, Houston NS, Kahler DL, et al. Incisional negative pressure wound therapy is protective against postoperative cardiothoracic wound infection. Ann Plast Surg. 2022;88(3 Suppl 3):S197-200.

Gaudino M, Puskas JD, di Franco A, Ohmes LB, Iannaccone M, Barbero U, et al. Three arterial grafts improve late survival: a meta-analysis of propensity-matched studies. Circulation. 2017;135(11):1036–44.

Tatoulis J, Buxton BF, Fuller JA. The right internal thoracic artery: the forgotten conduit5,766 patients and 991 angiograms. Ann Thorac Surg. 2011;92(1):9–17.

Taggart DP, Benedetto U, Gerry S, Altman DG, Gray AM, Lees B, et al. Bilateral versus single internal-thoracic-artery grafts at 10 years. N Engl J Med. 2019;380(5):437–46.

Gaudino M, Benedetto U, Fremes S, Biondi-Zoccai G, Sedrakyan A, Puskas JD, et al. Radial-artery or saphenous-vein grafts in coronary-artery bypass surgery. N Engl J Med. 2018;378(22):2069–77.

Davierwala PM, Verevkin A, Sgouropoulou S, Hasheminejad E, von Aspern K, Misfeld M, et al. Minimally invasive coronary bypass surgery with bilateral internal thoracic arteries: early outcomes and angiographic patency. J Thorac Cardiovasc Surg. 2021;162(4):1109-1119.e4.

Grossi EA, Esposito R, Harris LJ, Crooke GA, Galloway AC, Colvin SB, et al. Sternal wound infections and use of internal mammary artery grafts. J Thorac Cardiovasc Surg. 1991;102:342–7.

Ståhle E, Tammelin A, Bergström R, Hambreus A, Nyström SO, Hansson HE. Sternal wound complications–incidence, microbiology and risk factors. Eur J Cardiothorac Surg. 1997;11(6):1146–53.

Taggart DP, Altman DG, Gray AM, Lees B, Gerry S, Benedetto U, et al. Randomized trial of bilateral versus single internal-thoracic-artery grafts. N Engl J Med. 2016;375(26):2540–9.

Gatti G, Dell’Angela L, Barbati G, Benussi B, Forti G, Gabrielli M, et al. A predictive scoring system for deep sternal wound infection after bilateral internal thoracic artery grafting. Eur J Cardiothorac Surg. 2016;49(3):910–7.

Lemaignen A, Birgand G, Ghodhbane W, Alkhoder S, Lolom I, Belorgey S, et al. Sternal wound infection after cardiac surgery: incidence and risk factors according to clinical presentation. Clin Microbiol Infect. 2015;21(7):674.e11-674.e18.

Lu JCY, Grayson AD, Jha P, Srinivasan AK, Fabri BM. Risk factors for sternal wound infection and mid-term survival following coronary artery bypass surgery. Eur J Cardiothorac Surg. 2003;23(6):943–9.

Strugala V, Martin R. Meta-analysis of comparative trials evaluating a prophylactic single-use negative pressure wound therapy system for the prevention of surgical site complications. Surg Infect (Larchmt). 2017;18(7):810–9.

New study demonstrates PICO ◊ Single Use Negative Pressure Wound Therapy System (sNPWT) beats standard dressings in reducing surgical site complications resulting in fewer deep sternal wound infections (DSWI) following cardiac surgery | Smith & Nephew. Available from: https://www.smith-nephew.com/news-and-media/media-releases/news/new-study-demonstrates-pico-single-use-negative-pressure-wound-therapy-system-snpwt-beats-standard-dressings-in-reducing-surgical-site-complications-resulting-in-fewer-deep-sternal-wound-infections-dswi-following-cardiac-surgery

Download references

No funding was provided to produce this work.

Author information

Authors and affiliations.

Cardiothoracic Department, Hull University Teaching Hospital NHS Foundation Trust, Castle Rd, Cottingham, East Riding of Yorkshire, HU16 5JQ, UK

Hind Elhassan & Mahmoud Loubani

Lancaster Medical School, Lancaster University, Lancaster, UK

Ridha Amjad & Unna Palaniappan

Cardiothoracic Department, Lancashire Cardiac Centre, Blackpool Teaching Hospital NHS Foundation Trust, Blackpool, UK

You can also search for this author in PubMed   Google Scholar

Contributions

HE: design, data extraction, quality assessment, meta-analysis, interpretation of finding and drafting. RA: design, data extraction, interpretation of finding and drafting. UP: design graft. ML: revising and final approval. DR: conceptualization, revising and final approval.

Corresponding author

Correspondence to Hind Elhassan .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

HE: No competing interests to declare. RA: No competing interests to declare. UP: No competing interests to declare. ML: Educational and research honoraria from 3M. DR: No competing interests to declare.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. .

Newcastle‒Ottawa Scale Qauality Assessment for included articles.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Elhassan, H., Amjad, R., Palaniappan, U. et al. The negative pressure wound therapy for prevention of sternal wound infection: Can we reduce infection rate after the use of bilateral internal thoracic arteries? A systematic literature review and meta-analysis. J Cardiothorac Surg 19 , 87 (2024). https://doi.org/10.1186/s13019-024-02589-y

Download citation

Received : 05 August 2023

Accepted : 30 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1186/s13019-024-02589-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Negative pressure wound therapy
  • Sternal wound infections

Journal of Cardiothoracic Surgery

ISSN: 1749-8090

methods of literature search

U.S. flag

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

  • formulating the research question(s) and objective(s),
  • searching the extant literature,
  • screening for inclusion,
  • assessing the quality of primary studies,
  • extracting data, and
  • analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

  • Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
  • Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
  • Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
  • Analyzing data using quantitative or qualitative methods.
  • Presenting results in summary of findings tables.
  • Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Table 9.1. Typology of Literature Reviews (adapted from Paré et al., 2015).

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

  • Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
  • Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
  • Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
  • Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
  • A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
  • Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
  • Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
  • Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
  • Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
  • Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
  • Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
  • Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
  • Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
  • Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
  • Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
  • Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
  • Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
  • Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
  • Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
  • Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
  • Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
  • Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
  • Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
  • Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
  • Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
  • King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
  • Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
  • Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
  • Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
  • Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
  • Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
  • Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
  • Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
  • Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
  • Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
  • Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
  • Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
  • Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
  • Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
  • Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
  • Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
  • Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
  • Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
  • Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
  • Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
  • Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
  • Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
  • Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
  • Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
  • Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
  • Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
  • Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
  • Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
  • Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
  • Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
  • Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
  • Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
  • Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
  • Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
  • Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

  • Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
  • PDF version of this title (4.5M)
  • Disable Glossary Links

In this Page

  • Introduction
  • Overview of the Literature Review Process and Steps
  • Types of Review Articles and Brief Illustrations
  • Concluding Remarks

Related information

  • PMC PubMed Central citations
  • PubMed Links to PubMed

Recent Activity

  • Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

statistics

IMAGES

  1. Flow diagram of the literature search process.

    methods of literature search

  2. Literature search methods

    methods of literature search

  3. Methods of literature search.

    methods of literature search

  4. Flow chart showing methods of literature search.

    methods of literature search

  5. PPT

    methods of literature search

  6. Flow chart of methods of literature search.

    methods of literature search

VIDEO

  1. Literature Review

  2. Review of literature

  3. Approaches to searching the literature

  4. Effective Review of Literature

  5. Literature review and its process

  6. What is literature??

COMMENTS

  1. How to carry out a literature search for a systematic review: a

    Literature reviews are conducted for the purpose of (a) locating information on a topic or identifying gaps in the literature for areas of future study, (b) synthesising conclusions in an area of ambiguity and (c) helping clinicians and researchers inform decision-making and practice guidelines.

  2. A systematic approach to searching: an efficient and complete method to

    A systematic approach to searching an efficient and complete method to develop literature searches is a crucial skill for researchers and practitioners. This article presents a comprehensive and practical guide to design and conduct effective searches in various databases, using techniques such as truncation, Boolean operators, and PICOT framework.

  3. Methods for Literature Search

    Literature Review Methods Based on input from our expert advisors, our conceptual model, and practical considerations, we developed literature review methods that included: inclusion and exclusion criteria to identify potentially relevant articles, search strategies to retrieve articles, abstract review protocols, and a system of scoring ...

  4. How to undertake a literature search: a step-by-step guide

    Undertaking a literature search can be a daunting prospect. Breaking the exercise down into smaller steps will make the process more manageable. This article suggests 10 steps that will help readers complete this task, from identifying key concepts to choosing databases for the search and saving the …

  5. Literature search for research planning and identification of research

    Go to: Abstract Literature search is a key step in performing good authentic research. It helps in formulating a research question and planning the study. The available published data are enormous; therefore, choosing the appropriate articles relevant to your study in question is an art.

  6. Defining the process to literature searching in systematic reviews: a

    Method A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks.

  7. Rapid reviews methods series: Guidance on literature search

    Introduction. This paper is part of a series from the Cochrane Rapid Reviews Methods Group (RRMG) providing methodological guidance for rapid reviews (RRs).1-3 While the RRMG's guidance4 5 on Cochrane RR production includes brief advice on literature searching, we aim to provide in-depth recommendations for the entire search process. Literature searching is the foundation for all reviews ...

  8. How to Write a Literature Review

    Step 1 - Search for relevant literature Step 2 - Evaluate and select sources Step 3 - Identify themes, debates, and gaps Step 4 - Outline your literature review's structure Step 5 - Write your literature review Free lecture slides Other interesting articles Frequently asked questions Introduction Quick Run-through Step 1 & 2 Step 3 Step 4 Step 5

  9. Unpacking your literature search toolbox: on search styles and tactics

    While such tactics are now prominent in systematic review methods they are rightly viewed as supplementary, and not alternative to, a rigorous and systematic subject search using building block techniques. ... It is hoped that by documenting the 20 years plus legacy of literature on search styles and tactics this feature makes the first ...

  10. Methodological Approaches to Literature Review

    A literature review is defined as "a critical analysis of a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles." (The Writing Center University of Winconsin-Madison 2022) A literature review is an integrated analysis, not just a summary of scholarly work on a specific topic.

  11. PDF Doing a literature search: a step by step guide

    A good literature search entails systematically working through steps 1-5: 1, Background reading and preparation 2, Working with your title - identifying search terms 3, Identifying the resources to search 4, Searching - using search techniques 5, Collating your results Let's take a look at these 5 stages in detail:

  12. 3. Methods for Searching the Literature

    Methods for Searching the Literature 1. Introduction 2. Why Do a Literature Review? 3. Methods for Searching the Literature 4. Analysing the Literature 5. Organizing the Literature Review 6. Writing the Review 1. Tasks Involved in a Literature Review There are two major tasks involved in a literature review: Identifying and selecting literature

  13. Types of literature review, methods, & resources

    (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

  14. LibGuides: Best Practice for Literature Searching: What is literature

    Literature searching is the task of finding relevant information on a topic from the available research literature. Literature searches range from short fact-finding missions to comprehensive and lengthy funded systematic reviews. Or, you may want to establish through a literature review that no one has already done the research you are conducting.

  15. Systematic Reviews: Step 3: Conduct Literature Searches

    Search using other methods (e.g. hand searching) Validate and peer review the search; Document the search; ... When developing a literature search, a good place to start is to identify searchable concepts of the research question, and then expand by adding other terms to describe those concepts. Read below for more information and examples on ...

  16. Literature review as a research methodology: An ...

    This is why the literature review as a research method is more relevant than ever. Traditional literature reviews often lack thoroughness and rigor and are conducted ad hoc, rather than following a specific methodology. ... the limitations of the search method, and the quality of the search process, and they often lack details on how the ...

  17. Defining the process to literature searching in systematic reviews: a

    Method A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks.

  18. Method for conducting systematic literature review and meta-analysis

    Method details: the six basic steps Protocol - SLR methodology step 1. The need for a research protocol for SLR is for the consideration of transparency, transferability, and replicability of the work, which are the characteristics that make a literature review systematic [12].This helps to minimize the bias by conducting exhaustive literature searches.

  19. Search strategy formulation for systematic reviews: Issues, challenges

    This is a complex process (Cooper et al., 2018; Lefebvre et al., 2020), in which the search methods and choice of databases to be used to identify literature for the systematic review are specified and peer reviewed. It is recommended that the protocol is approved by peer review before searching is carried out i.e., that the process to identify ...

  20. How to Conduct a Literature Search

    There are several steps involved in conducting a literature search. You may discover more along the way, but these steps will provide a good foundation. Plan using PICO (T) to develop your clinical question and formulate a search strategy. Identify a database to search. Conduct your search in one or more databases. Select relevant articles.

  21. Types of reviews

    Learn more about the method: Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach. ... Main purpose is to map out and categorize existing literature, identify gaps in literature; Search comprehensiveness determined by time/scope constraints, could take longer than ...

  22. 'It depends': what 86 systematic reviews tell us about what strategies

    Methods. We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos.

  23. Key influences on university students' physical activity: a systematic

    Search strategy. Search terms and parameters were developed in collaboration with a Monash University librarian with expertise in systematic review methodology. The following databases were searched on 15.03.2023 to identify relevant literature: MEDLINE, PsycINFO, and SPORTDiscus. Key articles were also selected for citation searching via Scopus.

  24. Literature searching methods or guidance and their application to

    Keywords: bibliographic databases, database searching, grey literature, information sources, information storage and retrieval, knowledge synthesis, literature searching, public health, supplementary searching, web sites Key messages

  25. The negative pressure wound therapy for prevention of sternal wound

    A systematic literature review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was performed. Search strategy. A comprehensive systematic search was performed to identify studies that evaluated the usefulness of NPWT in preventing the risk of SWI post cardiac surgery.

  26. Literature searching methods or guidance and their application to

    Methods. A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed. Results. Seventy-two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature.

  27. Diagnostics

    Through a comprehensive literature search, 71 articles meeting the inclusion criteria were included in the qualitative analysis, investigating the hardware, software, and operational aspects of these systems. ... 3D surface-imaging technologies offer more detailed information and zero ionizing radiation than conventional imaging methods .

  28. Chapter 9 Methods for Literature Reviews

    Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generat...