Initiating a review

The development of systematic literature reviews and the surrounding methodology has given rise to established recommendations and requirements for reporting the review methodology. There are many guides for this, such as the SBU manual, a comprehensive guide in Swedish. Also, Cochrane and Campbell collaboration have published handbooks and guidelines for systematic literature reviews.

The research question

The research inquiry and its delimitation must, for a systematic review, be sufficiently focused and well-structured. The research question forms the basis for the inclusion and exclusion criteria of the search and defines the overall search strategy or search query. Make sure to consider the methodological implications of the kind of data to be dealt with in the review.

To exemplify a systematic search in this guide, we will proceed from the following research question:

How effective is the use of a bicycle helmet to protect you from facial damage?

Search protocol

A protocol for a systematic literature review describes the hypothesis or research question, planned method of approach, and the inclusion and exclusion criteria for the review. The protocol should be designed beforehand and be used as a guide for carrying out the review according to predefined parameters. This is recommended because it promotes transparency and reproducibility and reduces the risk of bias and duplication. Look at the evidence synthesis protocol template, which combines guidelines from PRISMA and Campbell collaboration.

In research disciplines where systematic reviews are well-established, it is possible to register the protocol in specific databases such as Prospero. Organizations that perform systematic reviews often register their publications on their own websites.

Choosing databases

Which database/s you choose to search is crucial to the result. To be relevant at all, the database must cover the research area in question. In view of the research question mentioned above, we have searched medical and interdisciplinary databases, but also technical databases and patent databases may contain information on bicycle helmets and their protective features.

In medicine, there are two equally large and relevant databases called PubMed and Embase. The latter is not as frequently used because it comes with a fee and requires special agreements which are not always in place. Linköping university nevertheless provides access to Embase.

Even if you search the same database, such as Medline, different user interfaces may cause different conditions of possibility to search and hence a variety of search results. Content, indexing, and search features look different if a search in PubMed is compared to a search in Medline via database hosts such as Ovid and EBSCO.

Subject coverage

There are differences in the subject coverage between different databases. On the one hand, a smaller, subject-specific database may be better in covering the subject in question than a large interdisciplinary database. On the other hand, it may fall short of studies in the outskirts of a research area, or which is interdisciplinary in character.

Databases may also differ as regards the timespan covered, which geographical and linguistic areas they represent, and which requirements are made as to quality of studies published via the database. If you want to know exactly which publications (journals and other serial publications) are included in a database, there is usually an index available in the help section of the database or on its website.

Besides the fact that a database must have a large degree of subject coverage within the field in question, it is crucial that its search interface is suitable for systematic searches. It must allow for searching different fields, using Boolean search operators like AND, OR, and NOT between blocks within parentheses as well as making it possible to search for phrases.

Libraries nowadays usually provide tools for searching simultaneously in many, but not all, of the resources of the university. It encompasses established databases, journal archives, monographs, and open materials made accessible through the producers of search tools.

Reproducibility

Also, a search must be possible to repeat or reproduce, with the same result each time. Unless this is possible, the different search results must be explainable with regard to changes in the database content. Based on such necessary and desirable criteria, Michael Gusenbauer and Neal R. Haddaway (2019) evaluated 28 academic databases and found significant differences in how suitable they were for systematic searches.

As to systematic searches, it is deemed problematic how the results of simultaneous searches vary over time. You cannot repeat the search with the same result. It means you must compromise as regards the specifically developed features of searching available in subject-specific databases and their built-in user interface. Nevertheless, searching several databases at the same time may provide an overview of what has been published in an area as well as suggestions as to which databases and journals to further search. You can also use them to find grey literature.