literature review on science and technology

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Credit: Getty

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

WENTING ZHAO: Be focused and avoid jargon

Assistant professor of chemical and biomedical engineering, Nanyang Technological University, Singapore.

When I was a research student, review writing improved my understanding of the history of my field. I also learnt about unmet challenges in the field that triggered ideas.

For example, while writing my first review 1 as a PhD student, I was frustrated by how poorly we understood how cells actively sense, interact with and adapt to nanoparticles used in drug delivery. This experience motivated me to study how the surface properties of nanoparticles can be modified to enhance biological sensing. When I transitioned to my postdoctoral research, this question led me to discover the role of cell-membrane curvature, which led to publications and my current research focus. I wouldn’t have started in this area without writing that review.

Collection: Careers toolkit

A common problem for students writing their first reviews is being overly ambitious. When I wrote mine, I imagined producing a comprehensive summary of every single type of nanomaterial used in biological applications. It ended up becoming a colossal piece of work, with too many papers discussed and without a clear way to categorize them. We published the work in the end, but decided to limit the discussion strictly to nanoparticles for biological sensing, rather than covering how different nanomaterials are used in biology.

My advice to students is to accept that a review is unlike a textbook: it should offer a more focused discussion, and it’s OK to skip some topics so that you do not distract your readers. Students should also consider editorial deadlines, especially for invited reviews: make sure that the review’s scope is not so extensive that it delays the writing.

A good review should also avoid jargon and explain the basic concepts for someone who is new to the field. Although I trained as an engineer, I’m interested in biology, and my research is about developing nanomaterials to manipulate proteins at the cell membrane and how this can affect ageing and cancer. As an ‘outsider’, the reviews that I find most useful for these biological topics are those that speak to me in accessible scientific language.

Bozhi Tian likes to get a variety of perspectives into a review. Credit: Aleksander Prominski

BOZHI TIAN: Have a process and develop your style

Associate professor of chemistry, University of Chicago, Illinois.

In my lab, we start by asking: what is the purpose of this review? My reasons for writing one can include the chance to contribute insights to the scientific community and identify opportunities for my research. I also see review writing as a way to train early-career researchers in soft skills such as project management and leadership. This is especially true for lead authors, because they will learn to work with their co-authors to integrate the various sections into a piece with smooth transitions and no overlaps.

After we have identified the need and purpose of a review article, I will form a team from the researchers in my lab. I try to include students with different areas of expertise, because it is useful to get a variety of perspectives. For example, in the review ‘An atlas of nano-enabled neural interfaces’ 2 , we had authors with backgrounds in biophysics, neuroengineering, neurobiology and materials sciences focusing on different sections of the review.

After this, I will discuss an outline with my team. We go through multiple iterations to make sure that we have scanned the literature sufficiently and do not repeat discussions that have appeared in other reviews. It is also important that the outline is not decided by me alone: students often have fresh ideas that they can bring to the table. Once this is done, we proceed with the writing.

I often remind my students to imagine themselves as ‘artists of science’ and encourage them to develop how they write and present information. Adding more words isn’t always the best way: for example, I enjoy using tables to summarize research progress and suggest future research trajectories. I’ve also considered including short videos in our review papers to highlight key aspects of the work. I think this can increase readership and accessibility because these videos can be easily shared on social-media platforms.

ANKITA ANIRBAN: Timeliness and figures make a huge difference

Editor, Nature Reviews Physics .

One of my roles as a journal editor is to evaluate proposals for reviews. The best proposals are timely and clearly explain why readers should pay attention to the proposed topic.

It is not enough for a review to be a summary of the latest growth in the literature: the most interesting reviews instead provide a discussion about disagreements in the field.

Careers Collection: Publishing

Scientists often centre the story of their primary research papers around their figures — but when it comes to reviews, figures often take a secondary role. In my opinion, review figures are more important than most people think. One of my favourite review-style articles 3 presents a plot bringing together data from multiple research papers (many of which directly contradict each other). This is then used to identify broad trends and suggest underlying mechanisms that could explain all of the different conclusions.

An important role of a review article is to introduce researchers to a field. For this, schematic figures can be useful to illustrate the science being discussed, in much the same way as the first slide of a talk should. That is why, at Nature Reviews, we have in-house illustrators to assist authors. However, simplicity is key, and even without support from professional illustrators, researchers can still make use of many free drawing tools to enhance the value of their review figures.

Yoojin Choi recommends that researchers be open to critiques when writing reviews. Credit: Yoojin Choi

YOOJIN CHOI: Stay updated and be open to suggestions

Research assistant professor, Korea Advanced Institute of Science and Technology, Daejeon.

I started writing the review ‘Biosynthesis of inorganic nanomaterials using microbial cells and bacteriophages’ 4 as a PhD student in 2018. It took me one year to write the first draft because I was working on the review alongside my PhD research and mostly on my own, with support from my adviser. It took a further year to complete the processes of peer review, revision and publication. During this time, many new papers and even competing reviews were published. To provide the most up-to-date and original review, I had to stay abreast of the literature. In my case, I made use of Google Scholar, which I set to send me daily updates of relevant literature based on key words.

Through my review-writing process, I also learnt to be more open to critiques to enhance the value and increase the readership of my work. Initially, my review was focused only on using microbial cells such as bacteria to produce nanomaterials, which was the subject of my PhD research. Bacteria such as these are known as biofactories: that is, organisms that produce biological material which can be modified to produce useful materials, such as magnetic nanoparticles for drug-delivery purposes.

Synchronized editing: the future of collaborative writing

However, when the first peer-review report came back, all three reviewers suggested expanding the review to cover another type of biofactory: bacteriophages. These are essentially viruses that infect bacteria, and they can also produce nanomaterials.

The feedback eventually led me to include a discussion of the differences between the various biofactories (bacteriophages, bacteria, fungi and microalgae) and their advantages and disadvantages. This turned out to be a great addition because it made the review more comprehensive.

Writing the review also led me to an idea about using nanomaterial-modified microorganisms to produce chemicals, which I’m still researching now.

PAULA MARTIN-GONZALEZ: Make good use of technology

PhD student, University of Cambridge, UK.

Just before the coronavirus lockdown, my PhD adviser and I decided to write a literature review discussing the integration of medical imaging with genomics to improve ovarian cancer management.

As I was researching the review, I noticed a trend in which some papers were consistently being cited by many other papers in the field. It was clear to me that those papers must be important, but as a new member of the field of integrated cancer biology, it was difficult to immediately find and read all of these ‘seminal papers’.

That was when I decided to code a small application to make my literature research more efficient. Using my code, users can enter a query, such as ‘ovarian cancer, computer tomography, radiomics’, and the application searches for all relevant literature archived in databases such as PubMed that feature these key words.

The code then identifies the relevant papers and creates a citation graph of all the references cited in the results of the search. The software highlights papers that have many citation relationships with other papers in the search, and could therefore be called seminal papers.

My code has substantially improved how I organize papers and has informed me of key publications and discoveries in my research field: something that would have taken more time and experience in the field otherwise. After I shared my code on GitHub, I received feedback that it can be daunting for researchers who are not used to coding. Consequently, I am hoping to build a more user-friendly interface in a form of a web page, akin to PubMed or Google Scholar, where users can simply input their queries to generate citation graphs.

Tools and techniques

Most reference managers on the market offer similar capabilities when it comes to providing a Microsoft Word plug-in and producing different citation styles. But depending on your working preferences, some might be more suitable than others.

Reference managers

Here is a comparison of the more popular collaborative writing tools, but there are other options, including Fidus Writer, Manuscript.io, Authorea and Stencila.

Collaborative writing tools

*Markdown and LaTex are code-based formatting languages favoured by physicists, mathematicians and computer scientists who code on a regular basis, and less popular in other disciplines such as biology and chemistry.

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

Tales of a migratory marine biologist

Career Feature 28 AUG 24

Nail your tech-industry interviews with these six techniques

Career Column 28 AUG 24

How to harness AI’s potential in research — responsibly and ethically

Career Feature 23 AUG 24

Binning out-of-date chemicals? Somebody think about the carbon!

Correspondence 27 AUG 24

No more hunting for replication studies: crowdsourced database makes them easy to find

Nature Index 27 AUG 24

Partners in drug discovery: how to collaborate with non-governmental organizations

Exclusive: the papers that most heavily cite retracted studies

News 28 AUG 24

Chain retraction: how to stop bad science propagating through the literature

Comment 28 AUG 24

Global Faculty Recruitment of School of Life Sciences, Tsinghua University

The School of Life Sciences at Tsinghua University invites applications for tenure-track or tenured faculty positions at all ranks (Assistant/Ass...

Beijing, China

Tsinghua University (The School of Life Sciences)

Tenure-Track/Tenured Faculty Positions

Tenure-Track/Tenured Faculty Positions in the fields of energy and resources.

Suzhou, Jiangsu, China

School of Sustainable Energy and Resources at Nanjing University

ATLAS - Joint PhD Program from BioNTech and TRON with a focus on translational medicine

5 PhD positions for ATLAS, the joint PhD Program from BioNTech and TRON with a focus on translational medicine.

Mainz, Rheinland-Pfalz (DE)

Translational Oncology (TRON) Mainz

Alzheimer's Disease (AD) Researcher/Associate Researcher

Xiaoliang Sunney XIE’s Group is recruiting researchers specializing in Alzheimer's disease (AD).

Changping Laboratory

Supervisory Bioinformatics Specialist CTG Program Head

The National Library of Medicine (NLM) is a global leader in biomedical informatics and computational health data science and the world’s largest b...

Bethesda, Maryland (US)

National Library of Medicine, National Center for Biotechnology Information

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

To read this content please select one of the options below:

Please note you do not have access to teaching notes, trends in science and technology research: literature review.

World Journal of Science, Technology and Sustainable Development

ISSN : 2042-5945

Article publication date: 1 March 2013

The paper aims to explore and identify the contributions to the literature available about trends in science and technology research at various levels.

Design/methodology/approach

A deep scan of literature was carried out in an attempt to identify considerable works that have been published concerning various facets related to science and technology research. Varied search terms like “research”, “research and research output”, “science and technology research”, “research collaboration”, “research in universities”, “importance of science and technology research”, “issues in research”, etc. were used for retrieving the literature from a range of online scholarly databases, search engines and allied web sources.

The literature review reveals that a considerable amount of literature has been published related to science and technology research. However, keeping in view immense advancements and innovations in science and technology, scholarly output is still in its emergent phase.

Practical implications

It is apparent from the study of existing literature that there is still vast scope for advanced exploration on the topic and the study paves the way for the concerned organizations and institutions (like universities, libraries and publishers) at national and international level to take substantial measures to boost research in the field of science and technology.

Originality/value

The paper is the first ordered and makes an endeavour to review the literature and provides a summary of emerging trends in science and technology research.

• Science and technology research
• Collaborative research
• University research
• Research issues
• Research work
• Research methods

Akhtar Khan, N. , Jan, S. and Amin, I. (2013), "Trends in science and technology research: literature review", World Journal of Science, Technology and Sustainable Development , Vol. 10 No. 3, pp. 168-178. https://doi.org/10.1108/WJSTSD-04-2013-0021

Emerald Group Publishing Limited

Copyright © 2013, Emerald Group Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 29, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Workshop: Literature Reviews- What you need to know

• How to write a literature review

Useful Titles for All of your Boxes

The Seven Steps to Producing a Literature Review:

1. Identify your question

2. Review discipline style

3. Search the literature

4. Manage your references

5. Critically analyze and evaluate

6. Synthisize

7. Write the review

• University of North Carolina Writing Center "How To" UNC Chapel Hill Writing Center "How To" on writing a literature review.
• Purdue Owl Purdue Owl help on writing a literature review

Feedback and Additional Information

• Library Workshop Evaluation Brief survey for workshop participants. Your feedback is greatly appreciated.
• TTU Library Workshops Additional information about TTU Library Workshops. All workshops are taught by Personal Librarians in Instruction Lab 150 unless otherwise noted.
• Last Updated: Aug 27, 2024 8:06 AM
• URL: https://guides.library.ttu.edu/litreview

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

How-to conduct a systematic literature review: A quick guide for computer science research

Angela carrera-rivera.

a Faculty of Engineering, Mondragon University

William Ochoa

Felix larrinaga.

b Design Innovation Center(DBZ), Mondragon University

Associated Data

• No data was used for the research described in the article.

Performing a literature review is a critical first step in research to understanding the state-of-the-art and identifying gaps and challenges in the field. A systematic literature review is a method which sets out a series of steps to methodically organize the review. In this paper, we present a guide designed for researchers and in particular early-stage researchers in the computer-science field. The contribution of the article is the following:

• • Clearly defined strategies to follow for a systematic literature review in computer science research, and
• • Algorithmic method to tackle a systematic literature review.

Graphical abstract

Specifications table

Method details

A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12] . An SLR updates the reader with current literature about a subject [6] . The goal is to review critical points of current knowledge on a topic about research questions to suggest areas for further examination [5] . Defining an “Initial Idea” or interest in a subject to be studied is the first step before starting the SLR. An early search of the relevant literature can help determine whether the topic is too broad to adequately cover in the time frame and whether it is necessary to narrow the focus. Reading some articles can assist in setting the direction for a formal review., and formulating a potential research question (e.g., how is semantics involved in Industry 4.0?) can further facilitate this process. Once the focus has been established, an SLR can be undertaken to find more specific studies related to the variables in this question. Although there are multiple approaches for performing an SLR ( [5] , [26] , [27] ), this work aims to provide a step-by-step and practical guide while citing useful examples for computer-science research. The methodology presented in this paper comprises two main phases: “Planning” described in section 2, and “Conducting” described in section 3, following the depiction of the graphical abstract.

Defining the protocol is the first step of an SLR since it describes the procedures involved in the review and acts as a log of the activities to be performed. Obtaining opinions from peers while developing the protocol, is encouraged to ensure the review's consistency and validity, and helps identify when modifications are necessary [20] . One final goal of the protocol is to ensure the replicability of the review.

Define PICOC and synonyms

The PICOC (Population, Intervention, Comparison, Outcome, and Context) criteria break down the SLR's objectives into searchable keywords and help formulate research questions [ 27 ]. PICOC is widely used in the medical and social sciences fields to encourage researchers to consider the components of the research questions [14] . Kitchenham & Charters [6] compiled the list of PICOC elements and their corresponding terms in computer science, as presented in Table 1 , which includes keywords derived from the PICOC elements. From that point on, it is essential to think of synonyms or “alike” terms that later can be used for building queries in the selected digital libraries. For instance, the keyword “context awareness” can also be linked to “context-aware”.

Planning Step 1 “Defining PICOC keywords and synonyms”.

Formulate research questions

Clearly defined research question(s) are the key elements which set the focus for study identification and data extraction [21] . These questions are formulated based on the PICOC criteria as presented in the example in Table 2 (PICOC keywords are underlined).

Research questions examples.

Select digital library sources

The validity of a study will depend on the proper selection of a database since it must adequately cover the area under investigation [19] . The Web of Science (WoS) is an international and multidisciplinary tool for accessing literature in science, technology, biomedicine, and other disciplines. Scopus is a database that today indexes 40,562 peer-reviewed journals, compared to 24,831 for WoS. Thus, Scopus is currently the largest existing multidisciplinary database. However, it may also be necessary to include sources relevant to computer science, such as EI Compendex, IEEE Xplore, and ACM. Table 3 compares the area of expertise of a selection of databases.

Planning Step 3 “Select digital libraries”. Description of digital libraries in computer science and software engineering.

Define inclusion and exclusion criteria

Authors should define the inclusion and exclusion criteria before conducting the review to prevent bias, although these can be adjusted later, if necessary. The selection of primary studies will depend on these criteria. Articles are included or excluded in this first selection based on abstract and primary bibliographic data. When unsure, the article is skimmed to further decide the relevance for the review. Table 4 sets out some criteria types with descriptions and examples.

Planning Step 4 “Define inclusion and exclusion criteria”. Examples of criteria type.

Define the Quality Assessment (QA) checklist

Assessing the quality of an article requires an artifact which describes how to perform a detailed assessment. A typical quality assessment is a checklist that contains multiple factors to evaluate. A numerical scale is used to assess the criteria and quantify the QA [22] . Zhou et al. [25] presented a detailed description of assessment criteria in software engineering, classified into four main aspects of study quality: Reporting, Rigor, Credibility, and Relevance. Each of these criteria can be evaluated using, for instance, a Likert-type scale [17] , as shown in Table 5 . It is essential to select the same scale for all criteria established on the quality assessment.

Planning Step 5 “Define QA assessment checklist”. Examples of QA scales and questions.

Define the “Data Extraction” form

The data extraction form represents the information necessary to answer the research questions established for the review. Synthesizing the articles is a crucial step when conducting research. Ramesh et al. [15] presented a classification scheme for computer science research, based on topics, research methods, and levels of analysis that can be used to categorize the articles selected. Classification methods and fields to consider when conducting a review are presented in Table 6 .

Planning Step 6 “Define data extraction form”. Examples of fields.

The data extraction must be relevant to the research questions, and the relationship to each of the questions should be included in the form. Kitchenham & Charters [6] presented more pertinent data that can be captured, such as conclusions, recommendations, strengths, and weaknesses. Although the data extraction form can be updated if more information is needed, this should be treated with caution since it can be time-consuming. It can therefore be helpful to first have a general background in the research topic to determine better data extraction criteria.

After defining the protocol, conducting the review requires following each of the steps previously described. Using tools can help simplify the performance of this task. Standard tools such as Excel or Google sheets allow multiple researchers to work collaboratively. Another online tool specifically designed for performing SLRs is Parsif.al 1 . This tool allows researchers, especially in the context of software engineering, to define goals and objectives, import articles using BibTeX files, eliminate duplicates, define selection criteria, and generate reports.

Build digital library search strings

Search strings are built considering the PICOC elements and synonyms to execute the search in each database library. A search string should separate the synonyms with the boolean operator OR. In comparison, the PICOC elements are separated with parentheses and the boolean operator AND. An example is presented next:

(“Smart Manufacturing” OR “Digital Manufacturing” OR “Smart Factory”) AND (“Business Process Management” OR “BPEL” OR “BPM” OR “BPMN”) AND (“Semantic Web” OR “Ontology” OR “Semantic” OR “Semantic Web Service”) AND (“Framework” OR “Extension” OR “Plugin” OR “Tool”

Gather studies

Databases that feature advanced searches enable researchers to perform search queries based on titles, abstracts, and keywords, as well as for years or areas of research. Fig. 1 presents the example of an advanced search in Scopus, using titles, abstracts, and keywords (TITLE-ABS-KEY). Most of the databases allow the use of logical operators (i.e., AND, OR). In the example, the search is for “BIG DATA” and “USER EXPERIENCE” or “UX” as a synonym.

Example of Advanced search on Scopus.

In general, bibliometric data of articles can be exported from the databases as a comma-separated-value file (CSV) or BibTeX file, which is helpful for data extraction and quantitative and qualitative analysis. In addition, researchers should take advantage of reference-management software such as Zotero, Mendeley, Endnote, or Jabref, which import bibliographic information onto the software easily.

Study Selection and Refinement

The first step in this stage is to identify any duplicates that appear in the different searches in the selected databases. Some automatic procedures, tools like Excel formulas, or programming languages (i.e., Python) can be convenient here.

In the second step, articles are included or excluded according to the selection criteria, mainly by reading titles and abstracts. Finally, the quality is assessed using the predefined scale. Fig. 2 shows an example of an article QA evaluation in Parsif.al, using a simple scale. In this scenario, the scoring procedure is the following YES= 1, PARTIALLY= 0.5, and NO or UNKNOWN = 0 . A cut-off score should be defined to filter those articles that do not pass the QA. The QA will require a light review of the full text of the article.

Performing quality assessment (QA) in Parsif.al.

Data extraction

Those articles that pass the study selection are then thoroughly and critically read. Next, the researcher completes the information required using the “data extraction” form, as illustrated in Fig. 3 , in this scenario using Parsif.al tool.

Example of data extraction form using Parsif.al.

The information required (study characteristics and findings) from each included study must be acquired and documented through careful reading. Data extraction is valuable, especially if the data requires manipulation or assumptions and inferences. Thus, information can be synthesized from the extracted data for qualitative or quantitative analysis [16] . This documentation supports clarity, precise reporting, and the ability to scrutinize and replicate the examination.

Analysis and Report

The analysis phase examines the synthesized data and extracts meaningful information from the selected articles [10] . There are two main goals in this phase.

The first goal is to analyze the literature in terms of leading authors, journals, countries, and organizations. Furthermore, it helps identify correlations among topic s . Even when not mandatory, this activity can be constructive for researchers to position their work, find trends, and find collaboration opportunities. Next, data from the selected articles can be analyzed using bibliometric analysis (BA). BA summarizes large amounts of bibliometric data to present the state of intellectual structure and emerging trends in a topic or field of research [4] . Table 7 sets out some of the most common bibliometric analysis representations.

Techniques for bibliometric analysis and examples.

Several tools can perform this type of analysis, such as Excel and Google Sheets for statistical graphs or using programming languages such as Python that has available multiple  data visualization libraries (i.e. Matplotlib, Seaborn). Cluster maps based on bibliographic data(i.e keywords, authors) can be developed in VosViewer which makes it easy to identify clusters of related items [18] . In Fig. 4 , node size is representative of the number of papers related to the keyword, and lines represent the links among keyword terms.

[1] Keyword co-relationship analysis using clusterization in vos viewer.

This second and most important goal is to answer the formulated research questions, which should include a quantitative and qualitative analysis. The quantitative analysis can make use of data categorized, labelled, or coded in the extraction form (see Section 1.6). This data can be transformed into numerical values to perform statistical analysis. One of the most widely employed method is frequency analysis, which shows the recurrence of an event, and can also represent the percental distribution of the population (i.e., percentage by technology type, frequency of use of different frameworks, etc.). Q ualitative analysis includes the narration of the results, the discussion indicating the way forward in future research work, and inferring a conclusion.

Finally, the literature review report should state the protocol to ensure others researchers can replicate the process and understand how the analysis was performed. In the protocol, it is essential to present the inclusion and exclusion criteria, quality assessment, and rationality beyond these aspects.

The presentation and reporting of results will depend on the structure of the review given by the researchers conducting the SLR, there is no one answer. This structure should tie the studies together into key themes, characteristics, or subgroups [ 28 ].

SLR can be an extensive and demanding task, however the results are beneficial in providing a comprehensive overview of the available evidence on a given topic. For this reason, researchers should keep in mind that the entire process of the SLR is tailored to answer the research question(s). This article has detailed a practical guide with the essential steps to conducting an SLR in the context of computer science and software engineering while citing multiple helpful examples and tools. It is envisaged that this method will assist researchers, and particularly early-stage researchers, in following an algorithmic approach to fulfill this task. Finally, a quick checklist is presented in Appendix A as a companion of this article.

CRediT author statement

Angela Carrera-Rivera: Conceptualization, Methodology, Writing-Original. William Ochoa-Agurto : Methodology, Writing-Original. Felix Larrinaga : Reviewing and Supervision Ganix Lasa: Reviewing and Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

Funding : This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant No. 814078.

Carrera-Rivera, A., Larrinaga, F., & Lasa, G. (2022). Context-awareness for the design of Smart-product service systems: Literature review. Computers in Industry, 142, 103730.

1 https://parsif.al/

Data Availability

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

The science of literature reviews: searching, identifying, selecting, and synthesising.

1. Introduction

2. making sense of a literature review: what it is all about, 2.1. literature review as a concrete document or standalone writing output, 2.2. literature review as a section of a scientific paper or document, 2.3. some notable literature review approaches, 3. searching, identifying, selecting, and synthesising contents of existing literature, 3.1. searching for literature, 3.2. types of literature search.

• Theme-centric literature search: This search is based on broad themes instead of specific or narrowed concepts. Such a search is bound to produce broad thematic outputs which the researcher must further process to identify specific articles suitable for the research being performed.
• Concept-centric literature search: The focus of this form of search is on the main concepts related to the subject. Put simply, it entails searches using the concepts as keywords.
• Approach-centric literature search: This form of searching the literature is performed according to specific methodological approaches relevant to the research being conducted.
• Author-centric literature search: This involves searching with a focus on the citations or specific authors. This is possible if the researcher knows influential or authoritative authors on a particular subject. This allows searching for specific authors to pull out their publications to ascertain suitable or unsuitable literature for the subject under investigation.
• Journal-centric literature search: This search is based on identifying articles published by a specific journal. It leads to broad outputs but can be necessary while applying broad filtering as part of the search process. It will lead to producing only articles published by a specific journal.
• Period-centric: Thousands of articles are published yearly on a subject. This form of search focuses on the years of publication considered relevant by the researcher. It is based on filtering published materials based on the year of publication (for example, from the last 2 to 5 years). This approach is highly relevant when searching within a subject-focused database.

3.3. Literature Search Techniques

• Manual searching approach: This technique involves surveying tables of contents in relevant key journals manually (in brick-and-mortar libraries) or in hard-copy materials within a physical environment such as an office. It helps in identifying relevant materials which can be further subjected to rigorous physical or desktop search.
• Citation searching (or cited reference searching) approach: This is an approach that is based on searching for articles that have been cited by other publications. It can be used to “find out whether articles have been cited by other authors, find more recent papers on the same or similar subject, discover how a known idea or innovation has been confirmed, applied, improved, extended or corrected” [ 43 ]. It is possible to apply this kind of search on repositories or databases such as OvidSP, Scopus, Web of Science or Google Scholar , among many others.
• Theme searching approach : A theme-based search involving the use of subject headings is crucial in a literature search. Using appropriate subject headings can enhance the literature search and will help a researcher to find more results on a topic/subject. This is because subject headings find articles according to their subject.
• Spider searching approach: This involves identifying specific relevant publications applicable to your research. A further search is performed based on what has been identified to gain additional information. For instance, if the researcher identified a publication that has been cited, a further search could be completed by consulting the reference list of that publication to know more about other works of that cited author. This is called a “backwards spider” approach [ 43 , 44 , 45 ]. The backward spider approach is very common because most literature review processes involve reading through cited paragraphs and identifying listed references to trace (backwardly). Another type of spider approach is when a researcher reads a publication by a particular author and decided to search for other publications written by that same author. This is called a “forwards spider” approach [ 46 ]. It can also take the form of an author reading a particular publication which motivates that researcher to search for other related articles linked to the previous one. This is described as a “sideways spider” approach [ 47 ]. This article does not promote any approach. A combination of search approaches is usually more effective.
• Truncation and wildcard searching approach: This involves the use of truncated and wildcard searches to find variations to widen or reduce the scope of searches. Truncation allows for finding singular and plural terms or keywords with variant endings. Applying truncations and wildcards is easy when using Boolean logic to combine search terms. Boolean logic is a form of algebra which is centred around three simple words known as Boolean operators (that is, AND, OR and NOT ) [ 48 ]. Boolean operators can be used for different combinations of search terms or keywords. Using a wildcard allows for finding variant spellings of search terms and keywords. For instance, applying wildcards are important for finding American and British spellings. In general, truncations and wildcards can take the following formats (with varying influences on the output of a search): - Linking keywords: Entering more than one keyword in a search engine can link those words with other connecting words. This can be completed with the use of AND , OR and NOT. The use of AND or OR or NOT can have different effects on a search. Linking keywords with AND will narrow your search, retrieving only results containing both terms. Linking keywords with OR will broaden your search, finding results that contain either or both terms. Put differently, OR is used to find articles that mention either of the keywords being searched; AND is used to find articles that mention both searched keywords; NOT is used to exclude a keyword or concept from the search. - Asterisking keyword endings: Inserting an asterisk (*) at the word-ending of a keyword will automatically produce a search result for all the possible endings for that word. Many databases use an asterisk (*) as their truncation symbol. It is necessary that researchers apply specific truncations in their search. For example, “therap*” will find therapy, therapies, therapist or therapists [ 49 ]. - Using variant spellings : Using OR to capture variant spellings (e.g., neighbour OR neighbor) will lead to searching for the variant keywords inclusively. - Exacting phrases: Enclosing terms in quotation marks (“”) will lead to a search for that specific term or quote.

3.4. Identifying and Selecting the Literature Materials

3.5. reading and synthesising content, 3.6. analysing research gaps in the literature, 4. typical problems and solutions for better literature review, 5. conclusions, author contributions, data availability statement, acknowledgments, conflicts of interest.

• Webster, J.; Watson, R.T. Analyzing the Past to Prepare for the Future: Writing a Literature Review. MIS Q. 2002 , 26 , xiii–xxiii. [ Google Scholar ]
• Pautasso, M. The Structure and Conduct of a Narrative Literature Review. In A Guide to the Scientific Career: Virtues, Communication, Research and Academic Writing ; Wiley: Hoboken, NJ, USA, 2019; pp. 299–310. [ Google Scholar ]
• Snyder, H. Literature Review as a Research Methodology: An Overview and Guidelines. J. Bus. Res. 2019 , 104 , 333–339. [ Google Scholar ] [ CrossRef ]
• Bolderston, A. Writing an Effective Literature Review. J. Med. Imaging Radiat. Sci. 2008 , 39 , 86–92. [ Google Scholar ] [ CrossRef ]
• Chigbu, U.E. Visually Hypothesising in Scientific Paper Writing: Confirming and Refuting Qualitative Research Hypotheses Using Diagrams. Publications 2019 , 7 , 22. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Galvan, J.L.; Galvan, M.C. Writing Literature Reviews: A Guide for Students of the Social and Behavioral Sciences ; Routledge: London, UK, 2017. [ Google Scholar ]
• Hart, C. Doing a Literature Review: Releasing the Research Imagination ; Sage Publications: Thousand Oaks, CA, USA, 2018. [ Google Scholar ]
• Okoli, C. A Guide to Conducting a Standalone Systematic Literature Review. Commun. Assoc. Inf. Syst. 2015 , 37 , 43. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Agin, S.; Karlsson, M. Mapping the Field of Climate Change Communication 1993–2018: Geographically Biased, Theoretically Narrow, and Methodologically Limited. Environ. Commun. 2021 , 15 , 431–446. [ Google Scholar ] [ CrossRef ]
• Haddaway, N.R.; Bethel, A.; Dicks, L.V.; Koricheva, J.; Macura, B.; Petrokofsky, G.; Pullin, A.S.; Savilaakso, S.; Stewart, G.B. Eight Problems with Literature Reviews and How to Fix Them. Nat. Ecol. Evol. 2020 , 4 , 1582–1589. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jesson, J.; Matheson, L.; Lacey, F.M. Doing Your Literature Review: Traditional and Systematic Techniques ; Sage Publications: Thousand Oaks, CA, USA, 2011. [ Google Scholar ]
• Page, M.J.; Moher, D.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. PRISMA 2020 Explanation and Elaboration: Updated Guidance and Exemplars for Reporting Systematic Reviews. BMJ 2021 , 372 , n160. [ Google Scholar ] [ CrossRef ]
• Iuliano, A.; Aranda, Z.; Colbourn, T.; Agwai, I.C.; Bahiru, S.; Bakare, A.A.; Burgess, R.A.; Cassar, C.; Shittu, F.; Graham, H. The Burden and Risks of Pediatric Pneumonia in Nigeria: A Desk-based Review of Existing Literature and Data. Pediatr. Pulmonol. 2020 , 55 , S10–S21. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Torraco, R.J. Writing Integrative Literature Reviews: Using the Past and Present to Explore the Future. Hum. Resour. Dev. Rev. 2016 , 15 , 404–428. [ Google Scholar ] [ CrossRef ]
• Field, A.P.; Gillett, R. How to Do a Meta-analysis. Br. J. Math. Stat. Psychol. 2010 , 63 , 665–694. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Witell, L.; Snyder, H.; Gustafsson, A.; Fombelle, P.; Kristensson, P. Defining Service Innovation: A Review and Synthesis. J. Bus. Res. 2016 , 69 , 2863–2872. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Torraco, R.J. Writing Integrative Literature Reviews: Guidelines and Examples. Hum. Resour. Dev. Rev. 2005 , 4 , 356–367. [ Google Scholar ] [ CrossRef ]
• Castro, E.M.; Van Regenmortel, T.; Vanhaecht, K.; Sermeus, W.; Van Hecke, A. Patient Empowerment, Patient Participation and Patient-Centeredness in Hospital Care: A Concept Analysis Based on a Literature Review. Patient Educ. Couns. 2016 , 99 , 1923–1939. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dangelico, R.M.; Vocalelli, D. “Green Marketing”: An Analysis of Definitions, Strategy Steps, and Tools through a Systematic Review of the Literature. J. Clean. Prod. 2017 , 165 , 1263–1279. [ Google Scholar ] [ CrossRef ]
• Siddaway, A.P.; Wood, A.M.; Hedges, L.V. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. Annu. Rev. Psychol. 2019 , 70 , 747–770. [ Google Scholar ] [ CrossRef ]
• Munn, Z.; Barker, T.H.; Moola, S.; Tufanaru, C.; Stern, C.; McArthur, A.; Stephenson, M.; Aromataris, E. Methodological Quality of Case Series Studies: An Introduction to the JBI Critical Appraisal Tool. JBI Evid. Synth. 2020 , 18 , 2127–2133. [ Google Scholar ] [ CrossRef ]
• Kwan, B.S.C. The Schematic Structure of Literature Reviews in Doctoral Theses of Applied Linguistics. Engl. Specif. Purp. 2006 , 25 , 30–55. [ Google Scholar ] [ CrossRef ]
• Macke, J.; Genari, D. Systematic Literature Review on Sustainable Human Resource Management. J. Clean. Prod. 2019 , 208 , 806–815. [ Google Scholar ] [ CrossRef ]
• Boote, D.N.; Beile, P. Scholars before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation. Educ. Res. 2005 , 34 , 3–15. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Borman, G.D.; Dowling, N.M. Teacher Attrition and Retention: A Meta-Analytic and Narrative Review of the Research. Rev. Educ. Res. 2008 , 78 , 367–409. [ Google Scholar ] [ CrossRef ]
• Wong, G.; Greenhalgh, T.; Westhorp, G.; Buckingham, J.; Pawson, R. RAMESES Publication Standards: Meta-narrative Reviews. J. Adv. Nurs. 2013 , 69 , 987–1004. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ Green Version ]
• Rodell, J.B.; Breitsohl, H.; Schröder, M.; Keating, D.J. Employee Volunteering: A Review and Framework for Future Research. J. Manag. 2016 , 42 , 55–84. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Antons, D.; Breidbach, C.F. Big Data, Big Insights? Advancing Service Innovation and Design with Machine Learning. J. Serv. Res. 2018 , 21 , 17–39. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Palmatier, R.W.; Houston, M.B.; Hulland, J. Review Articles: Purpose, Process, and Structure. J. Acad. Mark. Sci. 2018 , 46 , 1–5. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Munn, Z.; Peters, M.D.J.; Stern, C.; Tufanaru, C.; McArthur, A.; Aromataris, E. Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach. BMC Med. Res. Methodol. 2018 , 18 , 143. [ Google Scholar ] [ CrossRef ]
• Saunders, M.; Lewis, P.; Thornhill, A. Research Methods for Business Students , 8th ed.; Pearson: London, UK, 2019. [ Google Scholar ]
• Mitchell, A.; Rich, M. The Challenges of Writing an Effective Literature Review for Students and New Researchers of Business. In Proceedings of the 21st European Conference on Research Methodology for Business and Management Studies, University of Aveiro, Aveiro, Portugal, 2–3 June 2022; pp. 141–148. [ Google Scholar ]
• Boyd, B.K.; Solarino, A.M. Ownership of Corporations: A Review, Synthesis, and Research Agenda. J. Manag. 2016 , 42 , 1282–1314. [ Google Scholar ] [ CrossRef ]
• Dhillon, L.; Vaca, S. Refining Theories of Change. Evaluation 2018 , 14 , 64–87. [ Google Scholar ]
• Coughlan, M.; Cronin, P.; Ryan, F. Step-by-Step Guide to Critiquing Research. Part 1: Quantitative Research. Br. J. Nurs. 2007 , 16 , 658–663. [ Google Scholar ] [ CrossRef ]
• Merigó, J.M.; Blanco-Mesa, F.; Gil-Lafuente, A.M.; Yager, R.R. Thirty Years of the International Journal of Intelligent Systems: A Bibliometric Review. Int. J. Intell. Syst. 2017 , 32 , 526–554. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Grewal, A.; Kataria, H.; Dhawan, I. Literature Search for Research Planning and Identification of Research Problem. Indian J. Anaesth. 2016 , 60 , 635. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Metzger, M.J. Making Sense of Credibility on the Web: Models for Evaluating Online Information and Recommendations for Future Research. J. Am. Soc. Inf. Sci. Technol. 2007 , 58 , 2078–2091. [ Google Scholar ] [ CrossRef ]
• Pasko, O.; Chen, F.; Oriekhova, A.; Brychko, A.; Shalyhina, I. Mapping the Literature on Sustainability Reporting: A Bibliometric Analysis Grounded in Scopus and Web of Science Core Collection. Eur. J. Sustain. Dev. 2021 , 10 , 303. [ Google Scholar ] [ CrossRef ]
• Caputo, A.; Kargina, M. A User-Friendly Method to Merge Scopus and Web of Science Data during Bibliometric Analysis. J. Mark. Anal. 2022 , 10 , 82–88. [ Google Scholar ] [ CrossRef ]
• Pranckutė, R. Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications 2021 , 9 , 12. [ Google Scholar ] [ CrossRef ]
• Fagan, J.C. An Evidence-Based Review of Academic Web Search Engines, 2014–2016: Implications for Librarians’ Practice and Research Agenda. Inf. Technol. Libr. 2017 , 36 , 7–47. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Jalali, S.; Wohlin, C. Systematic Literature Studies: Database Searches vs. Backward Snowballing. In Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, Lund, Sweden, 19–20 September 2012; pp. 29–38. [ Google Scholar ]
• Methley, A.M.; Campbell, S.; Chew-Graham, C.; McNally, R.; Cheraghi-Sohi, S. PICO, PICOS and SPIDER: A Comparison Study of Specificity and Sensitivity in Three Search Tools for Qualitative Systematic Reviews. BMC Health Serv. Res. 2014 , 14 , 579. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Cooper, C.; Booth, A.; Varley-Campbell, J.; Britten, N.; Garside, R. Defining the Process to Literature Searching in Systematic Reviews: A Literature Review of Guidance and Supporting Studies. BMC Med. Res. Methodol. 2018 , 18 , 85. [ Google Scholar ] [ CrossRef ]
• Campbell, M.; Egan, M.; Lorenc, T.; Bond, L.; Popham, F.; Fenton, C.; Benzeval, M. Considering Methodological Options for Reviews of Theory: Illustrated by a Review of Theories Linking Income and Health. Syst. Rev. 2014 , 3 , 114. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Cooke, A.; Smith, D.; Booth, A. Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qual. Health Res. 2012 , 22 , 1435–1443. [ Google Scholar ] [ CrossRef ]
• Lian, X.; Wang, L. Boolean Logic Function Realized by Phase-Change Blade Type Random Access Memory. IEEE Trans. Electron Devices 2022 , 69 , 1849–1857. [ Google Scholar ] [ CrossRef ]
• Eriksen, M.B.; Frandsen, T.F. The Impact of Patient, Intervention, Comparison, Outcome (PICO) as a Search Strategy Tool on Literature Search Quality: A Systematic Review. J. Med. Libr. Assoc. JMLA 2018 , 106 , 420. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Batista, P.D.; Campiteli, M.G.; Kinouchi, O. Is It Possible to Compare Researchers with Different Scientific Interests? Scientometrics 2006 , 68 , 179–189. [ Google Scholar ] [ CrossRef ]
• Garfield, E. The History and Meaning of the Journal Impact Factor. JAMA 2006 , 295 , 90–93. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Egghe, L.; Rousseau, R. An Informetric Model for the Hirsch-Index. Scientometrics 2006 , 69 , 121–129. [ Google Scholar ] [ CrossRef ]
• Saad, G. Exploring the H-Index at the Author and Journal Levels Using Bibliometric Data of Productive Consumer Scholars and Business-Related Journals Respectively. Scientometrics 2006 , 69 , 117–120. [ Google Scholar ] [ CrossRef ]
• Dinis-Oliveira, R.J. The H-Index in Life and Health Sciences: Advantages, Drawbacks and Challenging Opportunities. Curr. Drug Res. Rev. Former. Curr. Drug Abus. Rev. 2019 , 11 , 82–84. [ Google Scholar ] [ CrossRef ]
• Mateos, M.; Solé, I. Synthesising Information from Various Texts: A Study of Procedures and Products at Different Educational Levels. Eur. J. Psychol. Educ. 2009 , 24 , 435–451. [ Google Scholar ] [ CrossRef ]
• Dixon-Woods, M.; Agarwal, S.; Jones, D.; Young, B.; Sutton, A. Synthesising Qualitative and Quantitative Evidence: A Review of Possible Methods. J. Health Serv. Res. Policy 2005 , 10 , 45–53. [ Google Scholar ] [ CrossRef ]
• Paulus, T.; Woods, M.; Atkins, D.P.; Macklin, R. The Discourse of QDAS: Reporting Practices of ATLAS. Ti and NVivo Users with Implications for Best Practices. Int. J. Soc. Res. Methodol. 2017 , 20 , 35–47. [ Google Scholar ] [ CrossRef ]
• Hilal, A.H.; Alabri, S.S. Using Nvivo For Data Analysis In Qualitative Research. Int. Interdiscip. J. Educ. 2013 , 2 , 181–186. [ Google Scholar ]
• Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group*, P. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann. Intern. Med. 2009 , 151 , 264–269. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ Green Version ]
• Khan, S.; Rana, S.; Goel, A. Presence of Digital Sources in International Marketing: A Review of Literature Using Leximancer. Int. J. Technol. Mark. 2022 , 16 , 246–274. [ Google Scholar ] [ CrossRef ]
• Nunez-Mir, G.C.; Iannone, B.V., III; Pijanowski, B.C.; Kong, N.; Fei, S. Automated Content Analysis: Addressing the Big Literature Challenge in Ecology and Evolution. Methods Ecol. Evol. 2016 , 7 , 1262–1272. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Müller-Bloch, C.; Kranz, J. A Framework for Rigorously Identifying Research Gaps in Qualitative Literature Reviews ; CoRe Publications: Camarillo, CA, USA, 2015. [ Google Scholar ]
• Robinson, K.A.; Saldanha, I.J.; Mckoy, N.A. Development of a Framework to Identify Research Gaps from Systematic Reviews. J. Clin. Epidemiol. 2011 , 64 , 1325–1330. [ Google Scholar ] [ CrossRef ]
• Miles, D.A. A Taxonomy of Research Gaps: Identifying and Defining the Seven Research Gaps. In Doctoral Student Workshop: Finding Research Gaps-Research Methods and Strategies ; Researchgate: Dallas, TX, USA, 2017. [ Google Scholar ]
• Zhang, C.; Li, Y.; Shuai, D.; Shen, Y.; Wang, D. Progress and Challenges in Photocatalytic Disinfection of Waterborne Viruses: A Review to Fill Current Knowledge Gaps. Chem. Eng. J. 2019 , 355 , 399–415. [ Google Scholar ] [ CrossRef ]
• McGaha, K.K. An Interpretive Phenomenological Study of America’s Emerging Workforce: Exploring Generation Z’s Leadership Preferences. Ph.D. Thesis, University of Phoenix, Phoenix, AZ, USA, 2018. [ Google Scholar ]
• Kushner, K.E.; Morrow, R. Grounded Theory, Feminist Theory, Critical Theory: Toward Theoretical Triangulation. Adv. Nurs. Sci. 2003 , 26 , 30–43. [ Google Scholar ] [ CrossRef ]
• Thurmond, V.A. The Point of Triangulation. J. Nurs. Scholarsh. 2001 , 33 , 253–258. [ Google Scholar ] [ CrossRef ]
• Dubois, N.S.; Gomez, A.; Carlson, S.; Russell, D. Bridging the Research-implementation Gap Requires Engagement from Practitioners. Conserv. Sci. Pract. 2020 , 2 , e134. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• McKendrick, J.H. Multi-Method Research: An Introduction to Its Application in Population Geography. Prof. Geogr. 1999 , 51 , 40–50. [ Google Scholar ] [ CrossRef ]
• Guion, L.A. Triangulation: Establishing the Validity of Qualitative Studies. Edis 1969 , 2002 , 2–4. [ Google Scholar ] [ CrossRef ]
• MacInnis, D.J. A Framework for Conceptual Contributions in Marketing. J. Mark. 2011 , 75 , 136–154. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Jennex, M.E. Literature Reviews and the Review Process: An Editor-in-Chief’s Perspective. Commun. Assoc. Inf. Syst. 2015 , 36 , 8. [ Google Scholar ] [ CrossRef ]
• Davis, J.; Mengersen, K.; Bennett, S.; Mazerolle, L. Viewing Systematic Reviews and Meta-Analysis in Social Research through Different Lenses. Springerplus 2014 , 3 , 511. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Colquhoun, H.L.; Levac, D.; O’Brien, K.K.; Straus, S.; Tricco, A.C.; Perrier, L.; Kastner, M.; Moher, D. Scoping Reviews: Time for Clarity in Definition, Methods, and Reporting. J. Clin. Epidemiol. 2014 , 67 , 1291–1294. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Chigbu, U.E.; Atiku, S.O.; Du Plessis, C.C. The Science of Literature Reviews: Searching, Identifying, Selecting, and Synthesising. Publications 2023 , 11 , 2. https://doi.org/10.3390/publications11010002

Chigbu UE, Atiku SO, Du Plessis CC. The Science of Literature Reviews: Searching, Identifying, Selecting, and Synthesising. Publications . 2023; 11(1):2. https://doi.org/10.3390/publications11010002

Chigbu, Uchendu Eugene, Sulaiman Olusegun Atiku, and Cherley C. Du Plessis. 2023. "The Science of Literature Reviews: Searching, Identifying, Selecting, and Synthesising" Publications 11, no. 1: 2. https://doi.org/10.3390/publications11010002

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Something old, new, and borrowed . Rise of the systematic reviews

• Published: 24 August 2024

Cite this article

• Gheorghe-Gavrilă Hognogi 1 &
• Ana-Maria Pop   ORCID: orcid.org/0000-0001-9958-1391 1  

83 Accesses

Explore all metrics

Systematic reviews and other types of literature reviews are more prevalent in clinical medicine than in other fields. The recurring need for improvement and updates in these disciplines has led to the Living Systematic Review (LSR) concept to enhance the effectiveness of scientific synthesis efforts. While LSR was introduced in 2014, its adoption outside clinical medicine has been limited, with one exception. However, it is anticipated that this will change in the future, prompting a detailed exploration of four key dimensions for LSR development, regardless of the scientific domain. These dimensions include (a) compliance with FAIR principles, (b) interactivity to facilitate easier access to scientific knowledge, (c) public participation for a more comprehensive review, and (d) extending the scope beyond mere updates to living systematic reviews. Each field needs to establish clear guidelines for drafting literature reviews as independent studies, with discussions centring around the central theme of the Living Systematic Review.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Artificial Intelligence

Aguilar Gómez, F., & Bernal, I. (2023). FAIR EVA: Bringing institutional multidisciplinary repositories into the FAIR picture. Scientific Data . https://doi.org/10.1038/s41597-023-02652-8

Article   Google Scholar  

Akl, E. A., Meerpohl, J. J., Elliott, J., Kahale, L. A., & Schünemann, H. J. (2017). Living systematic reviews: 4. Living guideline recommendations. Journal of Clinical Epidemiology, 91 , 47–53. https://doi.org/10.1016/j.jclinepi.2017.08.009

Amaral, O. B. (2022). To fix peer review, break it into stages. Nature, 611 , 637. https://doi.org/10.1038/d41586-022-03791-5

Breuer, C., Meerpohl, J. J., & Siemens, W. (2022). From standard systematic reviews to living systematic reviews. Zeitschrift fur Evidenz, Fortbildung und Qualitat Im Gesundheitswesen, 176 , 76–81. https://doi.org/10.1016/j.zefq.2022.11.007

Dajani, R. (2023). Scientists in diaspora are a powerful resource for their home countries. Nature . https://doi.org/10.1038/d41586-023-03300-2

Eisen, M. B., Akhmanova, A., Behrens, T. E., Harper, D. M., Weigel, D., & Zaidi, M. (2020). Implementing a “publish, then review” model of publishing. eLife, 9 , e64910. https://doi.org/10.7554/eLife.64910

Elliott, J. H., Synnot, A., Turner, T., Simmonds, M., Akl, E. A., McDonald, S., Salanti, G., Meerpohl, J., MacLehose, H., Hilton, J., Tovey, D., Shemilt, I., Thomas, J., Agoritsas, T., Hilton, J., Perron, C., Akl, E., Hodder, R., Pestridge, C., …, Pearson, L. (2017). Living systematic review: 1. Introduction—The why, what, when, and how. Journal of Clinical Epidemiology, 91 , 23–30. https://doi.org/10.1016/j.jclinepi.2017.08.010

Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014). Living systematic reviews: An emerging opportunity to narrow the evidence-practice gap. PLoS Medicine . https://doi.org/10.1371/journal.pmed.1001603

Enck, P. (2018). Living systematic reviews, not only for clinical (placebo) research. Journal of Clinical Epidemiology, 98 , 153–153. https://doi.org/10.1016/j.jclinepi.2018.01.001

Gerber, L. R. (2023). Bridging the gap between science and policy for a sustainable future. Nature Water . https://doi.org/10.1038/s44221-023-00145-x

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26 (2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Griebler, U., Dobrescu, A., Ledinger, D., Klingenstein, P., Sommer, I., Emprechtinger, R., Persad, E., Gadinger, A., Trivella, M., Klerings, I., & Nussbaumer-Streit, B. (2023). Evaluation of the interim Cochrane rapid review methods guidance—A mixed-methods study on the understanding of and adherence to the guidance. Research Synthesis Methods . https://doi.org/10.1002/jrsm.1656

Hill, J. E., Harris, C., & Clegg, A. (2023). Methods for using Bing’s AI-powered search engine for data extraction for a systematic review. Research Synthesis Methods, 15 (2), 347–353. https://doi.org/10.1002/jrsm.1689

Kahale, L. A., Piechotta, V., & McKenzie, J. E. (2022). Extension of the PRISMA 2020 statement for living systematic reviews (LSRs): Protocol [version 2; peer review: 1 approved]. F1000Research . https://doi.org/10.12688/f1000research.75449.2

Macdonald, H., Loder, E., & Abbasi, K. (2020). Living systematic reviews at The BMJ. BMJ, 370 , m2925. https://doi.org/10.1136/bmj.m2925

Marshall, I. J., & Wallace, B. C. (2019). Toward systematic review automation: A practical guide to using machine learning tools in research synthesis. Systematic Reviews . https://doi.org/10.1186/s13643-019-1074-9

Norström, A., V., Cvitanovic, C., Löf, M. F., West, S., Wyborn, C., Balvanera, P., Bednarek, A. T., Bennett, E. M., Biggs, R., Bremond, A., Campbell, B. M., Canadell, J. G., Carpenter, S. R., Folke, C., Fulton, E. A., Gaffney, O., Gelcich, S., Jouffray, J.-B., Leach, M., …, Österblom, H. (2020). Principles for knowledge co-production in sustainability research. Nature Sustainability, 3 , 182–190. https://doi.org/10.1038/s41893-019-0448-2

Paul, M., & Leeflang, M. M. (2023). Living systematic reviews: Aims and standards. Clinical Microbiology and Infection, 30 (3), 265–266. https://doi.org/10.1016/j.cmi.2023.08.005

Polonioli, A. (2019). A plea for minimally biased naturalistic philosophy. Synthese, 196 , 3841–3867. https://doi.org/10.1007/s11229-017-1628-0

Polonioli, A. (2020). In search of better science: On the epistemic costs of systematic reviews and the need for a pluralistic stance to literature search. Scientometrics, 122 , 1267–1274. https://doi.org/10.1007/s11192-019-03333-3

Riley, S. P., Swanson, B. T., Shaffer, S. M., Flowers, D. W., Cook, C. E., & Brismée, J. M. (2023). Why do ‘Trustworthy’ living systematic reviews matter? Journal of Manual & Manipulative Therapy, 31 (4), 215–219. https://doi.org/10.1080/10669817.2023.2229610

Ripberger, J., Bell, A., Fox, A., Forney, A., Livingston, W., Gaddie, C., Silva, C., & Jenkins-Smith, H. (2022). Communicating probability information in weather forecasts: Findings and recommendations from a living systematic review of the research literature. Weather, Climate, and Society, 14 (2), 481–498. https://doi.org/10.1175/WCAS-D-21-0034.1

Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015). Public data archiving in ecology and evolution: How well are we doing? PLoS Biology, 13 (11), e1002295. https://doi.org/10.1371/journal.pbio.1002295

Saulnier, K. M., Bujold, D., Dyke, S. O. M., Dupras, C., Beck, S., Bourque, G., & Joly, Y. (2019). Benefits and barriers in the design of harmonized access agreements for international data sharing. Scientific Data . https://doi.org/10.1038/s41597-019-0310-4

Schimidt, L., Mohamed, S., Meader, N., Bacardit, J., & Craig, D. (2023). Automated data analysis of unstructured grey literature in health research: A mapping review. Research Synthesis Methods, 15 (2), 178–197. https://doi.org/10.1002/jrsm.1692

Siemieniuk, R. A., Bartoszko, J. J., Zeraatkar, D., Kum, E., Qasim, A., Martinez, J. P. D., Izcovich, A., Lamontagne, F., Han, M. A., Agarwal, A., Agoritsas, T., Azab, M., Bravo, G., Chu, D. K., Couban, R., Devji, T., Escamilla, Z., Foroutan, F., Gao, Y., …, Brignardello-Petersen, R. (2020). Drug treatments for Covid-19: Living systematic review and network meta-analysis BMJ, 370 , m3536. https://doi.org/10.1136/bmj.m2980

Simmonds, M., Salanti, G., McKenzie, J., & Elliott, J. (2017). Living systematic reviews: 3. Statistical methods for updating meta-analyses. Journal of Clinical Epidemiology, 91 , 38–46. https://doi.org/10.1016/j.jclinepi.2017.08.008

Siontis, K. C., & Ioannidis, J. P. A. (2018). Replication, duplication, and waste in a quarter million systematic reviews and meta-analyses. Circulation. Cardiovascular Quality and Outcomes, 11 (12), e005212. https://doi.org/10.1161/CIRCOUTCOMES.118.005212

Thibault, R. T., Amaral, O. B., Argolo, F., Bandrowski, A. E., Davidson, A. R., & Drude, N. I. (2023). Open Science 2.0: Towards a truly collaborative research ecosystem. PLoS Biology, 21 (10), e3002362. https://doi.org/10.1371/journal.pbio.3002362

Thomas, J., Noel-Storr, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilt, I., Synnot, A., Turner, T., & Elliott, J. (2017). Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology, 91 , 31–37. https://doi.org/10.1016/j.jclinepi.2017.08.011

Thorp, H. H. (2023). Correction is courageous. Science, 382 , 743–743. https://doi.org/10.1126/science.adm8205

Turk, V. (2023). Protect the ‘right to science’ for people and the planet. Nature . https://doi.org/10.1038/d41586-023-03332-8

Turner, T., Lavis, J. N., Grimshaw, J. M., Green, S., & Elliott, J. (2023). Living evidence and adaptive policy: Perfect partners? Health Research Policy and Systems . https://doi.org/10.1186/s12961-023-01085-4

Uttley, L., Quintana, D. S., Montgomery, P., Carroll, C., Page, M. J., Falzon, L., Sutton, A., & Moher, D. (2023). The problems with systematic reviews: A living systematic review. Journal of Clinical Epidemiology, 156 , 30–41. https://doi.org/10.1016/j.jclinepi.2023.01.011

Vallet, A., Locatelli, B., Valdivia-Díaz, M., Vallet, A., Locatelli, B., Valdivia-Díaz, M., Conde, Y. Q., Matencio García, G., Criales, A. R., Huamanñahui, F. V., Criales, S. R., Makowski, D., & Lavorel, S. (2023). Knowledge coproduction to improve assessments of nature’s contributions to people. Conservation Biology . https://doi.org/10.1111/cobi.14182

van Noorden, R. (2023). How big is science’s fake-paper problem? Nature, 623 , 466–467. https://doi.org/10.1038/d41586-023-03464-x

Wilkinson, M., Dumontier, M., Aalbersberg, I., Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., ..., Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3 , 160018. https://doi.org/10.1038/sdata.2016.18

Download references

Acknowledgements

The authors would like to thank both reviewers for their useful and precious suggestions.

Author information

Authors and affiliations.

Universitatea Babeș-Bolyai, Facultatea de Geografie, Centrul de Geografie Regională, Str. Clinicilor 5-7, 400006, Cluj-Napoca, Romania

Gheorghe-Gavrilă Hognogi & Ana-Maria Pop

You can also search for this author in PubMed   Google Scholar

Contributions

Gheorghe-Gavrilă Hognogi—substantial contribution to conception and design, acquisition and interpretation of data, writing the comment, revision; Ana-Maria Pop—interpretation of data.

Corresponding author

Correspondence to Gheorghe-Gavrilă Hognogi .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Hognogi, GG., Pop, AM. Something old, new, and borrowed . Rise of the systematic reviews. Scientometrics (2024). https://doi.org/10.1007/s11192-024-05133-w

Download citation

Received : 07 February 2024

Accepted : 02 August 2024

Published : 24 August 2024

DOI : https://doi.org/10.1007/s11192-024-05133-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Living systematic reviews
• Scientific knowledge
• FAIR principles
• Systematic review
• Find a journal
• Publish with us
• Track your research