research gap for social media

The Research Gap (Literature Gap)

Everything you need to know to find a quality research gap

By: Ethar Al-Saraf (PhD) | Expert Reviewed By: Eunice Rautenbach (DTech) | November 2022

If you’re just starting out in research, chances are you’ve heard about the elusive research gap (also called a literature gap). In this post, we’ll explore the tricky topic of research gaps. We’ll explain what a research gap is, look at the four most common types of research gaps, and unpack how you can go about finding a suitable research gap for your dissertation, thesis or research project.

Overview: Research Gap 101

• What is a research gap
• Four common types of research gaps
• Practical examples
• How to find research gaps
• Recap & key takeaways

What (exactly) is a research gap?

Well, at the simplest level, a research gap is essentially an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space. Alternatively, a research gap can also exist when there’s already a fair deal of existing research, but where the findings of the studies pull in different directions , making it difficult to draw firm conclusions.

For example, let’s say your research aims to identify the cause (or causes) of a particular disease. Upon reviewing the literature, you may find that there’s a body of research that points toward cigarette smoking as a key factor – but at the same time, a large body of research that finds no link between smoking and the disease. In that case, you may have something of a research gap that warrants further investigation.

Now that we’ve defined what a research gap is – an unanswered question or unresolved problem – let’s look at a few different types of research gaps.

Types of research gaps

While there are many different types of research gaps, the four most common ones we encounter when helping students at Grad Coach are as follows:

• The classic literature gap
• The disagreement gap
• The contextual gap, and
• The methodological gap

Need a helping hand?

1. The Classic Literature Gap

First up is the classic literature gap. This type of research gap emerges when there’s a new concept or phenomenon that hasn’t been studied much, or at all. For example, when a social media platform is launched, there’s an opportunity to explore its impacts on users, how it could be leveraged for marketing, its impact on society, and so on. The same applies for new technologies, new modes of communication, transportation, etc.

Classic literature gaps can present exciting research opportunities , but a drawback you need to be aware of is that with this type of research gap, you’ll be exploring completely new territory . This means you’ll have to draw on adjacent literature (that is, research in adjacent fields) to build your literature review, as there naturally won’t be very many existing studies that directly relate to the topic. While this is manageable, it can be challenging for first-time researchers, so be careful not to bite off more than you can chew.

2. The Disagreement Gap

As the name suggests, the disagreement gap emerges when there are contrasting or contradictory findings in the existing research regarding a specific research question (or set of questions). The hypothetical example we looked at earlier regarding the causes of a disease reflects a disagreement gap.

Importantly, for this type of research gap, there needs to be a relatively balanced set of opposing findings . In other words, a situation where 95% of studies find one result and 5% find the opposite result wouldn’t quite constitute a disagreement in the literature. Of course, it’s hard to quantify exactly how much weight to give to each study, but you’ll need to at least show that the opposing findings aren’t simply a corner-case anomaly .

3. The Contextual Gap

The third type of research gap is the contextual gap. Simply put, a contextual gap exists when there’s already a decent body of existing research on a particular topic, but an absence of research in specific contexts .

For example, there could be a lack of research on:

• A specific population – perhaps a certain age group, gender or ethnicity
• A geographic area – for example, a city, country or region
• A certain time period – perhaps the bulk of the studies took place many years or even decades ago and the landscape has changed.

The contextual gap is a popular option for dissertations and theses, especially for first-time researchers, as it allows you to develop your research on a solid foundation of existing literature and potentially even use existing survey measures.

Importantly, if you’re gonna go this route, you need to ensure that there’s a plausible reason why you’d expect potential differences in the specific context you choose. If there’s no reason to expect different results between existing and new contexts, the research gap wouldn’t be well justified. So, make sure that you can clearly articulate why your chosen context is “different” from existing studies and why that might reasonably result in different findings.

4. The Methodological Gap

Last but not least, we have the methodological gap. As the name suggests, this type of research gap emerges as a result of the research methodology or design of existing studies. With this approach, you’d argue that the methodology of existing studies is lacking in some way , or that they’re missing a certain perspective.

For example, you might argue that the bulk of the existing research has taken a quantitative approach, and therefore there is a lack of rich insight and texture that a qualitative study could provide. Similarly, you might argue that existing studies have primarily taken a cross-sectional approach , and as a result, have only provided a snapshot view of the situation – whereas a longitudinal approach could help uncover how constructs or variables have evolved over time.

Practical Examples

Let’s take a look at some practical examples so that you can see how research gaps are typically expressed in written form. Keep in mind that these are just examples – not actual current gaps (we’ll show you how to find these a little later!).

Context: Healthcare

Despite extensive research on diabetes management, there’s a research gap in terms of understanding the effectiveness of digital health interventions in rural populations (compared to urban ones) within Eastern Europe.

Context: Environmental Science

While a wealth of research exists regarding plastic pollution in oceans, there is significantly less understanding of microplastic accumulation in freshwater ecosystems like rivers and lakes, particularly within Southern Africa.

Context: Education

While empirical research surrounding online learning has grown over the past five years, there remains a lack of comprehensive studies regarding the effectiveness of online learning for students with special educational needs.

As you can see in each of these examples, the author begins by clearly acknowledging the existing research and then proceeds to explain where the current area of lack (i.e., the research gap) exists.

How To Find A Research Gap

Now that you’ve got a clearer picture of the different types of research gaps, the next question is of course, “how do you find these research gaps?” .

Well, we cover the process of how to find original, high-value research gaps in a separate post . But, for now, I’ll share a basic two-step strategy here to help you find potential research gaps.

As a starting point, you should find as many literature reviews, systematic reviews and meta-analyses as you can, covering your area of interest. Additionally, you should dig into the most recent journal articles to wrap your head around the current state of knowledge. It’s also a good idea to look at recent dissertations and theses (especially doctoral-level ones). Dissertation databases such as ProQuest, EBSCO and Open Access are a goldmine for this sort of thing. Importantly, make sure that you’re looking at recent resources (ideally those published in the last year or two), or the gaps you find might have already been plugged by other researchers.

Once you’ve gathered a meaty collection of resources, the section that you really want to focus on is the one titled “ further research opportunities ” or “further research is needed”. In this section, the researchers will explicitly state where more studies are required – in other words, where potential research gaps may exist. You can also look at the “ limitations ” section of the studies, as this will often spur ideas for methodology-based research gaps.

By following this process, you’ll orient yourself with the current state of research , which will lay the foundation for you to identify potential research gaps. You can then start drawing up a shortlist of ideas and evaluating them as candidate topics . But remember, make sure you’re looking at recent articles – there’s no use going down a rabbit hole only to find that someone’s already filled the gap 🙂

Let’s Recap

We’ve covered a lot of ground in this post. Here are the key takeaways:

• A research gap is an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space.
• The four most common types of research gaps are the classic literature gap, the disagreement gap, the contextual gap and the methodological gap. 
• To find potential research gaps, start by reviewing recent journal articles in your area of interest, paying particular attention to the FRIN section .

If you’re keen to learn more about research gaps and research topic ideation in general, be sure to check out the rest of the Grad Coach Blog . Alternatively, if you’re looking for 1-on-1 support with your dissertation, thesis or research project, be sure to check out our private coaching service .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

41 Comments

This post is REALLY more than useful, Thank you very very much

Very helpful specialy, for those who are new for writing a research! So thank you very much!!

I found it very helpful article. Thank you.

it very good but what need to be clear with the concept is when di we use research gap before we conduct aresearch or after we finished it ,or are we propose it to be solved or studied or to show that we are unable to cover so that we let it to be studied by other researchers ?

Just at the time when I needed it, really helpful.

Very helpful and well-explained. Thank you

VERY HELPFUL

We’re very grateful for your guidance, indeed we have been learning a lot from you , so thank you abundantly once again.

hello brother could you explain to me this question explain the gaps that researchers are coming up with ?

Am just starting to write my research paper. your publication is very helpful. Thanks so much

How to cite the author of this?

your explanation very help me for research paper. thank you

Very important presentation. Thanks.

Very helpful indeed

Best Ideas. Thank you.

I found it’s an excellent blog to get more insights about the Research Gap. I appreciate it!

Kindly explain to me how to generate good research objectives.

This is very helpful, thank you

How to tabulate research gap

Very helpful, thank you.

Thanks a lot for this great insight!

This is really helpful indeed!

This article is really helpfull in discussing how will we be able to define better a research problem of our interest. Thanks so much.

Reading this just in good time as i prepare the proposal for my PhD topic defense.

Very helpful Thanks a lot.

Thank you very much

This was very timely. Kudos

Great one! Thank you all.

Thank you very much.

This is so enlightening. Disagreement gap. Thanks for the insight.

How do I Cite this document please?

Research gap about career choice given me Example bro?

I found this information so relevant as I am embarking on a Masters Degree. Thank you for this eye opener. It make me feel I can work diligently and smart on my research proposal.

This is very helpful to beginners of research. You have good teaching strategy that use favorable language that limit everyone from being bored. Kudos!!!!!

This plat form is very useful under academic arena therefore im stil learning a lot of informations that will help to reduce the burden during development of my PhD thesis

This information is beneficial to me.

Insightful…

I have found this quite helpful. I will continue using gradcoach for research assistance

Doing research in PhD accounting, my research topic is: Business Environment and Small Business Performance: The Moderating Effect of Financial Literacy in Eastern Uganda. I am failing to focus the idea in the accounting areas. my supervisor tells me my research is more of in the business field. the literature i have surveyed has used financial literacy as an independent variable and not as a moderator. Kindly give me some guidance here. the core problem is that despite the various studies, small businesses continue to collapse in the region. my vision is that financial literacy is still one of the major challenges hence the need for this topic.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Public Response to Alerts and Warnings Using Social Media: Report of a Workshop on Current Knowledge and Research Gaps (2013)

Chapter: 6 research gaps and implementation challenges.

Research Gaps and Implementation Challenges

S ocial media represent a relatively new and still rapidly evolving phenomenon, and their application to alerts, warnings, and other aspects of emergency management is still in its infancy. But there is much current interest in the use of social media because they have been embraced by a large segment of the population and because they enable new, two-way interactions among those affected by and responding to disasters. To date, formal study of the use of social media in disasters has been limited ( Box 6.1 explores the state of research on the use of social media in emergency management), and there are many outstanding questions about how they can be used most effectively by emergency managers and other public officials, organizations, communities, and individuals.

The following sections outline research opportunities and associated implementation challenges identified by the committee and workshop attendees during the plenary and breakout sessions of the workshop. The opportunities and challenges compiled here from presentations and discussions at the workshop do not reflect a consensus of the committee or the workshop participants, nor are they intended to be a comprehensive list of research questions.

MESSAGE CONTENT AND DISSEMINATION

A significant body of past research has considered what types of messages and communications strategies are most effective for alerting the public with traditional emergency alerting tools like broadcast radio

BOX 6.1 State of Research on Social Media in Emergency Management

In her remarks at the February 2012 workshop on alerts and warnings using social media, Leysia Palen of the University of Colorado, Boulder, discussed the evolving application of social media for emergency management and the associated stages of research maturity. She suggested that growing interest in examining the role of social media reflects in part the progress that has been made toward their adoption, and that research together with learning from the practical application of social media will increase understanding of both possibilities and pitfalls and thus foster greater, more effective use.

From roughly 2008 to 2011 was a period in which the potential for using social media was first recognized and was marked by scattered grassroots experimentation, said Palen. In this first stage, publications by practitioners and researchers, workshops, and discussion developed a case that social media would inevitably play an important role in emergency management, although just how was unclear. Not all embraced the new technologies. Some felt that the use of social media was simply a passing fad, and even as late as 2011 otherwise knowledgeable people remained fearful about social-media-abetted change and sought to understand how social media could be “held back.” Still others embraced the trend but did not fully understand its grassroots and spontaneous nature; one result was attempts to shape it in order to gain commercial or tactical advantage.

Indications abound that both practice and research have since yielded significant advances, observed Palen. Local emergency managers are experimenting with how to incorporate social media into their daily practices, for example, and the American Red Cross has incorporated certified volunteers into its social media response plans. Formal policy discussions are being held worldwide.

and television. 1 Comparatively little research has examined similar questions for messages disseminated via social media. 2 One of social media’s

1 National Research Council. Public Response to Alerts and Warnings on Mobile Devices: Summary of a Workshop on Current Knowledge and Research Gaps. The National Academies Press, Washington, D.C., 2011.

2 Research that has been done in this area includes Kate Starbird, Leysia Palen, Amanda Hughes, and Sarah Vieweg, Chatter on the red: What hazards threat reveals about the social life of microblogged information, Proceedings of the ACM 2010 Conference on Computer Supported Cooperative Work (CSCW 2010), pp. 241-250, 2010; Kate Starbird and Leysia Palen, Pass it on?: Retweeting in mass emergencies, Proceedings of the Conference on Information Systems for Crisis Response and Management (ISCRAM 2010), Seattle, Wash., 2010; Leysia Palen, Sarah Vieweg, Sophia Liu, and Amanda Hughes, Crisis in a networked world: Features of computer-mediated communication in the April 16, 2007, Virginia Tech event, Social Science Computing Review, Sage, pp. 467-480, 2009; and Clarence Wardell and Yee San Su, Social Media + Emergency Management Camp: Transforming the Response Enterprise , 2011, available at http://www.wilsoncenter.org/sites/default/files/SMEM_Report.pdf .

particular strengths, that messages can be widely shared, also presents challenges because messages can be readily altered as they are spread. In addition, messages that may no longer be accurate can continue to propagate through social media long after they are no longer current. Their interactive nature makes social media useful as a medium for both receiving and confirming disaster information, which suggests opportunities to reduce the gap between the time individuals receive disaster information and when they take action.

Some specific research questions include the following:

• How should broadcast messages from emergency managers be crafted in light of the limitations (e.g., short message lengths) and strengths (e.g., opportunities to include images, maps, and URLs) presented by social media?

• How much of the word-of-mouth dissemination of information about disasters occurs through social media? Are there ways of designing messages that could increase the speed and breadth of their spread?

• How are messages altered as they are spread through social networks? How might messages be formulated to discourage or reduce the impact of these changes?

• What strategies and techniques can be applied to deal with messages that have “aged” to the point that they are no longer relevant?

• What types of messages and strategies would reduce the time lag before individuals take action (i.e., reduce milling time)?

• What challenges or opportunities will social media present in reaching unique populations such as non-English speakers or individuals with disabilities?

TRUST AND CREDIBILITY

In addition to sharing and commenting on messages they receive, citizens often use social media to share firsthand text, image, and video reports about disasters. This firsthand information can be useful for decision making by emergency officials as well as other individuals but raises questions about how to assess its trustworthiness. The nature of social media suggests the possibility for self-correcting information by combining reports supplied by many individuals provided that the number of reports is sufficiently large.

• How do consumers of social media messages distinguish credible from less credible information? How can emergency managers and other officials create and disseminate messages that have high credibility?

• What are practical ways that officials can evaluate and signal the credibility of unofficial messages during an event?

• What are the relationships between the number and density of social media users or the size of an event and the effectiveness of mechanisms for self-correcting information users supply?

• What mechanisms and approaches foster such self-correction?

• What are effective strategies officials can use to intercede when misinformation is proliferated via social media?

The use of social media for alerts and warnings raises privacy issues that were not in play with traditional methods of sending alerts and warnings. For example, the social media communications being monitored by government officials, while technically public, may have been sent with certain expectations of privacy such as that they would not be read by government officials.

• How, if at all, do people differentiate the privacy implications of message monitoring by government agencies, by commercial entities, and by the general public during disasters versus at other times?

• It has been suggested that people are willing to accept reduced privacy safeguards during disasters. What are people’s actual attitudes in these circumstances?

• How might the government’s use of social media be adjusted during disasters? For example, are there mechanisms that could be used to trigger monitoring when a disaster begins? What safeguards could be established to ensure that people have full control of adjustment to and reactivation of privacy settings?

• Is widespread adoption of social media, which relies on users sharing information about themselves, altering the privacy expectations of users of social media? What are the implications for the use of social media during disasters?

Social media have enabled the emergence of online groups of volunteers to respond to disasters. Some of these groups, such as the Standby Task Force and Humanity Road, have evolved from ad hoc groups to more structured volunteer organizations that designate individuals responsible

for coordinating response activities. These more formal groups as well as spontaneously formed groups help curate disaster information from social media and other sources and use social media to provide relevant information to both official responders and the affected population.

• What organizational theory provides an understanding of how ad hoc volunteer organizations form, function, and evolve—and what the implications are for disaster management?

• Although official first responders in government and nongovernment organizations have had training to deal with emergency situations, most ad hoc volunteers have not. Are there ways that social media can be used to make the efforts of ad hoc volunteers more effective?

• How do legal and policy concerns constrain the interactions of volunteers with formal emergency managers? What measures might be taken to address these concerns?

• What are points of cooperation and tension between officials and volunteers?

TECHNOLOGY DIFFUSION

Several instances of technologies that could have immediate application for disaster management were discussed during the workshop, such as support for visualization of information derived from social media. However, it was also evident that there were relatively few points of engagement between researchers developing or investigating new tools and emergency managers and other potential end users. Emergency managers are most likely to encounter new technologies only when such tools are made available by vendors. Given the rapid pace of change in social media and the associated rapid pace of change in tools for using social media, workshop participants suggested that more rapid and effective technology transfer would be valuable.

• Are there emerging best practices for how social media can be used effectively by emergency managers?

• How can diffusion of available technologies be promoted? What are the special characteristics of the emergency management community that limit the adoption of new technologies and techniques, and how might such characteristics be addressed?

• How can the growing body of knowledge on how users behave in online communities be transferred to emergency management practices?

EMERGENCY MANAGEMENT PRACTICE

Although the rate of adoption of social media and the sophistication of their use by emergency managers vary considerably, it does appear that emergency managers have come to generally appreciate the potential value of social media. However, workshop participants cited a number of barriers that still exist to the effective use of social media in the practice of emergency management. These barriers stem in no small part from an incomplete understanding, as discussed above, of how to use social media in disasters and the relative newness of the medium.

Some specific challenges include the following:

• Limited knowledge about information-sharing techniques and collection of information;

• Limited staff plus budget challenges that create barriers to using social media for situational awareness;

• Lack of policies and discussion about the use of social media for dissemination of alerts and warnings and for situational awareness; and

• Concerns about potential liabilities that are created when new technology is introduced, specifically with respect to fair representation of victims’ needs and the distribution of resources.

Following an earlier NRC workshop on public response to alerts and warnings delivered to mobile devices, a related workshop was held on February 28 and 29, 2012 to look at the role of social media in disaster response. This was one of the first workshops convened to look systematically at the use of social media for alerts and warnings—an event that brought together social science researchers, technologists, emergency management professionals, and other experts on how the public and emergency managers use social media in disasters.In addition to exploring how officials monitor social media, as well as the resulting privacy considerations, the workshop focused on such topics as: what is known about how the public responds to alerts and warnings; the implications of what is known about such public responses for the use of social media to provide alerts and warnings to the public; and approaches to enhancing the situational awareness of emergency managers.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Time to bridge the gaps: issues with current social media research in public relations

• Conference: Australian and New Zealand Communication Association conference: Global Networks-Global Divides: Bridging New and Traditional Communication Challenges
• At: Freemantle, WA, Australia

• University of Canberra

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• PUBLIC RELAT REV

• Seydou Harouna Fatouma

• Publ Relat J
• Angelica Evans
• Jane Twomey
• Scott Talan
• Christine Daymon

• Donald K Wright
• Pubic Harold
• Michelle Drifka Hinson
• Corp Comm Int J
• Simone Byrd

• Brooke Fisher Liu

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

12th August 2024: digital purchasing is currently unavailable on Cambridge Core. Due to recent technical disruption affecting our publishing operation, we are experiencing some delays to publication. We are working hard to restore services as soon as possible and apologise for the inconvenience. For further updates please visit our website: https://www.cambridge.org/news-and-insights/technical-incident

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > Journals
• > Industrial and Organizational Psychology
• > Volume 14 Issue 3
• > Gaps in research and practice in social media-facilitated...

Article contents

Gaps in research and practice in social media-facilitated practices at work.

Published online by Cambridge University Press:  22 September 2021

Access options

Amber Schroeder is now affiliated with Amazon. All activities relating to this article were completed prior to her change in affiliation.

No CrossRef data available.

View all Google Scholar citations for this article.

Save article to Kindle

To save this article to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

• Volume 14, Issue 3
• Julia Hylton Whitaker (a1) , Amber Nicole Schroeder (a1) and Traci Megan Bricka (a1)
• DOI: https://doi.org/10.1017/iop.2021.83

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox .

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive .

Reply to: Submit a response

- No HTML tags allowed - Web page URLs will display as text only - Lines and paragraphs break automatically - Attachments, images or tables are not permitted

Your details

Your email address will be used in order to notify you when your comment has been reviewed by the moderator and in case the author(s) of the article or the moderator need to contact you directly.

You have entered the maximum number of contributors

Conflicting interests.

Please list any fees and grants from, employment by, consultancy for, shared ownership in or any close relationship with, at any time over the preceding 36 months, any organisation whose interests may be affected by the publication of the response. Please also list any non-financial associations or interests (personal, professional, political, institutional, religious or other) that a reasonable reader would want to know about in relation to the submitted work. This pertains to all the authors of the piece, their spouses or partners.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Social impact in social media: A new method to evaluate the social impact of research

Roles Investigation, Writing – original draft

* E-mail: [email protected]

Affiliation Department of Journalism and Communication Studies, Universitat Autonoma de Barcelona, Barcelona, Spain

Affiliation Department of Psychology and Sociology, Universidad de Zaragoza, Zaragoza, Spain

Roles Conceptualization, Investigation, Methodology, Supervision, Writing – review & editing

Affiliation Department of Sociology, Universitat Autonoma de Barcelona, Barcelona, Spain

Affiliation Department of Sociology, Universitat de Barcelona (UB), Barcelona, Spain

• Cristina M. Pulido, 
• Gisela Redondo-Sama, 
• Teresa Sordé-Martí, 
• Ramon Flecha

• Published: August 29, 2018
• https://doi.org/10.1371/journal.pone.0203117
• Reader Comments

The social impact of research has usually been analysed through the scientific outcomes produced under the auspices of the research. The growth of scholarly content in social media and the use of altmetrics by researchers to track their work facilitate the advancement in evaluating the impact of research. However, there is a gap in the identification of evidence of the social impact in terms of what citizens are sharing on their social media platforms. This article applies a social impact in social media methodology (SISM) to identify quantitative and qualitative evidence of the potential or real social impact of research shared on social media, specifically on Twitter and Facebook. We define the social impact coverage ratio (SICOR) to identify the percentage of tweets and Facebook posts providing information about potential or actual social impact in relation to the total amount of social media data found related to specific research projects. We selected 10 projects in different fields of knowledge to calculate the SICOR, and the results indicate that 0.43% of the tweets and Facebook posts collected provide linkages with information about social impact. However, our analysis indicates that some projects have a high percentage (4.98%) and others have no evidence of social impact shared in social media. Examples of quantitative and qualitative evidence of social impact are provided to illustrate these results. A general finding is that novel evidences of social impact of research can be found in social media, becoming relevant platforms for scientists to spread quantitative and qualitative evidence of social impact in social media to capture the interest of citizens. Thus, social media users are showed to be intermediaries making visible and assessing evidence of social impact.

Citation: Pulido CM, Redondo-Sama G, Sordé-Martí T, Flecha R (2018) Social impact in social media: A new method to evaluate the social impact of research. PLoS ONE 13(8): e0203117. https://doi.org/10.1371/journal.pone.0203117

Editor: Sergi Lozano, Institut Català de Paleoecologia Humana i Evolució Social (IPHES), SPAIN

Received: November 8, 2017; Accepted: August 15, 2018; Published: August 29, 2018

Copyright: © 2018 Pulido et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: The research leading to these results has received funding from the 7th Framework Programme of the European Commission under the Grant Agreement n° 613202 P.I. Ramon Flecha, https://ec.europa.eu/research/fp7/index_en.cfm . The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The social impact of research is at the core of some of the debates influencing how scientists develop their studies and how useful results for citizens and societies may be obtained. Concrete strategies to achieve social impact in particular research projects are related to a broader understanding of the role of science in contemporary society. There is a need to explore dialogues between science and society not only to communicate and disseminate science but also to achieve social improvements generated by science. Thus, the social impact of research emerges as an increasing concern within the scientific community [ 1 ]. As Bornmann [ 2 ] said, the assessment of this type of impact is badly needed and is more difficult than the measurement of scientific impact; for this reason, it is urgent to advance in the methodologies and approaches to measuring the social impact of research.

Several authors have approached the conceptualization of social impact, observing a lack of generally accepted conceptual and instrumental frameworks [ 3 ]. It is common to find a wide range of topics included in the contributions about social impact. In their analysis of the policies affecting land use, Hemling et al. [ 4 ] considered various domains in social impact, for instance, agricultural employment or health risk. Moving to the field of flora and fauna, Wilder and Walpole [ 5 ] studied the social impact of conservation projects, focusing on qualitative stories that provided information about changes in attitudes, behaviour, wellbeing and livelihoods. In an extensive study by Godin and Dore [ 6 ], the authors provided an overview and framework for the assessment of the contribution of science to society. They identified indicators of the impact of science, mentioning some of the most relevant weaknesses and developing a typology of impact that includes eleven dimensions, with one of them being the impact on society. The subdimensions of the impact of science on society focus on individuals (wellbeing and quality of life, social implication and practices) and organizations (speeches, interventions and actions). For the authors, social impact “refers to the impact knowledge has on welfare, and on the behaviours, practices and activities of people and groups” (p. 7).

In addition, the terms “social impact” and “societal impact” are sometimes used interchangeably. For instance, Bornmann [ 2 ] said that due to the difficulty of distinguishing social benefits from the superior term of societal benefits, “in much literature the term ‘social impact’ is used instead of ‘societal impact’”(p. 218). However, in other cases, the distinction is made [ 3 ], as in the present research. Similar to the definition used by the European Commission [ 7 ], social impact is used to refer to economic impact, societal impact, environmental impact and, additionally, human rights impact. Therefore, we use the term social impact as the broader concept that includes social improvements in all the above mentioned areas obtained from the transference of research results and representing positive steps towards the fulfilment of those officially defined social goals, including the UN Sustainable Development Goals, the EU 2020 Agenda, or similar official targets. For instance, the Europe 2020 strategy defines five priority targets with concrete indicators (employment, research and development, climate change and energy, education and poverty and social exclusion) [ 8 ], and we consider the targets addressed by objectives defined in the specific call that funds the research project.

This understanding of the social impact of research is connected to the creation of the Social Impact Open Repository (SIOR), which constitutes the first open repository worldwide that displays, cites and stores the social impact of research results [ 9 ]. The SIOR has linked to ORCID and Wikipedia to allow the synergies of spreading information about the social impact of research through diverse channels and audiences. It is relevant to mention that currently, SIOR includes evidence of real social impact, which implies that the research results have led to actual improvements in society. However, it is common to find evidence of potential social impact in research projects. The potential social impact implies that in the development of the research, there has been some evidence of the effectiveness of the research results in terms of social impact, but the results have not yet been transferred.

Additionally, a common confusion is found among the uses of dissemination, transference (policy impact) and social impact. While dissemination means to disseminate the knowledge created by research to citizens, companies and institutions, transference refers to the use of this knowledge by these different actors (or others), and finally, as already mentioned, social impact refers to the actual improvements resulting from the use of this knowledge in relation to the goals motivating the research project (such as the United Nations Sustainable Development Goals). In the present research [ 3 ], it is argued that “social impact can be understood as the culmination of the prior three stages of the research” (p.3). Therefore, this study builds on previous contributions measuring the dissemination and transference of research and goes beyond to propose a novel methodological approach to track social impact evidences.

In fact, the contribution that we develop in this article is based on the creation of a new method to evaluate the evidence of social impact shared in social media. The evaluation proposed is to measure the social impact coverage ratio (SICOR), focusing on the presence of evidence of social impact shared in social media. Then, the article first presents some of the contributions from the literature review focused on the research on social media as a source for obtaining key data for monitoring or evaluating different research purposes. Second, the SISM (social impact through social media) methodology[ 10 ] developed is introduced in detail. This methodology identifies quantitative and qualitative evidence of the social impact of the research shared on social media, specifically on Twitter and Facebook, and defines the SICOR, the social impact coverage ratio. Next, the results are discussed, and lastly, the main conclusions and further steps are presented.

Literature review

Social media research includes the analysis of citizens’ voices on a wide range of topics [ 11 ]. According to quantitative data from April 2017 published by Statista [ 12 ], Twitter and Facebook are included in the top ten leading social networks worldwide, as ranked by the number of active users. Facebook is at the top of the list, with 1,968 million active users, and Twitter ranks 10 th , with 319 million active users. Between them are the following social networks: WhatsApp, YouTube, Facebook Messenger, WeChat, QQ, Instagram,Qzone and Tumblr. If we look at altmetrics, the tracking of social networks for mentions of research outputs includes Facebook, Twitter, Google+,LinkedIn, Sina Weibo and Pinterest. The social networks common to both sources are Facebook and Twitter. These are also popular platforms that have a relevant coverage of scientific content and easy access to data, and therefore, the research projects selected here for application of the SISM methodology were chosen on these platforms.

Chew and Eysenbach [ 13 ] studied the presence of selected keywords in Twitter related to public health issues, particularly during the 2009 H1N1 pandemic, identifying the potential for health authorities to use social media to respond to the concerns and needs of society. Crooks et al.[ 14 ] investigated Twitter activity in the context of a 5.8 magnitude earthquake in 2011 on the East Coast of the United States, concluding that social media content can be useful for event monitoring and can complement other sources of data to improve the understanding of people’s responses to such events. Conversations among young Canadians posted on Facebook and analysed by Martinello and Donelle [ 15 ] revealed housing and transportation as main environmental concerns, and the project FoodRisc examined the role of social media to illustrate consumers’ quick responses during food crisis situations [ 16 ]. These types of contributions illustrate that social media research implies the understanding of citizens’ concerns in different fields, including in relation to science.

Research on the synergies between science and citizens has increased over the years, according to Fresco [ 17 ], and there is a growing interest among researchers and funding agencies in how to facilitate communication channels to spread scientific results. For instance, in 1998, Lubchenco [ 18 ] advocated for a social contract that “represents a commitment on the part of all scientists to devote their energies and talents to the most pressing problems of the day, in proportion to their importance, in exchange for public funding”(p.491).

In this framework, the recent debates on how to increase the impact of research have acquired relevance in all fields of knowledge, and major developments address the methods for measuring it. As highlighted by Feng Xia et al. [ 19 ], social media constitute an emerging approach to evaluating the impact of scholarly publications, and it is relevant to consider the influence of the journal, discipline, publication year and user type. The authors revealed that people’s concerns differ by discipline and observed more interest in papers related to everyday life, biology, and earth and environmental sciences. In the field of biomedical sciences, Haustein et al. [ 20 ] analysed the dissemination of journal articles on Twitter to explore the correlations between tweets and citations and proposed a framework to evaluate social media-based metrics. In fact, different studies address the relationship between the presence of articles on social networks and citations [ 21 ]. Bornmann [ 22 ] conducted a case study using a sample of 1,082 PLOS journal articles recommended in F1000 to explore the usefulness of altmetrics for measuring the broader impact of research. The author presents evidence about Facebook and Twitter as social networks that may indicate which papers in the biomedical sciences can be of interest to broader audiences, not just to specialists in the area. One aspect of particular interest resulting from this contribution is the potential to use altmetrics to measure the broader impacts of research, including the societal impact. However, most of the studies investigating social or societal impact lack a conceptualization underlying its measurement.

To the best of our knowledge, the assessment of social impact in social media (SISM) has developed according to this gap. At the core of this study, we present and discuss the results obtained through the application of the SICOR (social impact coverage ratio) with examples of evidence of social impact shared in social media, particularly on Twitter and Facebook, and the implications for further research.

Following these previous contributions, our research questions were as follows: Is there evidence of social impact of research shared by citizens in social media? If so, is there quantitative or qualitative evidence? How can social media contribute to identifying the social impact of research?

Methods and data presentation

A group of new methodologies related to the analysis of online data has recently emerged. One of these emerging methodologies is social media analytics [ 23 ], which was initially used most in the marketing research field but also came to be used in other domains due to the multiple possibilities opened up by the availability and richness of the data for different research purposes. Likewise, the concern of how to evaluate the social impact of research as well as the development of methodologies for addressing this concern has occupied central attention. The development of SISM (Social Impact in Social Media) and the application of the SICOR (Social Impact Coverage Ratio) is a contribution to advancement in the evaluation of the social impact of research through the analysis of the social media selected (in this case, Twitter and Facebook). Thus, SISM is novel in both social media analytics and among the methodologies used to evaluate the social impact of research. This development has been made under IMPACT-EV, a research project funded under the Framework Program FP7 of the Directorate-General for Research and Innovation of the European Commission. The main difference from other methodologies for measuring the social impact of research is the disentanglement between dissemination and social impact. While altmetrics is aimed at measuring research results disseminated beyond academic and specialized spheres, SISM contribute to advancing this measurement by shedding light on to what extent evidence of the social impact of research is found in social media data. This involves the need to differentiate between tweets or Facebook posts (Fb/posts) used to disseminate research findings from those used to share the social impact of research. We focus on the latter, investigating whether there is evidence of social impact, including both potential and real social impact. In fact, the question is whether research contributes and/or has the potential to contribute to improve the society or living conditions considering one of these goals defined. What is the evidence? Next, we detail the application of the methodology.

Data collection

To develop this study, the first step was to select research projects with social media data to be analysed. The selection of research projects for application of the SISM methodology was performed according to three criteria.

Criteria 1. Selection of success projects in FP7. The projects were success stories of the 7 th Framework Programme (FP7) highlighted by the European Commission [ 24 ] in the fields of knowledge of medicine, public health, biology and genomics. The FP7 published calls for project proposals from 2007 to 2013. This implies that most of the projects funded in the last period of the FP7 (2012 and 2013) are finalized or in the last phase of implementation.

Criteria 2. Period of implementation. We selected projects in the 2012–2013 period because they combine recent research results with higher possibilities of having Twitter and Facebook accounts compared with projects of previous years, as the presence of social accounts in research increased over this period.

Criteria 3. Twitter and Facebook accounts. It was crucial that the selected projects had active Twitter and Facebook accounts.

Table 1 summarizes the criteria and the final number of projects identified. As shown, 10 projects met the defined criteria. Projects in medical research and public health had higher presence.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0203117.t001

After the selection of projects, we defined the timeframe of social media data extraction on Twitter and Facebook from the starting date of the project until the day of the search, as presented in Table 2 .

https://doi.org/10.1371/journal.pone.0203117.t002

The second step was to define the search strategies for extracting social media data related to the research projects selected. In this line, we defined three search strategies.

Strategy 1. To extract messages published on the Twitter account and the Facebook page of the selected projects. We listed the Twitter accounts and Facebook pages related to each project in order to look at the available information. In this case, it is important to clarify that the tweets published under the corresponding Twitter project account are original tweets or retweets made from this account. It is relevant to mention that in one case, the Twitter account and Facebook page were linked to the website of the research group leading the project. In this case, we selected tweets and Facebook posts related to the project. For instance, in the case of the Twitter account, the research group created a specific hashtag to publish messages related to the project; therefore, we selected only the tweets published under this hashtag. In the analysis, we prioritized the analysis of the tweets and Facebook posts that received some type of interaction (likes, retweets or shares) because such interaction is a proxy for citizens’ interest. In doing so, we used the R program and NVivoto extract the data and proceed with the analysis. Once we obtained the data from Twitter and Facebook, we were able to have an overview of the information to be further analysed, as shown in Table 3 .

https://doi.org/10.1371/journal.pone.0203117.t003

We focused the second and third strategies on Twitter data. In both strategies, we extracted Twitter data directly from the Twitter Advanced Search tool, as the API connected to NVivo and the R program covers only a specific period of time limited to 7/9 days. Therefore, the use of the Twitter Advanced Search tool made it possible to obtain historic data without a period limitation. We downloaded the results in PDF and then uploaded them to NVivo.

Strategy 2. To use the project acronym combined with other keywords, such as FP7 or EU. This strategy made it possible to obtain tweets mentioning the project. Table 4 presents the number of tweets obtained with this strategy.

https://doi.org/10.1371/journal.pone.0203117.t004

Strategy 3. To use searchable research results of projects to obtain Twitter data. We defined a list of research results, one for each project, and converted them into keywords. We selected one searchable keyword for each project from its website or other relevant sources, for instance, the brief presentations prepared by the European Commission and published in CORDIS. Once we had the searchable research results, we used the Twitter Advanced Search tool to obtain tweets, as presented in Table 5 .

https://doi.org/10.1371/journal.pone.0203117.t005

The sum of the data obtained from these three strategies allowed us to obtain a total of 3,425 tweets and 1,925 posts on public Facebook pages. Table 6 presents a summary of the results.

https://doi.org/10.1371/journal.pone.0203117.t006

We imported the data obtained from the three search strategies into NVivo to analyse. Next, we select tweets and Facebook posts providing linkages with quantitative or qualitative evidence of social impact, and we complied with the terms of service for the social media from which the data were collected. By quantitative and qualitative evidence, we mean data or information that shows how the implementation of research results has led to improvements towards the fulfilment of the objectives defined in the EU2020 strategy of the European Commission or other official targets. For instance, in the case of quantitative evidence, we searched tweets and Facebook posts providing linkages with quantitative information about improvements obtained through the implementation of the research results of the project. In relation to qualitative evidence, for example, we searched for testimonies that show a positive evaluation of the improvement due to the implementation of research results. In relation to this step, it is important to highlight that social media users are intermediaries making visible evidence of social impact. Users often share evidence, sometimes sharing a link to an external resource (e.g., a video, an official report, a scientific article, news published on media). We identified evidence of social impact in these sources.

Data analysis

γ i is the total number of messages obtained about project i with evidence of social impact on social media platforms (Twitter, Facebook, Instagram, etc.);

T i is the total number of messages from project i on social media platforms (Twitter, Facebook, Instagram, etc.); and

n is the number of projects selected.

Analytical categories and codebook

The researchers who carried out the analysis of the social media data collected are specialists in the social impact of research and research on social media. Before conducting the full analysis, two aspects were guaranteed. First, how to identify evidence of social impact relating to the targets defined by the EU2020 strategy or to specific goals defined by the call addressed was clarified. Second, we held a pilot to test the methodology with one research project that we know has led to considerable social impact, which allowed us to clarify whether or not it was possible to detect evidence of social impact shared in social media. Once the pilot showed positive results, the next step was to extend the analysis to another set of projects and finally to the whole sample. The construction of the analytical categories was defined a priori, revised accordingly and lastly applied to the full sample.

Different observations should be made. First, in this previous analysis, we found that the tweets and Facebook users play a key role as “intermediaries,” serving as bridges between the larger public and the evidence of social impact. Social media users usually share a quote or paragraph introducing evidence of social impact and/or link to an external resource, for instance, a video, official report, scientific article, news story published on media, etc., where evidence of the social impact is available. This fact has implications for our study, as our unit of analysis is all the information included in the tweets or Facebook posts. This means that our analysis reaches the external resources linked to find evidence of social impact, and for this reason, we defined tweets or Facebook posts providing linkages with information about social impact.

Second, the other important aspect is the analysis of the users’ profile descriptions, which requires much more development in future research given the existing limitations. For instance, some profiles are users’ restricted due to privacy reasons, so the information is not available; other accounts have only the name of the user with no description of their profile available. Therefore, we gave priority to the identification of evidence of social impact including whether a post obtained interaction (retweets, likes or shares) or was published on accounts other than that of the research project itself. In the case of the profile analysis, we added only an exploratory preliminary result because this requires further development. Considering all these previous details, the codebook (see Table 7 ) that we present as follows is a result of this previous research.

https://doi.org/10.1371/journal.pone.0203117.t007

How to analyse Twitter and Facebook data

To illustrate how we analysed data from Twitter and Facebook, we provide one example of each type of evidence of social impact defined, considering both real and potential social impact, with the type of interaction obtained and the profiles of those who have interacted.

QUANESISM. Tweet by ZeroHunger Challenge @ZeroHunger published on 3 May 2016. Text: How re-using food waste for animal feed cuts carbon emissions.-NOSHAN project hubs.ly/H02SmrP0. 7 retweets and 5 likes.

The unit of analysis is all the content of the tweet, including the external link. If we limited our analysis to the tweet itself, it would not be evidence. Examining the external link is necessary to find whether there is evidence of social impact. The aim of this project was to investigate the process and technologies needed to use food waste for feed production at low cost, with low energy consumption and with a maximal evaluation of the starting wastes. This tweet provides a link to news published in the PHYS.org portal [ 25 ], which specializes in science news. The news story includes an interview with the main researcher that provides the following quotation with quantitative evidence:

'Our results demonstrated that with a NOSHAN 10 percent mix diet, for every kilogram of broiler chicken feed, carbon dioxide emissions were reduced by 0.3 kg compared to a non-food waste diet,' explains Montse Jorba, NOSHAN project coordinator. 'If 1 percent of total chicken broiler feed in Europe was switched to the 10 percent NOSHAN mix diet, the total amount of CO2 emissions avoided would be 0.62 million tons each year.'[ 25 ]

This quantitative evidence “a NOSHAN 10 percent mix diet, for every kilogram of broiler chicken feed, carbon dioxide emissions carbon dioxide emissions were reduced by 0.3 kg to a non-food waste diet” is linked directly with the Europe 2020 target of Climate Change & Energy, specifically with the target of reducing greenhouse gas emissions by 20% compared to the levels in 1990 [ 8 ]. The illustrative extrapolation the coordinator mentioned in the news is also an example of quantitative evidence, although is an extrapolation based on the specific research result.

This tweet was captured by the Acronym search strategy. It is a message tweeted by an account that is not related to the research project. The twitter account is that of the Zero Hunger Challenge movement, which supports the goals of the UN. The interaction obtained is 7 retweets and 5 likes. Regarding the profiles of those who retweeted and clicked “like”, there were activists, a journalist, an eco-friendly citizen, a global news service, restricted profiles (no information is available on those who have retweeted) and one account with no information in its profile.

The following example illustrates the analysis of QUALESISM: Tweet by @eurofitFP7 published on4 October 2016. Text: See our great new EuroFIT video on youtube! https://t.co/TocQwMiW3c 9 retweets and 5 likes.

The aim of this project is to improve health through the implementation of two novel technologies to achieve a healthier lifestyle. The tweet provides a link to a video on YouTube on the project’s results. In this video, we found qualitative evidence from people who tested the EuroFit programme; there are quotes from men who said that they have experienced improved health results using this method and that they are more aware of how to manage their health:

One end-user said: I have really amazing results from the start, because I managed to change a lot of things in my life. And other one: I was more conscious of what I ate, I was more conscious of taking more steps throughout the day and also standing up a little more. [ 26 ]

The research applies the well researched scientific evidence to the management of health issues in daily life. The video presents the research but also includes a section where end-users talk about the health improvements they experienced. The quotes extracted are some examples of the testimonies collected. All agree that they have improved their health and learned healthy habits for their daily lives. These are examples of qualitative evidence linked with the target of the call HEALTH.2013.3.3–1—Social innovation for health promotion [ 27 ] that has the objectives of reducing sedentary habits in the population and promoting healthy habits. This research contributes to this target, as we see in the video testimonies. Regarding the interaction obtained, this tweet achieved 9 retweets and 5 likes. In this case, the profiles of the interacting citizens show involvement in sport issues, including sport trainers, sport enthusiasts and some researchers.

To summarize the analysis, in Table 8 below, we provide a summary with examples illustrating the evidence found.

https://doi.org/10.1371/journal.pone.0203117.t008

Quantitative evidence of social impact in social media

There is a greater presence of tweets/Fb posts with quantitative evidence (14) than with qualitative evidence (9) in the total number of tweets/Fb posts identified with evidence of social impact. Most of the tweets/Fb posts with quantitative evidence of social impact are from scientific articles published in peer-reviewed international journals and show potential social impact. In Table 8 , we introduce 3 examples of this type of tweets/Fb posts with quantitative evidence:

The first tweet with quantitative social impact selected is from project 7. The aim of this project was to provide high-quality scientific evidence for preventing vitamin D deficiency in European citizens. The tweet highlighted the main contribution of the published study, that is, “Weekly consumption of 7 vitamin D-enhanced eggs has an important impact on winter vitamin D status in adults” [ 28 ]. The quantitative evidence shared in social media was extracted from a news publication in a blog on health news. This blog collects scientific articles of research results. In this case, the blog disseminated the research result focused on how vitamin D-enhanced eggs improve vitamin D deficiency in wintertime, with the published results obtained by the research team of the project selected. The quantitative evidence illustrates that the group of adults who consumed vitamin D-enhanced eggs did not suffer from vitamin D deficiency, as opposed to the control group, which showed a significant decrease in vitamin D over the winter. The specific evidence is the following extracted from the article [ 28 ]:

With the use of a within-group analysis, it was shown that, although serum 25(OH) D in the control group significantly decreased over winter (mean ± SD: -6.4 ± 6.7 nmol/L; P = 0.001), there was no change in the 2 groups who consumed vitamin D-enhanced eggs (P>0.1 for both. (p. 629)

This evidence contributes to achievement of the target defined in the call addressed that is KBBE.2013.2.2–03—Food-based solutions for the eradication of vitamin D deficiency and health promotion throughout the life cycle [ 29 ]. The quantitative evidence shows how the consumption of vitamin D-enhanced eggs reduces vitamin D deficiency.

The second example of this table corresponds to the example of quantitative evidence of social impact provided in the previous section.

The third example is a Facebook post from project 3 that is also tweeted. Therefore, this evidence was published in both social media sources analysed. The aim of this project was to measure a range of chemical and physical environmental hazards in food, consumer products, water, air, noise, and the built environment in the pre- and postnatal early-life periods. This Facebook post and tweet links directly to a scientific article [ 30 ] that shows the precision of the spectroscopic platform:

Using 1H NMR spectroscopy we characterized short-term variability in urinary metabolites measured from 20 children aged 8–9 years old. Daily spot morning, night-time and pooled (50:50 morning and night-time) urine samples across six days (18 samples per child) were analysed, and 44 metabolites quantified. Intraclass correlation coefficients (ICC) and mixed effect models were applied to assess the reproducibility and biological variance of metabolic phenotypes. Excellent analytical reproducibility and precision was demonstrated for the 1H NMR spectroscopic platform (median CV 7.2%) . (p.1)

This evidence is linked to the target defined in the call “ENV.2012.6.4–3—Integrating environmental and health data to advance knowledge of the role of environment in human health and well-being in support of a European exposome initiative” [ 31 ]. The evidence provided shows how the project’s results have contributed to building technology for improving the data collection to advance in the knowledge of the role of the environment in human health, especially in early life. The interaction obtained is one retweet from a citizen from Nigeria interested in health issues, according to the information available in his profile.

Qualitative evidence of social impact in social media

We found qualitative evidence of the social impact of different projects, as shown in Table 9 . Similarly to the quantitative evidence, the qualitative cases also demonstrate potential social impact. The three examples provided have in common that they are tweets or Facebook posts that link to videos where the end users of the research project explain their improvements once they have implemented the research results.

https://doi.org/10.1371/journal.pone.0203117.t009

The first tweet with qualitative evidence selected is from project 4. The aim of this project is to produce a system that helps in the prevention of obesity and eating disorders, targeting young people and adults [ 32 ]. The twitter account that published this tweet is that of the Future and Emerging Technologies Programme of the European Commission, and a link to a Euronews video is provided. This video shows how the patients using the technology developed in the research achieved control of their eating disorders, through the testimonies of patients commenting on the positive results they have obtained. These testimonies are included in the news article that complements the video. An example of these testimonies is as follows:

Pierre Vial has lost 43 kilos over the past nine and a half months. He and other patients at the eating disorder clinic explain the effects obesity and anorexia have had on their lives. Another patient, Karin Borell, still has some months to go at the clinic but, after decades of battling anorexia, is beginning to be able to visualise life without the illness: “On a good day I see myself living a normal life without an eating disorder, without problems with food. That’s really all I wish right now”.[ 32 ]

This qualitative evidence shows how the research results contribute to the achievement of the target goals of the call addressed:“ICT-2013.5.1—Personalised health, active ageing, and independent living”. [ 33 ] In this case, the results are robust, particularly for people suffering chronic diseases and desiring to improve their health; people who have applied the research findings are improving their eating disorders and better managing their health. The value of this evidence is the inclusion of the patients’ voices stating the impact of the research results on their health.

The second example is a Facebook post from project 9, which provides a link to a Euronews video. The aim of this project is to bring some tools from the lab to the farm in order to guarantee a better management of the farm and animal welfare. In this video [ 34 ], there are quotes from farmers using the new system developed through the research results of the project. These quotes show how use of the new system is improving the management of the farm and the health of the animals; some examples are provided:

Cameras and microphones help me detect in real time when the animals are stressed for whatever reason,” explained farmer Twan Colberts. “So I can find solutions faster and in more efficient ways, without me being constantly here, checking each animal.”

This evidence shows how the research results contribute to addressing the objectives specified in the call “KBBE.2012.1.1–02—Animal and farm-centric approach to precision livestock farming in Europe” [ 29 ], particularly, to improve the precision of livestock farming in Europe. The interaction obtained is composed of6 likes and 1 share. The profiles are diverse, but some of them do not disclose personal information; others have not added a profile description, and only their name and photo are available.

Interrater reliability (kappa)

The analysis of tweets and Facebook posts providing linkages with information about social impact was conducted following a content analysis method in which reliability was based on a peer review process. This sample is composed of 3,425 tweets and 1,925 Fb/posts. Each tweet and Facebook post was analysed to identify whether or not it contains evidence of social impact. Each researcher has the codebook a priori. We used interrater reliability in examining the agreement between the two raters on the assignment of the categories defined through Cohen’s kappa. We used SPSS to calculate this coefficient. We exported an excel sheet with the sample coded by the two researchers being 1 (is evidence of social impact, either potential or real) and 0 (is not evidence of social impact) to SPSS. The cases where agreement was not achieved were not considered as containing evidence of social impact. The result obtained is 0.979; considering the interpretation of this number according to Landis & Koch [ 35 ], our level of agreement is almost perfect, and thus, our analysis is reliable. To sum up the data analysis, the description of the steps followed is explained:

Step 1. Data analysis I. We included all data collected in an excel sheet to proceed with the analysis. Prior to the analysis, researchers read the codebook to keep in mind the information that should be identified.

Step 2. Each researcher involved reviewed case by case the tweets and Facebook posts to identify whether they provide links with evidence of social impact or not. If the researcher considers there to be evidence of social impact, he or she introduces the value of 1into the column, and if not, the value of 0.

Step 3. Once all the researchers have finished this step, the next step is to export the excel sheet to SPSS to extract the kappa coefficient.

Step 4. Data Analysis II. The following step was to analyse case by case the tweets and Facebook posts identified as providing linkages with information of social impact and classify them as quantitative or qualitative evidence of social impact.

Step 5. The interaction received was analysed because this determines to which extent this evidence of social impact has captured the attention of citizens (in the form of how many likes, shares, or retweets the post has).

Step 6. Finally, if available, the profile descriptions of the citizens interacting through retweeting or sharing the Facebook post were considered.

Step 7. SICOR was calculated. It could be applied to the complete sample (all data projects) or to each project, as we will see in the next section.

The total number of tweets and Fb/posts collected from the 10 projects is 5,350. After the content analysis, we identified 23 tweets and Facebook posts providing linkages to information about social impact. To respond to the research question, which considered whether there is evidence of social impact shared by citizens in social media, the answer was affirmative, although the coverage ratio is low. Both Twitter and Facebook users retweeted or shared evidence of social impact, and therefore, these two social media networks are valid sources for expanding knowledge on the assessment of social impact. Table 10 shows the social impact coverage ratio in relation to the total number of messages analysed.

https://doi.org/10.1371/journal.pone.0203117.t010

The analysis of each of the projects selected revealed some results to consider. Of the 10 projects, 7 had evidence, but those projects did not necessarily have more Tweets and Facebook posts. In fact, some projects with fewer than 70 tweets and 50 Facebook posts have more evidence of social impact than other projects with more than 400 tweets and 400 Facebook posts. This result indicates that the number of tweets and Facebook posts does not determine the existence of evidence of social impact in social media. For example, project 2 has 403 tweets and 423 Facebooks posts, but it has no evidence of social impact on social media. In contrast, project 9 has 62 tweets, 43 Facebook posts, and 2 pieces of evidence of social impact in social media, as shown in Table 11 .

https://doi.org/10.1371/journal.pone.0203117.t011

The ratio of tweets/Fb posts to evidence is 0.43%, and it differs depending on the project, as shown below in Table 12 . There is one project (P7) with a ratio of 4.98%, which is a social impact coverage ratio higher than that of the other projects. Next, a group of projects (P3, P9, P10) has a social impact coverage ratio between 1.41% and 2,99%.The next slot has three projects (P1, P4, P5), with a ratio between 0.13% and 0.46%. Finally, there are three projects (P2, P6, P8) without any tweets/Fb posts evidence of social impact.

https://doi.org/10.1371/journal.pone.0203117.t012

Considering the three strategies for obtaining data, each is related differently to the evidence of social impact. In terms of the social impact coverage ratio, as shown in Table 13 , the most successful strategy is number 3 (searchable research results), as it has a relation of 17.86%, which is much higher than the ratios for the other 2 strategies. The second strategy (acronym search) is more effective than the first (profile accounts),with 1.77% for the former as opposed to 0.27% for the latter.

https://doi.org/10.1371/journal.pone.0203117.t013

Once tweets and Facebook posts providing linkages with information about social impact(ESISM)were identified, we classified them in terms of quantitative (QUANESISM) or qualitative evidence (QUALESISM)to determine which type of evidence was shared in social media. Table 14 indicates the amount of quantitative and qualitative evidence identified for each search strategy.

https://doi.org/10.1371/journal.pone.0203117.t014

First, the results obtained indicated that the SISM methodology aids in calculating the social impact coverage ratio of the research projects selected and evaluating whether the social impact of the corresponding research is shared by citizens in social media. The social impact coverage ratio applied to the sample selected is low, but when we analyse the SICOR of each project separately, we can observe that some projects have a higher social impact coverage ratio than others. Complementary to altmetrics measuring the extent to which research results reach out society, the SICOR considers the question whether this process includes evidence of potential or real social impact. In this sense, the overall methodology of SISM contributes to advancement in the evaluation of the social impact of research by providing a more precise approach to what we are evaluating.

This contribution complements current evaluation methodologies of social impact that consider which improvements are shared by citizens in social media. Exploring the results in more depth, it is relevant to highlight that of the ten projects selected, there is one research project with a social impact coverage ratio higher than those of the others, which include projects without any tweets or Facebook posts with evidence of social impact. This project has a higher ratio of evidence than the others because evidence of its social impact is shared more than is that of other projects. This also means that the researchers produced evidence of social impact and shared it during the project. Another relevant result is that the quantity of tweets and Fb/posts collected did not determine the number of tweets and Fb/posts found with evidence of social impact. Moreover, the analysis of the research projects selected showed that there are projects with less social media interaction but with more tweets and Fb/posts containing evidence of social media impact. Thus, the number of tweets and Fb/posts with evidence of social impact is not determined by the number of publication messages collected; it is determined by the type of messages published and shared, that is, whether they contain evidence of social impact or not.

The second main finding is related to the effectiveness of the search strategies defined. Related to the strategies carried out under this methodology, one of the results found is that the most effective search strategy is the searchable research results, which reveals a higher percentage of evidence of social impact than the own account and acronym search strategies. However, the use of these three search strategies is highly recommended because the combination of all of them makes it possible to identify more tweets and Facebook posts with evidence of social impact.

Another result is related to the type of evidence of social impact found. There is both quantitative and qualitative evidence. Both types are useful for understanding the type of social impact achieved by the corresponding research project. In this sense, quantitative evidence allows us to understand the improvements obtained by the implementation of the research results and capture their impact. In contrast, qualitative evidence allows us to deeply understand how the resultant improvements obtained from the implementation of the research results are evaluated by the end users by capturing their corresponding direct quotes. The social impact includes the identification of both real and potential social impact.

Conclusions

After discussing the main results obtained, we conclude with the following points. Our study indicates that there is incipient evidence of social impact, both potential and real, in social media. This demonstrates that researchers from different fields, in the present case involved in medical research, public health, animal welfare and genomics, are sharing the improvements generated by their research and opening up new venues for citizens to interact with their work. This would imply that scientists are promoting not only the dissemination of their research results but also the evidence on how their results may lead to the improvement of societies. Considering the increasing relevance and presence of the dissemination of research, the results indicate that scientists still need to include in their dissemination and communication strategies the aim of sharing the social impact of their results. This implies the publication of concrete qualitative or quantitative evidence of the social impact obtained. Because of the inclusion of this strategy, citizens will pay more attention to the content published in social media because they are interested in knowing how science can contribute to improving their living conditions and in accessing crucial information. Sharing social impact in social media facilitates access to citizens of different ages, genders, cultural backgrounds and education levels. However, what is most relevant for our argument here is how citizens should also be able to participate in the evaluation of the social impact of research, with social media a great source to reinforce this democratization process. This contributes not only to greatly improving the social impact assessment, as in addition to experts, policy makers and scientific publications, citizens through social media contribute to making this assessment much more accurate. Thus, citizens’ contribution to the dissemination of evidence of the social impact of research yields access to more diverse sectors of society and information that might be unknown by the research or political community. Two future steps are opened here. On the one hand, it is necessary to further examine the profiles of users who interact with this evidence of social impact considering the limitations of the privacy and availability of profile information. A second future task is to advance in the articulation of the role played by citizens’ participation in social impact assessment, as citizens can contribute to current worldwide efforts by shedding new light on this process of social impact assessment and contributing to making science more relevant and useful for the most urgent and poignant social needs.

Supporting information

S1 file. interrater reliability (kappa) result..

This file contains the SPSS file with the result of the calculation of Cohen’s Kappa regards the interrater reliability. The word document exported with the obtained result is also included.

https://doi.org/10.1371/journal.pone.0203117.s001

S2 File. Data collected and SICOR calculation.

This excel contains four sheets, the first one titled “data collected” contains the number of tweets and Facebook posts collected through the three defined search strategies; the second sheet titled “sample” contains the sample classified by project indicating the ID of the message or code assigned, the type of message (tweet or Facebook post) and the codification done by researchers being 1 (is evidence of social impact, either potential or real) and 0 (is not evidence of social impact); the third sheet titled “evidence found” contains the number of type of evidences of social impact founded by project (ESISM-QUANESIM or ESISM-QUALESIM), search strategy and type of message (tweet or Facebook posts); and the last sheet titled “SICOR” contains the Social Impact Coverage Ratio calculation by projects in one table and type of search strategy done in another one.

https://doi.org/10.1371/journal.pone.0203117.s002

Acknowledgments

The research leading to these results received funding from the 7 th Framework Programme of the European Commission under Grant Agreement n° 613202. The extraction of available data using the list of searchable keywords on Twitter and Facebook followed the ethical guidelines for social media research supported by the Economic and Social Research Council (UK) [ 36 ] and the University of Aberdeen [ 37 ]. Furthermore, the research results have already been published and made public, and hence, there are no ethical issues.

• View Article
• PubMed/NCBI
• Google Scholar
• 6. Godin B, Dore C. Measuring the impacts of science; beyond the economic dimension. INRS Urbanisation, Culture et Sociult, HSIT Lecture. 2005. Helsinki, Finland: Helsinki Institute for Science and Technology Studies.
• 7. European Commission. Better regulation Toolbox. 2017: 129–130. Available from https://ec.europa.eu/info/sites/info/files/better-regulation-toolbox_0.pdf
• 8. European 2020 Strategy. 2010. Available from https://ec.europa.eu/info/business-economy-euro/economic-and-fiscal-policy-coordination/eu-economic-governance-monitoring-prevention-correction/european-semester/framework/europe-2020-strategy_en
• 10. Flecha R, Sordé-Martí T. SISM Methodology (Social Impact through Social Media).2016. Available from https://archive.org/details/SISMMethodology
• 12. Statista. Statistics and facts about social media usage. 2017. Available from https://www.statista.com/topics/1164/social-networks/
• 16. FoodRisc Consortium, European Commission, Main researcher: Patrick Wall. FoodRisc—Result in Brief (Seventh Framework Programme). 2017. Available from http://cordis.europa.eu/result/rcn/90678_en.html
• 23. Murphy, L. Grit Report. Greenbook research industry trends report. Grit Q3-Q4 2016; (Vol. 3–4). New York. Retrieved from https://www.greenbook.org/grit
• 24. EC Research & Innovation. Success Stories [Internet]. 2017 [cited 2017 Apr 25]. Available from: http://ec.europa.eu/research/infocentre/success_stories_en.cfm
• 25. PhysOrg. How re-using food waste for animal feed cuts carbon emissions. 2016. Available from: https://phys.org/news/2016-04-re-using-food-animal-carbon-emissions.html
• 26. EuroFIT. EuroFIT International PP Youtube. 2016; minute 2.05–2.20. Available from: https://www.youtube.com/watch?time_continue=155&v=CHkbnD8IgZw
• 27. European Commission<. Work Programme 2013. Cooperation Theme 1 Health. (2013). Available from: https://ec.europa.eu/research/participants/portal/doc/call/fp7/common/1567645-1._health_upd_2013_wp_27_june_2013_en.pdf
• 29. European Commission. Work Programme 2013. Cooperation Theme 2. Food, agriculture, and fisheries, and biotechnology. (2013). Available from: http://ec.europa.eu/research/participants/data/ref/fp7/192042/b-wp-201302_en.pdf
• 31. European Commission. Work Programme 2012. Cooperation Theme 6 Environment (including climate change). (2011). Available from: http://ec.europa.eu/research/participants/data/ref/fp7/89467/f-wp-201201_en.pdf
• 32. Research Information Center. Technology trialled in fight against ticking timebomb of obesity. 2016. Available from: http://www.euronews.com/2016/08/10/technology-trialled-in-fight-against-ticking-timebomb-of-obesity
• 33. European Commission. Work Programme 2013. Cooperation Theme 3. ICT (Information and Communication Technologies) (2012). Available from: http://ec.europa.eu/research/participants/data/ref/fp7/132099/c-wp-201301_en.pdf
• 34. Euronews. Big farmer is watching! Surveillance technology monitors animal wellbeing. 2016. Available from http://www.euronews.com/2016/05/09/big-farmer-is-watching-surveillance-technology-monitors-animal-wellbeing
• 36. Economic and Social Research Council. Social Media Best Practice and Guidance. Using Social Media. 2017. Available from: http://www.esrc.ac.uk/research/impact-toolkit/social-media/using-social-media/
• 37. Townsend, L. & Wallace, C. Social Media Research: A Guide to Ethics. 2016. Available from http://www.gla.ac.uk/media/media_487729_en.pdf