Exploring the diverse definitions of ‘evidence’: a scoping review

Authors: Xuan Yu, Shouyuan Wu, Yajia Sun, Ping Wang, Ling Wang, Renfeng Su, Junxian Zhao, Racha Fadlallah, Laura Boeira, Sandy Oliver, Yoseph G Abraha, Nelson K Sewankambo, Fadi El-Jardali, Susan L Norris, Yaolong Chen

Abstract

Objectives

To systematically collect and analyse diverse definitions of ‘evidence’ in both health and social sciences, and help users to correctly use the term ‘evidence’ and rethink what is the definition of ‘evidence’ in scientific research.

Design

Scoping review.

Methods

Definitions of evidence in the health sciences and social sciences were included. We have excluded the definition of evidence applied in the legal field, abstracts without full text, documents not published in either Chinese or English and so on. We established a multidisciplinary working group and systematically searched five electronic databases including Medline, Web of Science, EBSCO, the Chinese Social Sciences Citation Index and the Chinese Science Citation Database from their inception to 26 February 2022. We also searched websites and reviewed the reference lists of the identified studies. Six reviewers working in pairs, independently, selected studies according to the inclusion and exclusion criteria, and extracted information. Any differences were discussed in pairs, and if there was disagreement, it was resolved via discussion or with the help of a third reviewer. Reviewers extracted document characteristics, the original content for the definitions of ‘evidence’, assessed definitions as either intensional or extensional, and any citations for the given definition.

Results

Forty-nine documents were finally included after screening, and 68 definitions were obtained. After excluding duplicates, a total of 54 different definitions of ‘evidence’ were identified. There were 42 intensional definitions and 12 extensional definitions. The top three definiens were ‘information’, ‘fact’ and ‘research/study’. The definition of ‘evidence’ differed between health and social sciences. The term ‘research’ appeared most frequently in the definitions.

Conclusions

The definition of ‘evidence’ has gradually attracted the attention of many scholars and decision-makers in health and social sciences. Nevertheless, there is no widely recognised and accepted definition in scientific research. Given the wide use of the term, we need to think about whether, or under what circumstances, a standardised, clear, meaningful and widely applicable definition of ‘evidence’ might be helpful.

Introduction

The term ‘evidence’ is widely used in today’s society. Reviewing The New England Journal of Medicine, The Lancet and The BMJ, it is not difficult to find that the term ‘evidence’ has been used in scientific research articles since its publication, and the use is gradually increasing in scholarly papers.1 In addition to researchers, the public also use evidence to make judgements and decisions in their daily life. In Europe and the USA, beginning in the 18th century, and escalating in the 20th century, increasing attention was given to the empirical underpinnings of rational decision-making in medicine, particularly with respect to therapeutics.2 3 By the 1990s, this concern was crystallised as an attempt to promote ‘evidence-based medicine’, explicitly juxtaposed to authority-based (‘eminence-based’) medicine.4 At these points, evidence-based medicine was born5; evidence-based health sciences and evidence-based social sciences have since gradually emerged with a similar emphasis on ‘evidence’ as the foundation for scientific discussions and decision-making.6–8

A ‘definition’ is an accurate description of a concept or the meaning of a word without changing the object itself. A definition is composed of two the word or expression being defined (referred to by linguists as the ‘definiendum’) and the words or concepts used in the definition that are supposed to have the same meaning as the definiendum (the ‘definiens’).9 Definition helps to clarify the position and boundary of items in a comprehensive classification system and helps people determine the scope and attributes of objects. Although there are several types of definitions, it is more common to categorise definitions into intensional and extensional definitions.10 An intensional definition presents the features that characterise a category, while an extensional definition presents members of that category.11 12

Nowadays, the term ‘evidence’ is defined differently across widely used dictionaries and their definiens are not the same.13–20 Given that medicine is both a social and scientific activity, health sciences is closely interconnected with social sciences. However, compared with the definition of ‘evidence’ in the legal field, the definition of ‘evidence’ in the health and social sciences displays more diversity.21–24 In recent years, researchers and decision-makers have not only affirmed evidence-based medicine, but also pointed out its limitations, one of which is the restricted view of evidence.25 26 In order to optimally promote and apply evidence-based practices and evidence-informed decision-making, it is imperative for decision-makers, researchers, practitioners, evaluators, and the public to have a clear understanding and a wide range of what defines ‘evidence’ in scientific research.

To the best of our knowledge, there is no comprehensive and systematic collection, and analysis of the diverse definitions of ‘evidence’ in the field of both health and social sciences. This scoping review aims to fill this gap, trying to find related definitions of ‘evidence’ and analysing the differences and connections between different definitions.

Materials and methods

This scoping review was based on the methods outlined in the framework proposed by Arksey and O’Malley and extended by Levac et al.27 28 We report our findings according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist.29 The research protocol was registered on the Open Science Framework platform (https://osf.io/j3urd).

Identification of the research question

The main research question in this scoping review ‘What are the definitions of ‘evidence’ used in health sciences and social sciences?’

Search strategy

We comprehensively searched five electronic bibliographic databases including Medline (via PubMed), Web of Science, the Chinese Social Sciences Citation Index (CSSCI), the Chinese Science Citation Database (CSCD) and EBSCO. The latter database includes Academic Search Premier, Business Source Premier, Education Resource Information Center (ERIC), GreenFILE, Library, Information Science & Technology Abstracts, Newspaper Source, Regional Business News, Teacher Reference Center, European Views of the Americas: 1493–1750, eBook Collection (EBSCOhost), EBSCO eClassics Collection (EBSCOhost), Open Dissertations, and The Belt and Road Initiative Reference Source. All databases were searched from their individual inception to 26 February 2022, using the terms “evidence”, “definition”, “term”, “glossary” and their derivatives. The search was carried out by combining the relevant MeSH terms and free-text words. For detailed search strategies for each database, please see online supplemental material.

Concurrently, we searched Google Scholar (https://scholar.google.com/), Google (https://www.google.com.hk/advanced_search), Baidu (https://www.baidu.com/gaoji/advanced.html) and Baidu Academic (https://xueshu.baidu.com/), using the same search terms. Baidu is a Chinese internet search engine primarily used in China, and Baidu Academic is a related, free academic resource search platform. We browsed the first 1000 records of the search results using these platforms, seeking documents that met inclusion criteria. In addition, we reviewed the reference lists of the identified studies for further potential documents. Finally, the Working Group further supplemented relevant documents from their own knowledge.

Inclusion and exclusion criteria

Definitions of evidence in the health sciences and social sciences were included. Definitions were sourced (1) reports published on websites of governmental organisations (such as the United Nations or the WHO), academic institutions or professional organisations and explicitly mentioned the term ‘evidence’; (2) websites, reports, books or literature on the evidence-based medicine and evidence-based social sciences and (3) publications in peer-reviewed journals for which the title of the paper explicitly mentions a discussion of what is ‘evidence’.

The exclusion criteria were as (1) the definition of evidence applied in the legal field; (2) conference abstracts without full text; (3) documents not published in either English or Chinese; (4) Chinese translations of English documents; (5) the full text was not available and (6) duplicate documents.

Study selection and data extraction

After eliminating duplicates, six reviewers independently working in pairs (group SW and RS; Group YS and JZ; group PW and LW) selected studies using a two-stage process. They first screened the titles and abstracts of studies and included any study judged as potentially eligible by at least one of the reviewers. They then assessed the full text of the selected studies in detail for eligibility. Disagreements were resolved through discussion or with the help of a third reviewer (XY) if consensus could not be achieved. All reasons for excluding ineligible studies were recorded, and the process of study selection was documented using a PRISMA flow diagram. EndNote X9.3.3 (https://endnote.com/) was used for document tracking and selection.

Six reviewers working in pairs extracted information independently and then compared the extractions. Any differences were discussed and if there was disagreement, it was resolved via discussion with a third reviewer (XY). The following information was (1) document characteristics, such as document source, publication year and discipline. In this study, the discipline ‘health sciences’ includes the study of medicine, nutrition and other health-related topics; ‘Social sciences’ includes any branch of academic study or science that deals with human behaviour in its social and cultural aspects; (2) the definitions of ‘evidence’ and (3) any citations for the given definition. Data were extracted using Microsoft Excel 2019 software. For the definition of ‘evidence’, the original content of the definition was extracted verbatim. Each definition was decomposed into individual keywords, which were summarised and classified. Moreover, the definition was assessed as either intensional or extensional, or a combination thereof.

Data analysis

The analysis was descriptive. A citation analysis diagram was used to represent the relationships and numbers of citations. Microsoft Excel 2019 was used to develop all tables and figures. We deduplicated all the extracted definitions by first excluding definitions cited in included documents that had later publication dates. Second, we excluded definitions with duplicate sentence expressions, keeping the earlier-published definitions.

For each unique definition of evidence, we performed a two-stage classification. In the first stage, we classified the definition according to the definiens. In the second stage, we classified the definition according to the attributive modifier of the definiens. For example, in the definition of ‘evidence is the result of research’, we considered ‘result’ as the definiens and ‘research’ as the attribute modifier.

Results

Document screening process and results

A total of 8107 relevant documents were initially identified via 5 main databases and 41 documents were identified via other methods. Forty-nine documents were finally included after screening (14 journal articles were identified from databases and 35 via the search of reference lists and websites). See figure 1 for the process and results of document screening. For excluded documents after full-text review, please see online supplemental material.

Characteristics of included documents

Of the 49 documents that met inclusion criteria, the majority were journal articles (n=28, 57.2%). The number of documents increased steadily since 1996, with more than 20 documents (25, 51.0%) published from 2011 to 2020. More than half of the documents discussed the definition of ‘evidence’ in the health sector (29, 59.2%). See table 1 for details.

Definitions of evidence

From the 49 included documents, we obtained 68 definitions of ‘evidence’. After deduplication, a total of 54 different definitions of ‘evidence’ were identified. There were 42 intensional definitions (see online supplemental table 1), including 14 information (16), research/study (9), fact (9), knowledge (4), data (4), observation (2), result (2), tool (1), ability (1), testimony (1), judgement (1), belief (1), idea (1) and reason (1). There were also 12 extensional definitions which included definiens mentioned in the intensional definitions, as well as cost-effectiveness analysis, context, public perceptions, economic and statistical modelling, stakeholder opinions, media data, expert knowledge and others (see online supplemental table 2). The term ‘research’ appeared most frequently in 25 definitions (46.3%), followed by ‘information’ (19, 35.2%) and ‘fact’ (10, 18.5%).

There was little difference in the number of definitions proposed in the field of health sciences and in that of social sciences (n=29 in health sciences vs n=25 in social Sciences); however, the definition of ‘evidence’ differed between the two disciplines. In the field of health sciences, more scholars believe that evidence is ‘research/study’ (n=5) and ‘information’ (n=5), while in the social sciences, most scholars believe that evidence is ‘information’ (n=11). Moreover, only health sciences scholars think that evidence is ‘observation’, ‘ability’, ‘testimony’, ‘judgement’ and ‘belief’; and only social sciences scholars mention evidence is ‘tool’, ‘idea’ and ‘reason’. See table 2 for details.

The definition of ‘evidence’ by Sackett et al was cited most frequently,30 appearing in more than three references. On the other hand, Chen et al’s document was most frequently cited in other records (more than four times).31 The majority of studies were cited only once or cited one pre-existing document. Interestingly, most of the research published in Chinese cited the definition of ‘evidence’ published in English; however, none of the research published in English cited the definition of ‘evidence’ published in Chinese.

Discussion

This scoping review systematically collected diverse definitions of the term ‘evidence’ from journals, websites, reports and books that were used in the fields of health and social sciences. These definitions varied widely within and across disciplines. The majority of definitions were intensional and the most frequent definiens was ‘information’, followed by ‘fact’ and ‘research/study’. The term ‘research’ appeared most frequently in the definitions of ‘evidence’.

‘Information’ was the most frequent definiens in the intensional definitions, and usually refers to the objects transmitted and processed by audio, message and communication systems, in other words, all types of things used for communication in human society.32 Information itself is thus a very broad concept, so the definition of evidence as ‘information’ looks equally broad. In addition, different scholars limit or explain information from specific perspectives when defining ‘evidence’. Many definitions purport that information can be considered evidence when it is systematically collected or evaluated, or has truth and validity, or can support conclusions and/or decisions. There may be potential links across these concepts. For example, the information that is systematically collected or evaluated may have truth and validity, and information with truth and validity may be used to support conclusions and/or decisions.

In addition to ‘information’, ‘fact’, ‘research’, ‘knowledge’ and ‘result’ have also been important definiens in the definition of ‘evidence’. ‘Fact’ indicates a thing that is known to be true, especially when it can be proven33; ‘knowledge’ denotes the sum of the results of human exploration of the material world as well as the spiritual world, in line with the direction of civilisation34; and ‘result’ indicates a thing that is caused or produced because of something else.35 From these definitions, it is obvious that there is an overlap among these terms. For example, ‘knowledge’ belongs to ‘information’ or ‘result’, and ‘knowledge’ and ‘fact’ emphasise the truth and value of events. At the same time, these definientia appear more frequently in definitions. Among them, the classification of ‘research’ is particularly remarkable. While it is classified as a definiens, many secondary classifications also include ‘research’. Most scholars also believe that ‘evidence’ is ‘research’; however, there are different perspectives. Some scholars believe that only conclusions of research constitute ‘evidence’, while others believe that the information generated during research is also the ‘evidence’. The scope of the latter perspective is broader; however, it looks like the feasibility and understandability of the former perspective in the implementation and application process are stronger.

Interconnections can also be observed between definiens and their attributable modifiers in the definitions of ‘evidence’. These attributable modifiers emphasise whether they are systematically and/or unsystematically collected, used to support conclusions or decision-making, or require the best available and true and valid. We assume that this situation may be influenced by the development of evidence-based medicine and its associated methodologies. For example, the evolving hierarchy of evidence lets decision-makers and researchers rethink what is evidence. From the initial consideration of Randomised Controlled Trials as high-quality evidence,36 37 and systematic review/meta-analysis as the highest level of evidence in the next decade,38 39 and developed the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system to rating quality of evidence and strength of recommendations recently,40 the ‘evidence’ emphasises from research to systematic research, and from the body of evidence to support decision-making.

Although the extensional definition could help people understand more comprehensively what is ‘evidence’, it may not accurately reflect the sum of essential attributes of evidence. Most extensional definitions suggest that evidence is different types of ‘research’ (such as systematic review, cost-effectiveness analysis, model research). Only a small number of extensional definitions emphasise that evidence should include ‘stakeholder perspective’, ‘public experience’ and ‘professional experience’. For example, in evidence-informed decision-making, an extensional definition is commonly used,41–43 including such terms as ‘individual background’, ‘expert experience’ or ‘stakeholder perspective’ in order to address contextual factors and end-user needs for optimal decision-making. In fact, when making decisions regarding interventions, it is necessary to consider not only the effectiveness and safety of the intervention, but also its applicability to the local context or population, and the effect on equity across subpopulations.44 Some scholars also use evidence as a basis for developing health decision-making ecosystems, considering contextual factors at different levels (individual, interpersonal, organisational, sociocultural and community, and evidence of economic structures and systems), and advocating shared decision-making.45 Thus, if the extensional definition is too general, it is not very useful, however, if it is too precise, it may be incomplete and miss important contents. In addition, as scientific disciplines evolve, the extensional definition must be updated at intervals.

There are several limitations to this study. First, the search strategy may lack sensitivity as our research question (what are the definitions of ‘evidence’ in health and social sciences?) is difficult to describe in the specific format and search with standard database tools and strategies. However, we used supplementary search methods (such as searching reference lists, books and a range of websites) to try to ensure as comprehensive search as possible. Second, we only included documents published in Chinese and English. Given that the term ‘evidence’ can have different meanings across languages, and this study only focuses on the term ‘evidence’ and its translation in Chinese, some definitions published or used in other languages may have been missed. However, we plan to consult more experts in the next step to collect the definitions of ‘evidence’ from other languages. Third, this study only states the exact definition of ‘evidence’, and has not much considered the context for the definitions, the audience, or the basis for the definition.

Overall, this research solves several gaps such as systematically searching diverse definitions of ‘evidence’, finding definiens and their attributable modifiers, and analysing connections and differences in health sciences and social sciences. Given our findings that the definitions of evidence are diverse and not standardised at present, based on the results of this review, we plan further work to survey and interview representatives (such as researchers, policymakers, decision-makers and the public) in the field of health sciences and social sciences, further investigate the perceptions for the definition of ‘evidence’.

Conclusion

Although the definition of ‘evidence’ has attracted the attention of many scholars in different disciplines, there is no widely recognised and accepted definition of this term in scientific research. This raises the question of whether, or under what circumstances, a standardised, clear, meaningful and widely applicable definition of ‘evidence’ might be helpful, not only in both health and social sciences, but also in the interdisciplinary field. In the meantime, we recommend all scholars avoid misunderstanding within their own disciplines or when working across disciplines, by clarifying how they are using the term ‘evidence’.