Authors: Sophie Jullien, Harsha A Dissanayake, Marty Chaplin
Categories: Diagnosis, Infectious disease, Other
Source: The Cochrane Database of Systematic Reviews
Plague is a severe disease associated with high mortality. Late diagnosis leads to advance stage of the disease with worse outcomes and higher risk of spread of the disease. A rapid diagnostic test (RDT) could help in establishing a prompt diagnosis of plague. This would improve patient care and help appropriate public health response.
To determine the diagnostic accuracy of the RDT based on the antigen F1 (F1RDT) for detecting plague in people with suspected disease.
We searched the CENTRAL, Embase, Science Citation Index, Google Scholar, the World Health Organization International Clinical Trials Registry Platform and ClinicalTrials.gov up to 15 May 2019, and PubMed (MEDLINE) up to 27 August 2019, regardless of language, publication status, or publication date. We handsearched the reference lists of relevant papers and contacted researchers working in the field.
We included cross‐sectional studies that assessed the accuracy of the F1RDT for diagnosing plague, where participants were tested with both the F1RDT and at least one reference standard. The reference standards were bacterial isolation by culture, polymerase chain reaction (PCR), and paired serology (this is a four‐fold difference in F1 antibody titres between two samples from acute and convalescent phases).
Two review authors independently selected studies and extracted data. We appraised the methodological quality of each selected studies and applicability by using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS‐2) tool. When meta‐analysis was appropriate, we used the bivariate model to obtain pooled estimates of sensitivity and specificity. We stratified all analyses by the reference standard used and presented disaggregated data for forms of plague. We assessed the certainty of the evidence using GRADE.
We included eight manuscripts reporting seven studies. Studies were conducted in three countries in Africa among adults and children with any form of plague. All studies except one assessed the F1RDT produced at the Institut Pasteur of Madagascar (F1RDT‐IPM) and one study assessed a F1RDT produced by New Horizons (F1RDT‐NH), utilized by the US Centers for Disease Control and Prevention. We could not pool the findings from the F1RDT‐NH in meta‐analyses due to a lack of raw data and a threshold of the test for positivity different from the F1RDT‐IPM.
Risk of bias was high for participant selection (retrospective studies, recruitment of participants not consecutive or random, unclear exclusion criteria), low or unclear for index test (blinding of F1RDT interpretation unknown), low for reference standards, and high or unclear for flow and timing (time of sample transportation was longer than seven days, which can lead to decreased viability of the pathogen and overgrowth of contaminating bacteria, with subsequent false‐negative results and misclassification of the target condition).
F1RDT for diagnosing all forms of plague
F1RDT‐IPM pooled sensitivity against culture was 100% (95% confidence interval (CI) 82 to 100; 4 studies, 1692 participants; very low certainty evidence) and pooled specificity was 70.3% (95% CI 65 to 75; 4 studies, 2004 participants; very low‐certainty evidence).
The performance of F1RDT‐IPM against PCR was calculated from a single study in participants with bubonic plague (see below). There were limited data on the performance of F1RDT against paired serology.
F1RDT for diagnosing pneumonic plague
Performed in sputum, F1RDT‐IPM pooled sensitivity against culture was 100% (95% CI 0 to 100; 2 studies, 56 participants; very low‐certainty evidence) and pooled specificity was 71% (95% CI 59 to 80; 2 studies, 297 participants; very low‐certainty evidence).
There were limited data on the performance of F1RDT against PCR or against paired serology for diagnosing pneumonic plague.
F1RDT for diagnosing bubonic plague
Performed in bubo aspirate, F1RDT‐IPM pooled sensitivity against culture was 100% (95% CI not calculable; 2 studies, 1454 participants; low‐certainty evidence) and pooled specificity was 67% (95% CI 65 to 70; 2 studies, 1198 participants; very low‐certainty evidence).
Performed in bubo aspirate, F1RDT‐IPM pooled sensitivity against PCR for the caf1 gene was 95% (95% CI 89 to 99; 1 study, 88 participants; very low‐certainty evidence) and pooled specificity was 93% (95% CI 84 to 98; 1 study, 61 participants; very low‐certainty evidence).
There were no data providing data on both F1RDT and paired serology for diagnosing bubonic plague.
Against culture, the F1RDT appeared highly sensitive for diagnosing either pneumonic or bubonic plague, and can help detect plague in remote areas to assure management and enable a public health response. False positive results mean culture or PCR confirmation may be needed. F1RDT does not replace culture, which provides additional information on resistance to antibiotics and bacterial strains.
Plague has caused major historic pandemics including the 'Plague of Justinian' in the 6th century, the 'Black Death' in the 14th century (which resulted in the death of one‐third of the European population), and the 'Third Pandemic' in the 19th century (Rasmussen 2015). This severe disease remains a current threat in many parts of the world, and has increased over the last three decades (WHO 2009). Between 1989 and 2003, 25 countries reported 38,310 human cases of plague, including 2845 deaths (WHO 2019a). Since 2000, over 95% of the burden associated with plague has been concentrated in Africa, particularly the Democratic Republic of the Congo (DRC), Madagascar, Uganda, and the United Republic of Tanzania (WHO 2016; WHO 2019a). Peru and the USA also regularly report cases. Finally, although Asia is the region with the biggest natural foci of the disease, the reservoir consists of gerbils and marmots; there is a limited at‐risk population in contact with these animals, so outbreaks are sporadic (WHO 2016). As of 2017, the DRC, Madagascar, and Peru had the highest incidence of the disease. However, countries that have never experienced plague, or not experienced plague for a long while, can be affected as the limits of the existing foci are not fixed and new foci can emerge. Human plague outbreaks are continuously being reported, from Indonesia in 2007 and the DRC and Tanzania in 2014, to the outbreak of pneumonic plague more recently reported in Madagascar in 2017 (WHO 2009; WHO 2016; WHO 2019a).
Plague is caused by the bacteria Yersinia pestis. It is primarily a vector borne zoonotic disease, affecting rodents and other wild and domestic animals. It is most commonly transmitted to humans by rodent fleas, leading to bubonic plague. Less frequently, plague can be transmitted through scratches or bites from infected animals, direct handling of infected animals, and human‐to‐human transmission by inhalation of droplets from people with pneumonic plague (CDC 2019; Weniger 1984).
Plague can affect both adults and children, with no differences between genders or ethnicities. However, the disease presents more frequently among people involved in activities with an increased exposure to the disease, such as hunters, veterinarians, etc. Poverty is also associated with a greater risk of contracting plague due to increased exposure to rodents.
Plague is always a medical emergency and presents in a variety of forms, with three major clinical syndromes. Bubonic plague is the most common form and is characterized by enlarged lymph nodes with necrotic areas called buboes. Pneumonic plague is most often a fulminant form which affects the lungs and presents with cough and bloody sputum. The pneumonic form can be primary (as a result of inhalation of droplets from infected humans or animals), or secondary (as a result of the haematogenous spread of any other form of plague) (CIDRAP 2013). The third major clinical form is septicaemic plague, which occurs when the infection spreads to the circulatory system; it can be primary (without buboes or pulmonary affectation), or secondary (as a result of spreading bubonic or pneumonic plague). Less commonly, plague can present as meningitis (Prentice 2007).
Although efficient antimicrobials are available, plague still has a high mortality rate as most outbreaks take place in remote places in resource‐limited settings, where proper diagnosis and treatment remains challenging (WHO 2009). While bubonic plague is associated with case fatality ratios (CFRs) of 10% to 20%, pneumonic plague is highly fatal, with a CFR close to 100% if left untreated and over 50% when adequately treated with antimicrobials (Prentice 2007).
In addition to the sporadic cases and outbreaks, because of the characteristics of the disease resulting in high mortality, Y pestis has been used as a biological weapon and is currently a bioterrorism threat (CDC 2019).
Due to its historical pandemics and high fatality rate, plague continues to cause fear and panic, and is sometimes associated with a disproportionate public health response, which has considerable social and economic consequences (Mavalankar 1995; Mead 2018). A clinical diagnosis of plague is difficult and not reliable. The symptoms of pneumonic plague are not specific and can be present in a person with pneumonia caused by many other pathogens. These include other bacteria such as streptococcus pneumoniae or tuberculosis, that would require different antibiotics than plague, but also viruses, such as influenza, which would require no antibiotics. A person with a swollen lymph node in an endemic area or in the context of an outbreak is more likely to receive an accurate diagnosis of bubonic plague than people with suspected pneumonic plague. However, other diagnoses need to be considered, mainly other infections that cause swollen lymph nodes such as pyogenic abscess, tularaemia, tuberculosis, lymphogranuloma venereum, and cat scratch fever. A point‐of‐care diagnostic tool that is quick to use and highly accurate would help ensure appropriate response, especially in the context of outbreaks.
Rapid diagnostic tests (RDTs) detect pathogen‐specific antigens in a small quantity of different body fluids through lateral flow immunochromatography. RDTs are widely used in other diseases, such as malaria (WHO 2019b). They are usually easy to use and interpret. Indeed, they can be performed at the bedside of the patient without the requirement of special equipment or laboratory facilities. They give a simple result within around 15 minutes – positive or negative, at thresholds set by the manufacturer – that can easily be interpreted by health workers without advanced training. Therefore, RDTs are useful diagnostic tools for use at the community level and in low‐resource settings (WHO 2019b).
In the case of plague, the RDT detects the F1 capsular antigen of Y pestis (F1RDT), which is present in large amounts in buboes, blood, and sputum from patients infected with plague. F1RDT is the only RDT for plague that has been developed for clinical purposes that we are aware of. The test gives a semi‐quantitative result within 15 minutes according to the intensity of the line (from 1+ to 4+), although it is most commonly used as a qualitative test (positive or negative result) where positivity is interpreted from 1+ (as soon as the line is visible). The threshold for positivity will depend on the manufacturer, and is established by the lowest concentration of the F1 antigen that the test can detect. The F1RDT can be used in bubo aspirate, urine, and sputum (Chanteau 2000a); it is not usually used in blood as the pink result line would be difficult to see. Currently, the F1RDT that is mainly used in the field is produced in Madagascar. The most recent version was developed in 2001 (Chanteau 2003a). Other F1RDTs for plague are produced by New Horizons in the USA (New Horizons 2019) and in Taiwan (Hsu 2018), but they are not licensed for use in humans or available in the market.
Similar to other RDTs, F1RDT should be administered by trained staff, following the manufacturer's restrictions and warnings of the test. Although no advanced training is required, clear indications must be respected. These include sample collection, sample preparation, and timely and accurate reading of the result. Storage conditions are indicated by each manufacturer and are usually easy to comply with.
People of any age affected by plague will present with non‐specific symptoms such as fever, chills, headache, or nausea; these are usually associated with lymph node swelling in case of bubonic plague, with or without cough, haemoptysis, and chest pain for pneumonic plague. While a first diagnosis of suspected plague is based on clinical findings, the definitive diagnosis requires laboratory testing. Bacteriological identification of Y pestis through microscopy or culture (or both) is the reference standard for a confirmed case of plague (Rajerison 2020). Y pestis grows easily in standard culture media, and, while bacteriological isolation has a high specificity, it is also highly sensitive under ideal conditions. However, administration of antibiotics prior to sample collection is likely to lead to a false‐negative result. Delay in transportation from the time of collection to the central laboratory (which is not uncommon in low‐resource settings), associated or not with poor storage conditions, can lead to decreased viability of the pathogen and overgrowth of contaminating bacteria, with subsequent false‐negative results. Another confirmatory diagnosis of plague is obtained by serology with a four‐fold difference in F1 antibody titres between paired serum samples, from acute and convalescent phases. While this test is highly specific and allows diagnosis of true cases of plague, a limitation in practice is the collection of the second serum sample during the convalescence phase, as people might return to their home or work setting as soon as they feel well enough. Another technique more recently used in the diagnosis of plague is polymerase chain reaction (PCR), targeting several genes including the pla gene, encoding plasminogen activator, and the caf1 gene, encoding F1 capsule antigen. This technique only requires a small amount of sample, and detects specific genes of Y pestis whether the bacteria is alive or dead. A person with plague who has received antibiotics prior to sample collection might present a negative culture (false negative) but a positive PCR.
These diagnostic tests (culture, paired serology, and PCR) require technology and qualified staff, which are rarely available in resource‐limited areas. In addition, results from culture and paired serology take several days. These tests do not allow a fast confirmation or exclusion of plague diagnosis, and physicians cannot rely on them for the acute management of patients. Therefore, in practice, as soon as a case of plague is clinically suspected in an area where plague is endemic (or if the case visited an endemic area), the patient is immediately given antibiotics following collection of biological samples (blood, sputum, bubo aspirate, or a combination of these) whenever possible, and managed as a case of plague until microbiological diagnosis is confirmed or excluded, usually based on culture results. It is common that patients finish the treatment course or evolve to a fatal outcome before plague is microbiologically confirmed. In addition, public health measures are established, especially for cases of pneumonic plague, including tracing contacts and distributing chemoprophylaxis.
The use of the F1RDT, performed for all suspected cases of plague during the first contact of the patient with healthcare facilities, would support the clinical suspicion of plague when positive, and guide physicians to consider other diseases when negative, providing valuable guidance for both clinical and public health response.
It is very unlikely that patients will have had another diagnostic test for detecting plague prior to presentation to the health facility where the F1RDT would be performed, as the F1RDT should be performed at the first point of presentation with medical facilities.
The role of the F1RDT is to provide bedside rapid results in the identification of people with plague, to allow prompt treatment, and to establish preventive measures in order to limit transmission of the disease to others in case of a positive result for pneumonic plague. A negative F1RDT finding would prompt clinicians to consider other diagnoses for correct management of the patient and to avoid unnecessary preventive measures that would be essential in case of plague. An easy‐to‐use and accurate F1RDT for plague would, therefore, be of considerable help in daily clinical practice for the management of people with suspicion of plague in endemic areas by providing a fast diagnosis, as microbiological confirmation of plague takes several days. The F1RDT would be used in addition to the reference standard and would not replace culture, which is fundamental for assessing circulating strains and antibiotic resistance testing. According to the specificity of the test, the F1RDT could be used as a triage tool, meaning that a negative result would definitely exclude plague, and that other tests – such as culture – would only be collected in cases with a positive result, and managed accordingly.
The F1RDT cannot be used as a screening tool for plague in asymptomatic people, for example asymptomatic people who have been in contact with people with plague. Indeed, samples to perform the test are mainly bubo aspirate (in case of suspicion of bubonic plague) and sputum (in case of pneumonic plague) and those samples would be non‐existent in asymptomatic people.
Direct enzyme‐linked immunosorbent assay (ELISA) for detection of the F1 antigen is another test used for the diagnosis of plague. It requires equipment as well as trained personnel to perform it, and it is not easily available in low‐resource settings.
Plague is a serious illness with high mortality and rapid transmission from fleas or in between humans if control measures are not immediately implemented. Given the non‐specific symptoms of plague and the measures to be implemented in the event of a confirmed case (such as surveillance measures, identification of contacts for prophylaxis, and safe burial to avoid spread of the disease), it is crucial to make a formal diagnosis of plague as soon as possible to distinguish it from other infections with similar clinical presentation. This is particularly important for the pneumonic form of plague, where the high prevalence of other infections presenting with similar respiratory symptoms such as cough and fever makes an accurate clinical diagnosis difficult. The delay in a confirmed diagnosis may have two main repercussions. The first is at the individual level, as confirmation or exclusion of the diagnosis will help optimize the patient's management, including consideration of alternative diseases and treatments. A confirmed diagnosis allows targeted treatment with specific antibiotics. If the diagnosis is not made, patients are likely to be treated with routine empiric antibiotics which are ineffective against plague. The second is at a public health level, as pneumonic plague can be transmitted from human to human, leading to outbreaks, which are often associated with fear, panic, and sometimes with excessive measures that can lead to social and economic disruption, with considerable consequences (Mavalankar 1995).
A highly accurate and fast diagnostic test would undoubtedly be helpful. High accuracy of the test is imperative; indeed, low sensitivity (i.e. high numbers of false negatives) would lead to missed cases of plague in the situation where the test is used as a screening tool (i.e. to exclude plague diagnosis when a negative result is observed and pursue with microbiological analysis when a positive result is observed), and to initial mismanagement of the case until it is confirmed microbiologically. Low specificity would probably lead to a bigger concern. A false‐positive case might trigger unnecessary social alert, avoidable anxiety to the patients and their family, and avoidable use of resources, particularly in fragile health systems in countries where plague is endemic (Mavalankar 1995; Mead 2018). A highly specific test with a very low false‐positivity rate would allow the adequate management of all negative cases, considering them true‐negative cases.
Another important consideration to consider while evaluating the F1RDT is the use and importance of specificity of the test in diagnosing people with suspicion of pneumonic plague in the context of an outbreak. The pneumonic form of plague is a very severe and fatal disease that can be transmitted from human to human. Contrary to the bubonic form, where the presence of buboes might facilitate the suspicion of plague, symptoms presented with the pneumonic plague are less specific. In addition, the obtention of a good sample to run the F1RDT might be more difficult in the pneumonic plague, where it might be challenging to obtain good‐quality sputum from children and from severely ill people with decreased consciousness. Performing the test in saliva instead of sputum will certainly lead to different accuracy findings, and this should be taken into account.
The F1RDT is a simple diagnostic tool that can be performed at the bedside of the patient, with a fast result that allows prompt diagnosis and early treatment, as well as timely implementation of control measures to limit the spread of the disease. Therefore, the F1RDT has the potential to be useful to health workers, and could contribute to reducing the high mortality attributed to plague, as well as inadequate public health responses to it. However, there is no systematic review assessing the diagnostic test accuracy of the F1RDT for plague against standard diagnostic tests. Currently, a confirmed case of plague is made either by isolation of Y pestis (culture) or by acute and convalescent serological antibody testing (four‐fold difference in F1 antibody titres), according to World Health Organization (WHO) definitions (WHO 2019a). Positivity of either test provides a reliable diagnosis of plague and can be used interchangeably to consider a case of plague (although culture is preferred in order to identify the strain and resistance pattern). Therefore, it is reasonable to assess the accuracy of the F1RDT against both these tests. With the increasing inclusion of molecular biology for the diagnosis of many infectious diseases, we thought it was relevant to also assess accuracy of the F1RDT against PCR. However, to date, there is a lack of evidence supporting the superiority of one reference test above another.
The findings of this review will help to develop evidence‐based recommendations on the role of the F1RDT in the diagnosis of plague, which could be included in clinical guidelines about the management of plague as well as in guidelines on infection prevention and control.
To determine the diagnostic accuracy of the rapid diagnostic test (RDT) based on the antigen F1 (F1RDT) for detecting plague in people with suspected disease.
To assess the effect of forms of plague (bubonic, septicaemic, or pneumonic), specimen tested (bubonic aspirate, urine, or sputum), prior antibiotic treatment, location where the test is performed (field or laboratory studies), and threshold for detecting the disease (as set by the manufacturer) on the accuracy of the F1RDT for detecting the disease.
We included cross‐sectional studies that assess the accuracy of the F1RDT for diagnosing plague in the laboratory or in field conditions, where patients were tested for plague with both the F1RDT and at least one of the reference standards (culture, PCR, or serology). We excluded case‐control (two‐gate cross‐sectional) studies as we aim to determine accuracy of the RDT from only one set of participants, all of them with suspected plague.
We included participants (including children and pregnant women) living in or visiting areas where plague was endemic, who presented to any healthcare facility (primary, secondary, or tertiary care) with clinical suspicion of any form of plague. For studies where only a subgroup of participants was eligible for inclusion in the review, we included the study when there were disaggregated data that we could extract for that subgroup.
The index test we assessed was the F1RDT performed in any relevant sample to detect plague, this was bubo aspirate to detect bubonic plague, sputum to detect pneumonic plague, and other samples such as urine to detect septicaemic plague.
The target condition was any form of symptomatic plague (bubonic, septicaemic, or pneumonic).
We included studies that used one of the following reference standards to diagnose plague.
We conducted the literature search up to 15 May 2019 (27 August 2019 for PubMed), and identified potential studies regardless of language, publication status, or publication date.
We searched the following databases using the search terms and strategy described in Appendix 1: the Cochrane Central Register of Controlled Trials (CENTRAL, published in the Cochrane Library, Issue 5, May 2019), MEDLINE (PubMed, from 1966 to 27 August 2019), Embase (Ovid; from 1949 to 15 May 2019), Science Citation Index (Web of Science, from 1900 to 15 May 2019). On 15 May 2019, we also searched Google Scholar (scholar.google.co.uk), and the WHO International Clinical Trials Registry Platform (ICTRP; www.who.int/ictrp/en/), and ClinicalTrials.gov for trials in progress.
We searched the proceedings and abstracts of relevant conferences from the past five the International Symposium on Yersinia, the American Society of Tropical Medicine and Hygiene conference, and the congress of European Microbiologists. We handsearched the reference lists of relevant papers and contacted researchers working in the field. We also searched for related articles to the included studies using the PubMed "similar articles" function, on 27 August 2019.
Two review authors independently screened all the abstracts retrieved by the search strategy, using the predefined eligibility criteria. We excluded studies that were clearly irrelevant based on the titles and abstracts. We retrieved full‐text copies of the remaining studies and applied the predefined criteria for inclusion in the review. We resolved any disagreements in assessment through discussion. We listed all studies excluded after full‐text assessment in the Characteristics of excluded studies table. We illustrated the study selection process in a PRISMA diagram (Figure 1).
1 Study flow diagram.
One review author piloted the data extraction form on two studies. Based on the results of the pilot, we modified and finalized the data extraction form. Two review authors independently conducted data extraction and management, using the finalized data extraction form. We compared these data and resolved any disagreement through discussion. For each included study, we gathered information on the following (Appendix 2).
Two review authors independently assessed the methodological quality of each included study, using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS‐2) tool, which is based on four participant selection, index test, reference standard, and flow and timing (Whiting 2011). We have tailored the tool to the context of this review (Appendix 3). We answered each of the signalling questions as 'yes,' 'no,' or 'unclear,' and gave the reason for our judgement. We resolved any disagreements through discussion and through consultation with a third review author in case of persisting disagreement.
We stratified all analyses by the reference standard used. Within each stratum, we constructed a two‐by‐two table (containing the number of true‐positive, true‐negative, false‐positive, and false‐negative results) for each study. We entered the two‐by‐two data into Review Manager 5 (Review Manager 2014). We summarized estimates of sensitivity and specificity from each individual study on forest plots and plotted the estimates using summary receiver operating characteristic (SROC) plots.
We calculated positive (PPV) and negative predictive value (NPV) estimates for various scenarios to help interpret the impact of F1RDT findings. There are no data reporting estimates of pretest probability of plague for the different case scenarios. Therefore, we estimated three case scenarios for each form of plague (bubonic and pneumonic plague), to the best of our knowledge and in consultation with experts in the field. The prevalence of 4% was calculated based on the number of confirmed cases of plague among notified cases of plague during the 2017 outbreak in Madagascar, as a starting point. We estimated a lower pretest probability of the disease at 0.1% for the context where plague was known to occur but where there was no declared ongoing outbreak. Finally, we estimated a higher pretest probability of the test, where the test was used in the context of a declared outbreak. In the case of bubonic plague in a context of known outbreak, a person presenting with a bubo has high probability of having the disease. Therefore, we estimated a high pretest probability of 50%. In the case of pneumonic plague, the differential diagnosis is broader than for bubonic plague, including other respiratory infections that are highly prevalent. Therefore, we estimated a pretest probability lower than for the case of bubonic plague, at 20%.
When meta‐analysis was appropriate (given the number of studies and extent of clinical heterogeneity), we pooled results from the included studies. We used the bivariate model to obtain pooled estimates of sensitivity and specificity, as meta‐analysis was performed for a single threshold. If data were sparse and the bivariate model would not converge, we pooled data using methods described by Takwoingi and colleagues (Takwoingi 2017). All meta‐analyses were performed using the xtmelogit commands in Stata version 14 (Stata 2015). We plotted the pooled estimates of sensitivity and specificity using SROC plots in Review Manager 5 (Review Manager 2014).
We stratified the findings by type of reference standard bacterial isolation by culture, PCR, and serology showing a four‐fold difference in F1 antibody titres between two samples from acute and convalescent phases. For PCR, we described all the genes that the included studies assessed. However, when we needed to choose one gene (i.e. for meta‐analysis), we used caf1, which is the most relevant gene as according to experts in the field.
We planned to assess the impact of forms of plague on the accuracy of F1RDT by performing meta‐regression. However, there were an insufficient number of studies to obtain reliable results from meta‐regression. We instead presented the findings stratified by forms of plague, as this is clinically relevant.
We also planned to assess the impact of prior antibiotic treatment, location of performance of the F1RDT (field and central laboratory), and threshold for detecting the disease on accuracy of F1RDTs by performing subgroup analyses or meta‐regression. It was not possible to assess the impact of prior antibiotic treatment or location of performance of the F1RDT test due to scarcity of the data. Furthermore, it was not necessary to assess the impact of threshold as studies were consistent with regards to the threshold used to determine disease status.
We planned to perform a sensitivity analysis in which we only included studies that had a low risk of bias for the four domains (patient selection, index test, reference standard, and flow and timing), or restricted to those studies at low risk of bias for patient selection only. However, it was not possible to perform this sensitivity analysis as none of the included studies were at low risk of bias for the patient selection domain.
Two manuscripts analyzed two cohorts with overlap of participants (Andrianaivoarimanana 2019; Rajerison 2020). We included Rajerison 2020 in the primary analysis and performed a sensitivity analysis in which we repeated the analysis including Andrianaivoarimanana 2019.
We identified an outlier in the primary analysis assessing RDT against culture for all forms of plague. We performed a sensitivity analysis to explore the impact of this outlier in the pooled estimate.
Little is known on how to assess and detect reporting bias for diagnostic test accuracy studies (Macaskill 2010). We decided not to carry out a formal assessment of publication bias using methods such as funnel plots or regression tests because such techniques have important limitations when used in reviews of diagnostic test accuracy.
We assessed the certainty of the evidence using GRADE and GRADEpro GDT software (GRADE Handbook 2013; GRADEpro GDT 2015; Schünemann 2020). We rated the certainty of the evidence as high, moderate, low, or very low by assessing four domains (risk of bias, indirectness, inconsistency, and imprecision), as follows.
We constructed 'Summary of findings' tables, which showed the main review findings along with the certainty of the evidence.
The search identified 727 records. We identified six additional papers by handsearching or by contacting experts. After excluding duplicates and manuscripts published before 1995 (before which F1RDT was not available), we selected 623 records for screening. We excluded 580 of them based on title and abstract. Among the 43 publications identified for full‐text review, we could not retrieve the full‐text of one of them. We assessed the full‐text of 42 articles and excluded 34 of them, with the reasons for exclusion listed in the Characteristics of excluded studies table. See Figure 1 for the flow diagram of the study selection process.
Eight manuscripts met our inclusion criteria. Two manuscripts from Bertherat and colleagues reported the same plague outbreak that occurred in the DRC in 2005. As the manuscript published in 2005 did not provide additional data to the manuscript published in 2011, we merged the documents and referred to them in this review under one study ID (Bertherat 2011). Andrianaivoarimanana 2019 and Rajerison 2020 presented data on plague in Madagascar from national surveillance records, with an overlap of participants between the two manuscripts for the period between 2002 and 2007. We described the manuscripts separately in the Characteristics of included studies tables as they applied different exclusion criteria to the cohort of participants. However, we chose to include Rajerison 2020 in the primary analysis, as Rajerison 2020 presented additional data for two outbreaks with disaggregated data for bubonic and pneumonic plague.
We described the seven studies in the Characteristics of included studies table and their key findings in Table 5. Five studies reported findings of F1RDT used in plague epidemics in Madagascar between 2000 and 2019 (Andrianaivoarimanana 2019; Chanteau 2003a; Rajerison 2020; Richard 2015; Riehm 2011). One study assessed data from two outbreaks that occurred in the DRC in 2005 and 2006 (Bertherat 2011), and one study described performance of F1RDT from several clinics both in Madagascar and Uganda, from 2004 to 2017 (Petersen 2018).
The studies included in the primary analyses reported data from 4146 participants (Andrianaivoarimanana 2019 excluded) including adults and children of both sex. Among them, 3989 samples were analyzed with both the index test and at least one reference standard.
One study assessed only bubonic plague (Riehm 2011), two studies assessed only pneumonic plague (Bertherat 2011; Richard 2015), and four studies assessed all forms of plague (Andrianaivoarimanana 2019; Chanteau 2003a; Petersen 2018; Rajerison 2020). Samples used to perform F1RDT were bubo aspirates in participants with suspicion of bubonic plague and sputum in participants with suspicion of pneumonic plague (this was also assumed for Andrianaivoarimanana 2019 but was not clearly reported). F1RDT was also performed in a few postmortem tissue samples in two studies (Chanteau 2003a; Rajerison 2020). The postmortem tissue samples were not relevant to this review question; however, disaggregated data were not available, so we could not exclude them from our analysis. Rajerison 2020 excluded cases for which time of transport of the biological samples from collection to the laboratory was superior to seven days and cases who received antibiotics prior to sample collection.
The studies used two different F1RDT dipsticks. All studies except one (Petersen 2018) used the F1RDT produced at the Institut Pasteur of Madagascar (F1RDT‐IPM), which is based on the combination of two anti‐F1 antibodies (B18–1 and G6–18), with a lower detection threshold of the F1 antigen of 0.5 ng/mL. Petersen 2018 assessed a F1RDT from New Horizons (F1RDT‐NH), utilized by the US Centers for Disease Control and Prevention (CDC), with a lower detection threshold of the F1 antigen of 1 ng/mL (personal communication). Additional and extensive data on performance of this test are due to be published. According to the manufacturers, both F1RDT‐IPM and F1RDT‐NH can be stored at room temperature.
Three studies performed F1RDT in a central laboratory (Chanteau 2003a; Petersen 2018; Rajerison 2020), and two studies both on site and in a central laboratory (Bertherat 2011; Riehm 2011). It was also performed on site in Chanteau 2003a, but data included for analysis were based from the findings of those performed in the central laboratory. It was unclear where the test was performed in Richard 2015. Andrianaivoarimanana 2019 did not report the location of performance of F1RDT, but we deduced it was in a central laboratory, as there was an overlap of participants from Rajerison 2020. The people who performed the F1RDT were a trained biologist in Bertherat 2011, and trained doctors, nurses, and health workers in Chanteau 2003a. Such information was not reported for the remaining studies.
All studies compared F1RDT against culture, three studies against PCR (Bertherat 2011; Richard 2015; Riehm 2011), and two studies against paired serology (Petersen 2018; Richard 2015).
The Institut Pasteur of Madagascar, which produced one of the two dipsticks assessed in this review, supported two studies (Chanteau 2003a; Richard 2015). Other financial support or sponsors were the Institut Pasteur of Paris (Chanteau 2003a), the WHO (Rajerison 2020), and the President's Malaria Initiative/US Agency for International Development and the US Department of Homeland Security (Richard 2015). Four studies did not report the source of funding (Andrianaivoarimanana 2019; Bertherat 2011; Petersen 2018; Riehm 2011).
See the Characteristics of included studies table for the assessment of the methodological quality of each included manuscript, and Figure 2 for risk of bias and applicability concerns summaries.
2 Risk of bias and applicability concerns review authors' judgements about each domain for each included study.
In the patient selection domain, we considered all studies except one (Petersen 2018) at high risk of bias, because recruitment of participants was not consecutive or random but based on selection of participants based on clinicians who declared cases to the national surveillance system, following a retrospective design; in addition, there were unclear exclusion criteria. In some studies, administration of antibiotics prior to sample collection was not taken into account, which could have led to a false‐negative culture when the participant truly had plague, and therefore misclassification of the test to detect the disease. We considered Petersen 2018 at unclear risk of bias as there were insufficient data to judge on this domain. Regarding applicability, five studies had low concern in the patient selection domain because the studies included the appropriate participants and settings (outbreak and national surveillance in endemic areas). Two studies were at unclear concern because there were insufficient data from Petersen 2018, and because there were limited information from the only two participants in Richard 2015.
In the index test domain, we judged two studies at low risk of bias because the F1RDT was interpreted without the knowledge of the results of the reference standard, and we considered them to have low concern for applicability (Bertherat 2011; Chanteau 2003a). The other five studies were at unclear risk of bias because we do not know if the result of the F1RDT was interpreted without the knowledge of the results of the reference standard. In addition, these studies did not report the way that the test was conducted, such as who performed the test, where it was performed, or the need for sample preparation. We therefore considered the conduct and interpretation of the index test to be of unclear concern for applicability in these five studies (Andrianaivoarimanana 2019; Petersen 2018; Rajerison 2020; Richard 2015; Riehm 2011).
In the reference standard domain, we considered all studies at low risk of bias, as they all used culture, and four studies also used PCR, paired serology, or both, which correctly classify the target condition. For most studies, it was unclear whether the interpretation of the reference standard was blinded to the RDT result, but reference standard results are objective findings and we considered this to be at low risk to introduce bias, and at low concern of applicability.
In the flow and timing domain, we considered three studies at high risk of bias because the time of sample transportation was longer than seven days, which can lead to decreased viability of the pathogen and overgrowth of contaminating bacteria, with subsequent false‐negative results and misclassification of the target condition (Bertherat 2011; Chanteau 2003a; Riehm 2011). In addition, in one study, all participants did not receive the same reference standard (Bertherat 2011). We considered one study at low risk of bias in this domain as they excluded participants who received antibiotics prior to enrolment and those for whom samples took longer than seven days to reach the central laboratory (Rajerison 2020). This was unclear for the remaining three studies.
We present the findings for all forms of plague, and then present findings disaggregated for pneumonic and bubonic plague. We could not pool the findings from the two different F1RDT used (F1RDT‐IPM and F1RDT‐NH) as the tests present different lower detection thresholds for considering the test positive. Furthermore, the only study that assessed the F1RDT‐NH used combined reference standards to consider true cases of plague (Petersen 2018).
We present the summary of findings including true positive, false positive, false negative, true negative, sensitivity, and specificity for each study with disaggregated data on F1RDT for diagnosing the different forms of plague against the different reference standard in Table 6.
Six studies reported data on F1RDT‐IPM and culture findings for diagnosing all forms of plague. We used Rajerison 2020 in our primary analysis and performed sensitivity analysis with Andrianaivoarimanana 2019.
Sensitivity estimates ranged from 25% to 100% and specificity estimates ranged from 51% to 82% (Table 6).
Two data set were not included in the meta‐analysis; Richard 2015 reported zero true‐positive and zero true‐negative cases and, therefore, had no estimable sensitivity or specificity, and Bertherat 2011 reported a true‐positive value of zero for the 2005 outbreak and, therefore, had no estimable sensitivity.
Four studies contributed to the final primary meta‐analysis (Bertherat 2011 (2006 outbreak only); Chanteau 2003a; Rajerison 2020; Riehm 2011) (Figure 3). For the diagnosis of all forms of plague, F1RDT‐IPM pooled sensitivity against culture was 100% (95% CI 82 to 100; 4 studies, 1692 participants; very low‐certainty evidence) and specificity was 70% (95% CI 65 to 75; 4 studies, 2004 participants; very low‐certainty evidence) (Table 1) (Figure 4). These results were from two univariate random‐effects meta‐analyses, one for sensitivity and one for specificity.
3 Forest plot of 1 F1 antigen rapid diagnostic test (F1RDT) versus culture for all forms of plague, primary analysis.The data for Bertherat 2011 refer to the 2006 outbreak only. CI: confidence interval; FN: false negative; FP: false positive; TN: true negative; TP: true positive.
4 Summary receiver operating characteristic (SROC) plot of one F1 antigen rapid diagnostic test (F1RDT) versus culture for all forms of plague, primary analysis.Pooled 100% (95% CI 82 to 100); pooled 70% (95% CI 65 to 75).The solid black dot corresponds to the pooled estimate of sensitivity and specificity; the black circles show individual study results and their size corresponds to the sample size of the study contributing to the analysis.
By considering the forest plot, we found that sensitivity from participants with pneumonic plague in the 2017 to 2018 period reported by Rajerison 2020 were not in line with the rest of the sensitivity estimates by the other studies. There was a limited number of four cases in this period, all four had positive culture but only one had a positive RDT. This could have been due to several factors such as the quality of sputum samples. We decided to keep this outlier in the meta‐analysis as it did not reflect any situation inherently different from the other studies.
Out of the 691 biological samples evaluated by Chanteau 2003a, RDT was performed on 35 postmortem samples from lung or liver puncture. The authors did not present disaggregated data for those samples or by forms of plague. As these samples constituted 5% of the overall samples evaluated in this study, we decided to include these findings.
We did not investigate whether type of plague explained heterogeneity by introducing a covariate to the bivariate model, as there were an insufficient number of studies to produce reliable results. However, we did produce an SROC plot which allowed visual examination of how sensitivity and specificity varied between subgroups (Figure 5). There was no clear pattern within the limited number of studies to suggest that differences existed between the different forms of plague.
5 Summary receiver operating characteristic (SROC) plot of one F1 antigen rapid diagnostic test (F1RDT) versus culture for all forms of plague, primary analysis, for investigation of heterogeneity.Each circle (all forms of plague with no disaggregated data), square (pneumonic plague), and diamond (bubonic plague) corresponds to individual study results. Their size corresponds to the sample size that contributed to the estimate of sensitivity and specificity.
We performed sensitivity analysis by excluding Rajerison 2020 and including Andrianaivoarimanana 2019. Both studies were conducted from surveillance data in Madagascar, but Rajerison 2020 presented disaggregated data for participants with bubonic and pneumonic forms of plague for three different times (2002 to 2007, 2017 to 2018, and 2018 to 2019), and Andrianaivoarimanana 2019 presented sensitivity and specificity estimates from all forms of plague during the 2002 to 2007 period. In addition, and contrary to Andrianaivoarimanana 2019, Rajerison 2020 excluded from analysis cases for which time of transport of biological samples was more than seven days, and cases who received antibiotics prior to sample collection, which would ensure a better performance of the standard test for comparison.
We found very similar findings to the primary analysis, with a pooled sensitivity of 100% (95% CI 70 to 100) and pooled specificity of 71% (95% CI 65 to 77) (Figure 6). These results were from two univariate random‐effects meta‐analyses, one for sensitivity and one for specificity.
6 Summary receiver operating characteristic (SROC) plot of 2 F1 antigen rapid diagnostic test (F1RDT) versus culture for all forms of plague, sensitivity analysis.Pooled 100% (95% CI 70 to 100); pooled 71% (95% CI 65 to 77).The solid black dot corresponds to the pooled estimate of sensitivity and specificity; the black circles show individual study results and their size corresponds to the sample size of the study contributing to the analysis.
We also performed sensitivity analysis by excluding the data on pneumonic plague from Rajerison 2020 for the 2017 to 2018 period, identified as an outlier. Although the CI for sensitivity was wider, we found no difference to the pooled results as compared with the primary analysis (pooled sensitivity of 100%, 95% CI 49 to 100 and pooled specificity of 70%, 95% CI 65 to 74). These results were from one bivariate random‐effects meta‐analysis,
Three studies provided data on both F1RDT‐IPM and PCR findings (Bertherat 2011, 2005 outbreak; Richard 2015; Riehm 2011). We summarized the outcomes in Table 6. Sensitivity estimates ranged from 72% to 95% and specificity estimates ranged from 50% to 93%. The specificity estimate of 50% was based from a sample size of two participants (Bertherat 2011, 2005 outbreak).
Riehm 2011 assessed the performance of three different genes by PCR in the same set of 149 participants. We calculated a sensitivity of 95% and specificity of 93% for the caf1 gene, 72% and 93% for the pla gene, and 92% and 87% for the Ymt gene.
Bertherat 2011 (2005 outbreak) and Richard 2015 provided data on two participants each. It was not possible to estimate sensitivity in Bertherat 2011 (2005 outbreak) as the reference standard identified no cases of plague, and it was not possible to assess specificity in Richard 2015 as the reference standard only identified cases of plague. With one study remaining (Riehm 2011), we could not perform meta‐analysis to assess the performance of F1RDT‐IPM versus PCR.
Only one study reported data with F1RDT‐IPM and paired serology findings (Richard 2015). Both tests (F1RDT‐IPM and paired serology) were performed on sputum in two participants, and were positive in both participants (Table 6).
One study reported findings of the F1RDT‐NH for diagnosing plague (Petersen 2018).
Confirmed cases of plague were defined as isolation of Y pestis by culture in bubo, blood, or sputum, or 4‐fold change in titre between acute and convalescent serum samples. Participants confirmed as non‐plague were defined as Y pestis not isolated from culture and no significant change in titre between acute and convalescent serum samples.
From the limited available data, there were 118 confirmed cases of plague (34 from Madagascar and 84 from Uganda); among which 109 where cases of bubonic plague. There were 136 participants tested and confirmed as non‐plague (61 from Madagascar and 75 from Uganda), among which 116 were suspected bubonic plague.
F1RDT‐NH gives a semi‐quantitative result according to the intensity of the line in the dipstick, from 1+ to 4+. Where positivity of F1RDT‐NH was interpreted from 1+ (this is as soon as the line is visible), the test presented a sensitivity of 90.6% (95% CI 83.8 to 95.2) and a specificity of 88.0% (95% CI 82.9 to 93.3), as reported by the study authors. When evaluating F1RDT‐NH with positivity results from 2+, the test presented a sensitivity of 87.4% (95% CI 80.1 to 93.0) and a specificity of 97.7% (95% CI 93.4 to 99.5).
Comprehensive findings of this F1RDT‐NH should be published soon (personal communication).
Three studies provided findings from both F1RDT‐IPM and culture for diagnosing pneumonic plague (Bertherat 2011; Rajerison 2020; Richard 2015) (Table 6). Sensitivity was 100% with broad 95% CI (the lowest lower limit was 40%), except for the 2017 to 2018 period reported by Rajerison 2020 that showed a sensitivity of 25% (Figure 7). There was a limited number of four cases during this period, which had positive culture but only one of them had a positive RDT. This could be due to several factors such as technical issues in preparing the sputum sample for analysis or in running the F1RDT. Specificity estimates ranged from 51% to 82%.
7 Forest plot of 1 F1RDT versus culture for pneumonic plague. CI: confidence interval; FN: false negative; FP: false positive; TN: true negative; TP: true positive.
Only two studies provided data that allowed estimation of both sensitivity and specificity of the test (Bertherat 2011; Rajerison 2020), and so these two studies contributed to the meta‐analysis. Performed in sputum, F1RDT‐IPM pooled sensitivity against culture was 100% (95% CI 0 to 100; 2 studies, 56 participants; very low‐certainty evidence) and pooled specificity was 71% (95% CI 59 to 80; 2 studies, 297 participants) (Table 2) (Figure 8). These results were from one bivariate random‐effects meta‐analysis.
8 Summary receiver operating characteristic (SROC) plot of 1 F1 antigen rapid diagnostic test (F1RDT) versus culture for pneumonic plague.Pooled 100% (95% CI 0 to 100); pooled 71% (95% CI 59 to 80).The solid black dot corresponds to the pooled estimate of sensitivity and specificity; the black circles show individual study results and their size corresponds to the sample size of the study contributing to the analysis.
Bertherat 2011 (2005 outbreak) and Richard 2015 provided findings of the F1RDT‐IPM for diagnosing pneumonic plague. However, it was not possible to assess sensitivity in Bertherat 2011 (2005 outbreak) as there were no cases of plague identified by PCR (sensitivity cannot be calculated when there are no cases identified as this would involve dividing by zero), and it was not possible to assess specificity in Richard 2015 as PCR only identified cases of plague (specificity cannot be calculated when only cases are identified as this would involve dividing by zero). Therefore, we did not pool data from these two studies in a meta‐analysis.
Richard 2015 provided findings of the F1RDT‐IPM for diagnosing two participants with pneumonic plague, but it was not possible to assess specificity as the paired serology only identified cases of plague.
There were no disaggregated findings for pneumonic plague from the F1RDT‐NH test.
There was no disaggregated findings for bubonic plague from the New Horizons test. The findings below are from studies assessing the F1RDT‐IPM.
Two studies performed both the F1RDT‐IPM and culture in participants with suspicion of bubonic plague (Rajerison 2020; Riehm 2011) (Table 6). Sensitivity estimates were 100% for all the time periods presented by the two studies, thereby it was not possible to pool data for sensitivity estimates (Figure 9). Specificity estimates ranged from 60% to 82%. F1RDT‐IPM performed in bubo aspirate for diagnosing bubonic plague showed a pooled specificity of 67% (95% CI 65 to 70; 2 studies; 1198 participants).
9 Forest plot of 1 F1RDT versus culture for bubonic plague. CI: confidence interval; FN: false negative; FP: false positive; TN: true negative; TP: true positive.
One study assessed the performance of three different genes by PCR in the same set of 149 participants (Riehm 2011). We calculated the sensitivity and specificity of the F1RDT‐IPM against each of these genes, and found sensitivity of 95% (95% CI 89 to 99) and specificity of 93% (95% CI 84 to 98) for the caf1 gene, 72% (95% CI 63 to 80) and 93% (95% CI 77 to 99) for the pla gene, and 92% (95% CI 84 to 97) and 87% (95% CI 76 to 94) for the Ymt gene (Table 6).
We found no studies assessing the accuracy of F1RDT‐IPM against paired serology for the diagnosis of bubonic plague.
There were no disaggregated findings for bubonic plague from the F1RDT‐NH test.
This systematic review of the diagnostic accuracy of F1RDT for diagnosing any form of plague in people with suspected disease summarizes the current literature and includes seven studies described in eight manuscripts. Three studies described the performance of F1RDT for diagnosing pneumonic plague and two studies for diagnosing bubonic plague, with no disaggregated data for forms of plague in three studies. The evidence came from three African countries (Madagascar, the DRC, and Uganda) with findings of F1RDT used during outbreaks or surveillance system in settings where plague is endemic. All studies except one used the F1RDT produced at the Institut Pasteur of Madagascar (F1RDT‐IPM), while the remaining study assessed a F1RDT from New Horizons (F1RDT‐NH), which is utilized by the US CDC. All studies were considered at high risk of bias for the patient domain, mainly due to the retrospective design of the studies, the absence of consecutive or random sampling with clear inclusion and exclusion criteria, or both. The major findings of this review include the following.
For the diagnosis of any form of plague, the accuracy of F1RDT‐IPM
For the diagnosis of pneumonic plague, the accuracy of F1RDT‐IPM
The performance of F1RDT‐IPM for diagnosing bubonic
The performance of the F1RDT‐NH for diagnosing any form of plague against combined culture or paired serology showed sensitivity of 91% (95% CI 84 to 95) and specificity of 88% (95% CI 83 to 93), as reported by the study authors. When evaluating the F1RDT‐NH with positivity results from 2+, the sensitivity decreased to 87% (95% CI 80 to 93) and the specificity increased to 98% (95% CI 93 to 100).
Early diagnosis of pneumonic plague is critical so that prompt treatment is started (pneumonic plague is associated with high fatality rate if left untreated) and so that preventive measures are established to limit transmission of the disease (pneumonic plague can be transmitted from human to human by inhalation of respiratory droplets produced by coughing).
Against culture results, sensitivity appeared high (100%) but three participants testing negative in one outbreak in a meta‐analysis that included data for 56 cases resulted in very wide CIs (0 to 100). The specificity was 71% (95% CI 59 to 80). However, these estimates came from a small number of participants and there was very low‐certainty evidence.
We used different case scenarios where pretest probability of plague varied from 0.1% to 4% to 20% to simulate scenarios where F1RDT could be used in plague endemic areas and in situations where an outbreak is in progress (Table 2). This numerical approach should be interpreted with caution due to the limitations cited above.
10 Flow diagram summarizing the main results in a hypothetical cohort with 4% of people with pneumonic plague (adapted from van Hoving 2019).F1RDT: F1 antigen rapid diagnostic test.
The high sensitivity associated with F1RDT for diagnosing pneumonic plague means that the test would detect all cases of pneumonic plague and would not miss any people with diagnosis of pneumonic plague. The high NPV means that if result of F1RDT is negative, pneumonic plague can be ruled out. However, the relatively low specificity and low PPV associated with F1RDT are of concern for the potential significant repercussions in the context of this disease. False positives mean that people who do not have plague will be considered as having plague, with repercussion for both the person (missing the true diagnosis, unnecessary anxiety and stigma related to the diagnosis of plague) and for society (unnecessary social alert and use of economic resources) (Figure 10).
The high apparent level of false positives with culture as the reference standard may actually be a result of people with plague testing culture negative; prior antibiotic use, for example, could cause this. Culture is known to be an imperfect reference standard for plague, as reminded in Andrianaivoarimanana 2019 where authors mentioned that "negative culture results might have resulted from administration of antimicrobial drugs by local health officials before sampling, whereas F1RDT results remained positive >3 weeks after treatment initiation." This would result in an underestimation of the specificity.
There were insufficient data to make any estimates against PCR or paired serology as reference standard.
Early and correct diagnosis of bubonic plague is essential for correct management of the patient and adequate public health measures and rodent control.
Two studies assessed F1RDT against culture, with a mean sensitivity of 100% (95% CI not calculable), and specificity of 67% (95% CI 65 to 70). However, the evidence was low certainty for sensitivity and very‐low certainty for specificity.
One study assessed F1RDT against PCR for three genes (caf1, pla, Ymt). For caf1, which is considered the most relevant gene according to experts in the field, sensitivity was 95% (95% CI 89 to 99) and specificity was 93% (95% CI 84 to 98).
We used different case scenarios where pretest probability of plague varied from 0.1% to 4% to 50% to simulate scenarios where F1RDT could be used in plague endemic areas and in situations where an outbreak is in progress. This numerical approach should be interpreted with caution due to the limitations cited above.
When true cases of bubonic plague were diagnosed with culture, the main findings were the following (Table 3).
When true cases of bubonic plague are diagnosed with PCR targeting the caf1 gene, the main findings are the following (Table 4).
Similar to its performance for diagnosing pneumonic plague, the high sensitivity associated with F1RDT for diagnosing bubonic plague means that the test would detect all cases (when compared to culture) or most cases (when compared to PCR) of bubonic plague. The high NPV means that when F1RDT shows a negative finding, bubonic plague can be ruled out.
We found higher specificity and PPV estimates when F1RDT was compared to PCR than to culture. This further demonstrates that culture is an imperfect reference standard. In the case of prior use of antibiotics, culture is likely to become negative while PCR might still be positive by detecting DNA of Y pestis after several doses of antibiotics.
We found no studies that determined the performance of F1RDT against paired serology for diagnosing bubonic plague.
The small number of studies and participants included in the analyses is a major limitation of the review with regards to estimation of accuracy. The sensitivity estimates had broad CIs, which underlies the imprecision of the estimates. The lower and higher values of the CIs provide very different scenarios that would lead to different decisions in practice. The study design of the included studies contributing to the high risk of selection bias further weakened our confidence in the results – we judged all included studies at high risk of bias in the patient selection domain when applying the QUADAS‐2 tool. We found no study that assessed the accuracy of F1RDT for diagnosing plague by prospectively recruiting consecutive patients with clear inclusion and exclusion criteria. There are also some limitations with regards to the reference standards. Although they were considered at low risk of introducing bias due to the objective results they provided, culture could be considered as an imperfect reference standard when antibiotics are given prior to sample collection, and there is a lack of standardized targeted gene for identification Y pestis with PCR.
The findings of this review are based on a comprehensive literature search with no restriction in language, strict inclusion criteria, duplicate data extraction, and rigorous assessment of risk of bias using the QUADAS‐2 tool tailored to our review question. The inclusion of unpublished data at the time of conducting the review constitutes a strength of the review process.
We attempted to contact study authors when there was poor reporting that limited data extraction or our judgement on applicability of the findings.
The prevalence of plague is not well established. For the hypothetical cohort scenarios, we presented above and in the 'Summary of finding' tables, we chose different values from 0.1% to 50% corresponding to pretest probability in order to contemplate different scenarios. For this, we considered using RDT where there was no ongoing outbreak (lower pretest probability) or where there was ongoing outbreak (higher pretest probability), as well as the form of plague (signs and symptoms of bubonic plague are more specific than those of pneumonic plague, therefore the pretest probability of plague when conducting F1RDT in a person with fever and a bubo in an endemic area is likely to be higher than when conducting F1RDT in a person with fever and productive cough).
Sources of heterogeneity could not be explored due to the scarcity of the included studies and participants. However, we calculated the test accuracy and presented the findings disaggregated by form of plague and against different reference standards that are commonly used in practice.
Inclusion criteria of this review were broad and, as such, were a good representation of the real scenarios in which F1RDT would be used. Overall, we had low concern of the applicability of findings from the included studies to our review question, when assessed with the tailored QUADAS‐2 tool.
The participant characteristics and settings matched our review question. For the reference standard domain, all the studies had low concern for applicability. With regards to the index test domain, F1RDT was conducted in central laboratories for the majority of participants who contributed to the primary analyses. In most cases, it was unclear whether the index test was performed by specialized and trained staff or by healthcare workers who may not have been adequately trained or may not have used a rapid test by immunochromatography previously. Therefore, it is possible that the accuracy of the test is lower when used on the bedside of the patients in the field. Another aspect to consider is the sample collection. Collection of sputum from a sick person during an outbreak is often performed in a context of panic and can be challenging, leading to collection of saliva instead. In the case of people with suspected bubonic plague, the obtention of pus from the bubo is usually simpler but can also be challenging depending on the size of the bubo. However, this is likely to affect the yield of all diagnostic tests performed in the relevant sample (sputum or bubo aspirate), including F1RDT, culture, and PCR, with no expected repercussion on the accuracy of the F1RDT against culture or against PCR. Poor‐quality samples (sputum or bubo aspirate) may lead to a decreased sensitivity when comparing the F1RDT against paired serology (performed in blood samples).
Protocol first Issue 10, 2019 Review first Issue 6, 2020
The CIDG Academic Editor is Professor Mical Paul, and DTA Editor is Dr Mia Schmidt‐Hansen.
We are grateful to Vittoria Lutje, the Information Specialist of the Cochrane Infectious Diseases Group (CIDG), for her help with the literature search strategy, and to the Diagnostic Test Accuracy (DTA) editorial team for their input. We acknowledge Minoarisoa Rajerison and Jeanine Petersen for their expertise and support, and for providing unpublished data. We are grateful to Xin Wang for her help in screening Chinese manuscripts. We thank Paul Garner, CIDG Co‐ordinating Editor, for his feedback and for contributing to the assessment of risk of bias.
Marty Chaplin and the CIDG editorial base are funded by UK aid from the UK government for the benefit of low‐ and middle‐income countries (project number 300342‐104). The views expressed do not necessarily reflect the UK government's official policies.
Cochrane Central Register of Controlled Trials (CENTRAL)
#1 (plague):ti,ab,kw #2 yersinia pestis #3 #1 or #2 #4 diagnosis or diagnostic* or detect* #5 RDT* #6 "Enzyme‐Linked Immunosorbent Assay" or ELISA #7 lateral flow #8 chromatograph* #9 immunochromatograph* #10 #4 or #5 or #6 or #7 or #8 or #9 #11 #3 and #10
MEDLINE (PubMed)
Embase (Ovid)
1 plague.mp. or plague/ 2 yersinia pestis.mp. or Yersinia pestis/ 3 diagnosis/ 4 diagnostic test/ 5 RDT.mp. 6 "rapid diagnos ".ab. or "rapid diagnos".ti. 7 ELISA.mp. or enzyme linked immunosorbent assay/ 8 immunochromatography/ or immunoaffinity chromatography/ or chromatography/ or immunochromatograph.mp. 9 lateral flow.mp. 10 1 or 2 11 3 or 4 or 5 or 6 or 7 or 8 or 9 12 10 and 11
Science Citation Index‐Expanded (Web of Science)
Google Scholar = plague and diagnosis, plague and RDT
Clinicaltrials.gov, WHO ICTRP: plague and diagnosis, plague and RDT
PCR: polymerase chain reaction, RDT: rapid diagnostic test.
PCR: polymerase chain reaction, RDT: rapid diagnostic test.
Presented below are all the data for all of the tests entered into the review.
1 TestF1RDT versus culture for all forms of plague, primary analysis
2 TestF1RDT versus culture for all forms of plague, sensitivity analysis 1
3 TestF1RDT versus culture for all forms of plague, sensitivity analysis 2
We conducted the review according to the published protocol (Jullien 2019).
SJ and HAD assessed the eligibility of the studies, extracted the data, and assessed the methodological quality of the included studies.
MC conducted the statistical analyses and contributed to the assessment of the methodological quality of the included studies.
SJ interpreted the findings and assessed the certainty of the evidence using the GRADE approach.
SJ drafted the text with considerable input from MC.
All review authors read and approved the final manuscript draft.
SJ: worked for the CIDG at the Liverpool School of Tropical Medicine from September 2015 to April 2016. SJ has a contract with the World Health Organization (WHO) for the development of evidence synthesis in the process of updating the WHO plague guideline.
HAD: none.
MC: none.
New