Authors: Rachel L. Epstein (1Department of Medicine, Section of Infectious Diseases, Boston University Chobanian and Avedisian School of Medicine, Boston, Massachusetts, USA; 2Department of Pediatrics, Section of Infectious Diseases, Boston University Chobanian and Avedisian School of Medicine, Boston, Massachusetts, USA), Benjamin Buzzee (3Section of Infectious Diseases, Boston Medical Center, Boston, Massachusetts, USA), Laura F. White (4Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts, USA), Jordan J. Feld (5Toronto Centre for Liver Disease, University Health Network, University of Toronto, Toronto, Ontario, Canada), Laurent Castera (6Department of Hepatology, Beaujon Hospital, Assistance Publique-Hopitaux de Paris, Université Paris Cité, Clichy, France), Richard K. Sterling (7Department of Internal Medicine, Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, Virginia, USA), Benjamin P. Linas (1Department of Medicine, Section of Infectious Diseases, Boston University Chobanian and Avedisian School of Medicine, Boston, Massachusetts, USA; 8Department of Epidemiology, Boston University School of Public Health, Boston, Massachusetts, USA), Lynn E. Taylor (9College of Pharmacy, University of Rhode Island, Kingston, Rhode Island, USA)
Categories: Article, Hepatitis C virus, Predictive value of tests, Elasticity imaging techniques, Biomarkers, Liver fibrosis
Source: Journal of viral hepatitis
Doi: 10.1111/jvh.13925
Authors: Rachel L. Epstein, Benjamin Buzzee, Laura F. White, Jordan J. Feld, Laurent Castera, Richard K. Sterling, Benjamin P. Linas, Lynn E. Taylor
Non-invasive methods have largely replaced biopsy to identify advanced fibrosis in hepatitis C virus (HCV). Guidelines vary regarding testing strategy to balance accuracy, costs, and loss to follow-up. Although individual test characteristics are well-described, data comparing the accuracy of using two tests together are limited. We calculated combined test characteristics to determine the utility of combined strategies.
This study synthesizes empirical data from fibrosis staging trials and the literature to estimate test characteristics for Fibrosis-4 (FIB4), APRI or a commercial serum panel (FibroSure^®^), followed by transient elastography (TE) or FibroSure^®^. We simulated two testing 1) second test only for those with intermediate first test results (staged approach), and 2) second test for all. We summarized empiric data with multinomial distributions and used this to estimate test characteristics of each strategy on a simulated population of 10,000 individuals with 4.2% cirrhosis prevalence.
Negative predictive value (NPV) for cirrhosis from a single test ranged from 98.2% (95% CB 97.6–98.8%) for FIB-4 to 99.4% (95% CB 99.0–99.8%) for TE. Using a staged approach with TE second, sensitivity for cirrhosis rose to 93.3–96.9%, NPV to 99.7–99.8%, while PPV dropped to <32%. Using TE as a second test for all minimally changed estimated test characteristics compared with the staged approach.
Combining two non-invasive fibrosis tests barely improves NPV and decreases or does not change PPV compared with a single test, challenging the utility of serial testing modalities. These calculated combined test characteristics can inform best methods to identify advanced fibrosis in various populations.
Direct acting antiviral agents (DAAs) have revolutionized hepatitis C virus (HCV) treatment with high cure rates and short treatment duration. However, HCV-related liver disease continues to cause approximately 399,000 deaths annually.^1^ In the United States, only 34% of the 2.4 million infected individuals had been cured as of 2022,^2^ and globally, only 13% of 58 million infected individuals had been cured as of 2019.^1^ One major obstacle to HCV treatment is completion of liver disease staging to determine the presence of advanced fibrosis. Although a critical aspect of evaluation to determine best HCV treatment, predict hepatic disease progression, and in those with cirrhosis, conduct surveillance for hepatocellular carcinoma (HCC), esophageal varices, and decompensation, many patients are lost to follow-up before completing this step. Major guidelines also differ on which tests they recommend.^1,3–5^
Non-invasive fibrosis staging tests have now replaced liver biopsy as standard of care to estimate hepatic fibrosis given their reasonable accuracy, superior safety profile and accessibility, and lower cost.^1,4,6^ Imaging modalities such as transient elastography (TE), which uses liver stiffness measurement (LSM) as a hepatic fibrosis correlate, demonstrate the highest accuracy of non-invasive approaches for detecting severe fibrosis and cirrhosis.^7–9^ Imaging modalities, however, are expensive, not universally available, depend upon operator experience,^10,11^ and often require additional appointments and transportation to off-site hepatology or radiology offices. These barriers prevent many individuals from being treated due to loss to follow-up before completing staging.^12^
Although less accurate to predict advanced fibrosis, serum tests are useful when simplified pathways are sought to eliminate a step in the HCV care cascade.^13–15^ AST-to-platelet ratio index [APRI] and Fibrosis-4 [FIB-4] are easily calculated from blood tests included in routine initial HCV evaluation, thus do not add additional costs or blood draws.^16,17^ Multiple commercial serum panels using proprietary formulas are extensively validated (i.e. FibroTest^™^, Biopredictive Paris, France/FibroSure^®^, Labcorp, Burlington, NC, USA), but require an additional blood draw and may be restricted by cost and availability.
Combining non-invasive fibrosis staging methods is one solution to balance these limitations and improve accuracy.^7,11^ A ‘staged’ strategy can also improve accuracy but reduce costs and loss to follow-up compared to using two tests for calculate a serum marker score (i.e., APRI or FIB-4) or a commercial serum test (i.e., FibroSure^®^), followed by TE only if there is significant uncertainty following the initial result (i.e., a result between the two cutoffs for APRI/FIB-4 or a FibroSure^®^ result of METAVIR F2/F3).^18^
Few data exist, however, to inform the sensitivity, specificity, negative predictive values (NPV) and positive predictive values (PPV) of using two non-invasive fibrosis staging tests together using community-level cirrhosis prevalence.^5^ Our study’s objective was to use data from major trials of non-invasive tests to simulate the probabilities of obtaining accurate results when combining tests using each of these strategies in low and high prevalence cirrhosis settings. These results can help inform clinical care, decision model analyses, and guidelines committees in comparing the clinical utility of these tests in various settings.
We included the four main fibrosis staging methods described in the American Association for the Study of Liver Diseases (AASLD)/Infectious Diseases Society of America (IDSA) guidelines for evaluation of liver disease prior to HCV treatment APRI, FIB-4, FibroSure^®^, and TE for LSM.^4^ These tests have been validated in large studies for detection of significant fibrosis and cirrhosis^9,16,17,19–22^ and are recommended by the WHO.^6^
We reviewed the literature for published METAVIR fibrosis stage-specific sensitivities and specificities for each of these four fibrosis staging methods. Many studies estimate sensitivity, specificity, and area under the receiver operating curve for detecting significant fibrosis (portal fibrosis with few septa; F2) or cirrhosis (F4), and a few list patient-level data in an appendix^22^ or tables.^23^ However, we found no studies using liver biopsy as gold standard that specifically detail test characteristics based on an individual’s specific METAVIR stage or that use population-level cirrhosis prevalence to translate sensitivity and specificity for detecting cirrhosis to PPV and NPV. Therefore, we combined literature data with de-identified data obtained directly from authors of seminal studies on fibrosis staging methods to determine METAVIR fibrosis stage-specific sensitivity, specificity, NPV and PPV for each selected fibrosis staging method (See Appendix section 1 for full details of included data). We combined these raw datasets where possible to increase sample size for more precise estimates of stage-specific test characteristics. Studies included patients with chronic HCV with or without human immunodeficiency virus (HIV) co-infection who had both liver biopsy and at least one of the non-invasive tests performed. We used METAVIR fibrosis stage cutoffs defined in seminal studies for each test (Appendix section 2). For each, we considered liver biopsy to be the gold standard and used the most recently agreed upon conversion table between pathologic Ishak fibrosis score and METAVIR fibrosis stage (Appendix Section 2). In a sensitivity analysis, we changed the main outcome to test characteristics for detecting advanced fibrosis (F3/F4).
To estimate NPV and PPV, we used the distribution of fibrosis stages of individuals with known chronic HCV from a cohort of adults who accessed care at an OCHIN (formerly the Oregon Community Health Information Network) U.S. federally qualified health center between 2012–2017^24–26^ (Appendix Section 3). In sensitivity analyses, we varied cirrhosis prevalence from base case 4.2% down to 1.1% and up to 25% to reflect different clinical settings^27,28^ (Appendix Section 3).
For each fibrosis staging method, we pooled data (when ≥1 study contributed to the analysis) to create counts of individuals at each METAVIR fibrosis stage by liver biopsy who were staged at each METAVIR fibrosis stage by the given test (Appendix Section 4). Next, we sampled with replacement from multinomial distributions parameterized from these counts to estimate new, bootstrapped probabilities of an individual being staged at each METAVIR stage for the given test. If there was no sample data to inform the probability of a cell, we added a hypothetical ‘half’ individual to the cell and recalculated row probabilities before bootstrapping.
We then simulated 10,000 hypothetical individuals with an assigned ‘true’ fibrosis stage based on cirrhosis prevalence described above and drew a ‘tested’ stage from the multinomial distribution probability table for each test. From this tested cohort of 10,000 individuals with now both a tested and true fibrosis stage for each non-invasive fibrosis method, we calculated the sensitivity, specificity, NPV, and PPV of each test to detect cirrhosis (METAVIR stage F4). We further calculated a median estimate and confidence bounds (CBs) around these estimates for each test characteristic by repeating this process 1000 times and calculating the 2.5^th^ and 97.5^th^ percentile estimates of probabilities and test characteristics from these 1,000 samples (Appendix Section 5).
Even fewer data predict stage-specific test characteristics for combining data from two tests. Studies detailing these characteristics do so using cirrhosis prevalence in the sample population, which usually reflects a cohort with more severe liver disease on average than the general HCV population.^19^ We therefore calculated test characteristics for a general population completing an initial serum test followed by a FibroSure^®^ or TE, 1) for everyone and 2) as a staged approach just for those with intermediate values on the first test. We also determined test characteristics for performing FibroSure^®^ followed by TE, 1) for everyone and 2) as a staged approach just for those with intermediate values on FibroSure^®^ (see Table 1 for list of strategies used and Appendix Section 2 for cutoffs defining intermediate, i.e., F2/F3 values).
The only data source we found with patient-level data for individuals who had each of calculated and commercial serum tests, TE, and liver biopsy results had only 14 cirrhotic patients.^22^ Therefore, to create more robust estimates and confidence intervals for completing two different tests, we created a simulated dataset to calculate these second-level test characteristics based on data from the individual tests described above.
Using the bootstrapped multinomial distribution tables created for individual tests, we simulated that each of the 10,000 hypothetical individuals in our cohort was exposed to each of two tests (Appendix Section 5). We determined the maximum drawn test stage for each hypothetical individual for each testing strategy from Table 1 as their final test result from that strategy. We used the maximum stage from the two tests as most clinicians would likely err on the side of caution in determining treatment and future screening if either test predicted cirrhosis (expert opinion). For staged strategies, we used the simulated dataset to draw stage-specific probabilities for the first (serum) test (APRI, FIB-4 or FibroSure^®^), and then only used the simulated second test result (FibroSure^®^ or TE) if the first test resulted in an intermediate (F2, F3) stage result. We then created 2×2 tables to indicate the number of true negative, true positive, false negative and false positive results for detecting cirrhosis (METAVIR F4) for each strategy. From these 2×2 tables, we calculated from this simulated cohort the sensitivity, specificity, NPV and PPV of the combined and staged test approaches for detecting cirrhosis. We repeated these steps 1,000 times to create 1,000 2×2 tables from which we calculated 1,000 estimates of each test characteristic to obtain CBs for these estimates (Appendix Section 5).
We repeated this process for each sensitivity 1) lower and higher population cirrhosis prevalence, and 2) using the outcome advanced fibrosis (Stage F3 or F4; >3.25 on FIB-4 or >1.5 on APRI) rather than cirrhosis (F4) as FIB-4 and APRI were designed to detect advanced fibrosis and because AASLD guidelines recommend HCC screening for individuals with a score over those thresholds.^29^
The Boston University Medical Campus Institutional Review Board approved this study as not human subjects research (#H-31190) as only de-identified data and statistical modeling were involved.
Raw counts and corresponding probabilities of individuals staged at each METAVIR fibrosis stage (or combined stages depending on validated stage cutoffs; Appendix Section 2) for each non-invasive staging method are shown in Appendix Section 4. Total Ns for each test after combining sources APRI, N=580; FIB-4, N=579, FibroSure^®^, N=101, TE, N=282. Results from bootstrapping probabilities 1,000 times for a 10,000-hypothetical person simulated population are shown in Appendix Section 6 for individual tests and each combined strategy.
Calculated test sensitivities for cirrhosis for individual tests ranged from 61.2% (95% CBs, 47.7–74.6%) for FIB-4, to 87.1% (95% CB, 77.2–95.0%) for TE (Table 2). Specificities ranged from 73.9% (95% CB, 58.7–84.4%) for FibroSure^®^ to 96.4% (95% CB, 94.8–97.9%) for TE. Using a US population cirrhosis prevalence of 4.2%, NPV for single tests ranged from 98.2% (95% CB 97.6–98.8%) for FIB-4 to 99.4% (95% CB 99.0–99.8%) for TE, and PPV from 12.2% (7.7–19.7%) for FibroSure^®^ to 51.3% (41.4–64.1%) for TE.
Combined two-test strategy probabilities of being staged correctly or under- or over-staged based on ‘true’ biopsy-defined METAVIR fibrosis stage are shown in Appendix Section 6. Staged approach strategies, with the second test only performed in those with intermediate (F2/F3) results on the first test, yielded substantially higher sensitivities for detecting cirrhosis (where a result of F4 on either test qualifies someone as having cirrhosis): range 92.6% (95% CB 82.3–98.3%) with FIB-4 then FibroSure^®^ to 96.9% (95% CB 82.6–99.7%) for FibroSure^®^ then TE, compared with using any of the single tests alone (Table 2). Moving to a two-test strategy for all approach improved sensitivities for detecting cirrhosis only slightly over the staged approach (range 93.5% [95% CB 83.5–98.6%] for FIB-4 then FibroSure^®^ to 97.8% [95% CB 93.0–99.8%] for FibroSure^®^ then TE). Specificity for cirrhosis (where again, a result of F4 on either test categorizes someone as cirrhotic) remained similar when adding TE to APRI or FIB-4 (dropped by 1–3% with overlapping CBs for either staged or two tests for all approach, compared with performing APRI or FIB-4 alone). Adding FibroSure^®^ to APRI or FIB-4, however, decreased the specificity by 10–24%, with specificity only 60.2% (95% CB, 51.2–69.4%) for APRI then FibroSure^®^ for all and 68.5% (95% CB, 57.9–77.3%) for FIB-4 then FibroSure^®^ for all.
NPV improved to 99.6% (96% CB, 99–100%) for all strategies combining calculated and commercial serum tests, and to 99.7–99.9% (95% CB all within 99.0–100%) for all strategies using TE as a second test. PPV remained low for all two test strategies, with FIB-4 then TE having the highest PPV values of 31.8% (95% CB, 27.1–37.6%) for staged approach and 29.5% (95% CB 25.3–35.0%) in a two test for all approach. All strategies including APRI or FibroSure^®^ had PPVs <20%.
In the simulated higher prevalence cirrhosis cohort (20% of population cirrhotic, 20% divided into each of the other METAVIR fibrosis stages), sensitivities remained similar for all strategies as expected with just stochastic differences due to the simulation methods (Table 3). Specificity decreased across strategies. NPV decreased when compared with base case, particularly in single test strategies, but remained 90.7–96.5% (95% CB 87–100%) for single tests, 97.5–97.7% (95% CB 94–99%) for strategies involving FibroSure^®^ as test two, and 98.1–99.3% (95% CB 94–100%) for strategies with TE as test two. PPV was higher, as would be expected with higher prevalence of the outcome, ranging between 35–50% for strategies involving APRI or FibroSure^®^, between 58–61% for FIB-4 and TE combined strategies, and 75.4% with TE alone. Overall, NPV dropped slightly and PPV increased as cirrhosis prevalence increased from 1.1% to 25% for FIB-4 alone and FIB-4 and TE combined strategies (Figure 1).
In sensitivity analyses determining test characteristics to detect the outcome of advanced fibrosis (METAVIR stage F3 or F4) rather than cirrhosis (F4 alone), we found overall similar results (Appendix Section 7). However, NPVs were slightly lower across single tests (range 95.2% [95% CB 94.3–96.2%] with FIB-4 to 97.6% [94–100%] with FibroSure^®^) and staged approach strategies (range 95.2% for either FIB-4 strategy to 98.3% for FibroSure^®^ then TE) compared with two-test for all strategies (range 98.3–99.0%).
In this study, we combined raw data sources and simulation approaches to generate estimates of negative and positive predictive values for identifying advanced fibrosis and cirrhosis using common non-invasive fibrosis staging methods alone or in combination. We found that even in the setting of a high cirrhosis prevalence (20–25%), combining two non-invasive fibrosis staging modalities, particularly if completing the second test for all individuals, only marginally improves NPV estimates and actually decreases or does not change PPVs. Sensitivities improved when adding a second method with a higher baseline sensitivity as would be expected, but on a population level, these changed the clinical significance of a negative test (NPV) by very little. Given additional cost, lack of universal availability and on-site accessibility of TE, benefits of requiring all patients to complete TE as part of staging before beginning HCV treatment likely does not outweigh the risk of loss to follow-up. Similarly, with the cost and potential need for additional phlebotomy, systematically performing FibroSure^®^ following APRI/FIB-4 as part of staging prior to HCV treatment does not appear to be warranted, particularly given the decreased PPV and specificity it yields.
We also calculated stage-specific probabilities of each non-invasive test, alone or in combination with other modalities, to predict cirrhosis, along with CBs around these estimates. Our calculated test characteristics for individual tests are similar to those reported in the literature in meta-analyses and larger studies,^19,30^ and those for two test strategies provide new data to supplement smaller studies calculating these test characteristics.
Together, these data can help inform clinicians, policymakers, and guideline committees to determine the best fibrosis staging tests to use in a particular setting to maximize clinical benefits and HCV cure. They can also be used to populate decision models for researchers comparing downstream consequences of cirrhosis-related surveillance for HCC and esophageal varices. For example, what are the economic consequences and emotional toll of over-screening those falsely determined to have cirrhosis, and what are the clinical and economic risks of missing a cirrhosis diagnosis compared with the consequences of not treating a person with HCV because they are lost to follow-up before completing staging? These health economic investigations will be valuable for individual clinicians, insurers, and guideline committees in determining which modalities to recommend or require for all or subsets of patients. For example, in the U.S., Arkansas, Illinois, and Maryland Medicaid programs specifically require imaging (TE or shear wave elastography), a commercial panel (FibroSure^®^ or FibroMeter), or liver biopsy to satisfy their DAA prior authorization process (personal communications with individual state Medicaid program pharmacy representatives). Other states require imaging or commercial panels for certain populations or treatment choices. This elevates the cost per cure with potentially marginal benefit and the possibility of greater harm (if loss to follow-up prevents or delays cure). Our data could be applied to various settings based on clinic or population cirrhosis prevalence to determine the likelihood of and consequences of a false negative or positive test in each specific setting.
Overall, two-test strategies did increase test sensitivity, but staged strategies (only completing the second test for individuals staged F2/F3 on the first test) performed nearly as well as two tests for all strategies in sensitivity, NPV and PPV. This suggests that benefits of performing two tests for all individuals are minimal; staged strategies yield nearly all the benefits but involve fewer off-site imaging referrals and phlebotomy draws, and presumably less loss to follow-up and lower cost per cure. Furthermore, in settings such as the U.S. with a population prevalence of cirrhosis of <5%, NPV of even calculated serum tests (APRI, FIB-4) exceeds 98%, calling into question the necessity of more expensive and involved testing for fibrosis assessment for individuals for HCV, particularly those at low clinical risk for cirrhosis based on younger age or shorter duration of suspected infection. On an individual patient-level, as non-invasive tests are only validated for pre-treatment HCV evaluation (not after cure), some clinical situations may call for obtaining a more sensitive test before HCV treatment (e.g., a patient at high risk for cirrhosis with a low FIB-4 score)^31,32^. However, these data contest the utility of doing so for the low-risk patient with a correspondingly low FIB-4 when an additional test may lead to loss-to-follow up before treatment.
Our study is limited by the data and methods we utilized. Despite combining literature and primary trial data from multiple sources, we still had modest sample sizes for certain tests and true fibrosis stages. We also did not have sufficient primary data to help inform results of two-test strategies and therefore simulated independent probabilities of predicting cirrhosis on each test. While this limits the accuracy of results, the numerous iterations and large simulated population size with a true population prevalence of cirrhosis strengthens our findings compared with smaller individual studies with falsely high cirrhosis prevalence due to convenience samples. We also display CBs that reflect the degree of uncertainty based on the original sample size to allow estimation of uncertainty through probabilistic sensitivity analyses for use as parameters in modeling studies. Some individuals are less likely to have accurate fibrosis staging for certain tests due to extremes of age, obesity, operator experience, etc.,^10,11,33^ and we were not fully able to account for this. However, we do represent some degree of uncertainty for the populations the data reflect using confidence bounds. Although we included both HIV/HCV co-infected and HCV mono-infected populations in our study data, multiple studies have demonstrated excellent sensitivity and specificity of non-invasive fibrosis staging tests in HIV co-infected patients,^19,34,35^ and coinfected patients are an important subpopulation that could benefit from inclusion to ensure applicability of these data. Finally, we did not include shear-wave elastography because we found very limited person-level data.^23^ This study demonstrates that combining results from two fibrosis staging tests improves NPV for ruling out cirrhosis only minimally compared with using a single test, and that performing a second test in all yields virtually no clinical benefit over the staged approach. In many settings, user-friendly, on-line calculators can enable clinicians to reliably exclude cirrhosis and promptly treat and cure more HCV. To facilitate achievement of US and WHO HCV elimination goals, primary care physicians and other non-specialist clinicians may be able to evaluate and treat patients with the tools at hand.^36,37^ In the U.S, only one quarter of insured people diagnosed with HCV are treated in the first year after diagnosis,^38^ and even fewer uninsured individuals have been treated.^25^ The complexity and protracted timeline of navigating the continuum of care is a major factor, with drop-offs at each incomplete HCV screening, diagnosis, linkage, fibrosis evaluation or treatment. We need more efficient, decentralized strategies that actualize test-to-treat evidence.^15^ Paradoxically, non-specialists may be more reliant on TE without the clinical experience to appreciate nuances of different testing options. Routine TE may be ordered in lieu of inexpensive blood tests, and patients may be lost to follow-up and cure if they cannot access off-site TE or make another appointment.^14^
The World Health Organization (WHO) guidelines endorse APRI and FIB-4 use in resource-limited settings.^6^ The AASLDIDSA Simplified HCV Treatment Algorithm for Treatment-Naive Adults Without Cirrhosis also relies on FIB-4, as do society guidelines for non-specialist care of other major liver diseases such as metabolic dysfunction-associated steatotic liver disease (MASLD).^3,4,39^ This study adds to mounting evidence that indirect serum biomarkers or at minimum a single non-invasive test should be standard of care in pre-treatment evaluation of most individuals with HCV given minimal increases in NPV and decreases in PPV when combining two methods. Simplifying hepatic fibrosis evaluation as the US and WHO Elimination Programs recommend, using inexpensive, non-invasive serum methods could promote decision-making by non-specialist physicians and other clinicians, expanding capacity for care and curbing public health expenses and waste in health care spending.^40^