Authors: Amy Board (1Division of Birth Defects and Infant Disorders, National Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, Atlanta, Georgia), Alana Vivolo-Kantor (2Division of Overdose Prevention, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, Atlanta, Georgia), Shin Y. Kim (1Division of Birth Defects and Infant Disorders, National Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, Atlanta, Georgia), Emmy L. Tran (2Division of Overdose Prevention, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, Atlanta, Georgia), Shawn A. Thomas (1Division of Birth Defects and Infant Disorders, National Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, Atlanta, Georgia), Mishka Terplan (3Friends Research Institute, Baltimore, Maryland), Marcela C. Smid (4University of Utah Health, Salt Lake City, Utah), Pilar M. Sanjuan (5Department of Family and Community Medicine, University of New Mexico Health Sciences Center, Albuquerque, New Mexico), Tanner Wright (6University of South Florida, Tampa, Florida), Autumn Davidson (7Center for Health Research—Northwest, Kaiser Permanente, Portland, Oregon), Elisha M. Wachman (8Department of Pediatrics, Boston Medical Center, Boston, Massachusetts), Kara M. Rood (9The Ohio State University, Columbus, Ohio), Diane Morse (10University of Rochester School of Medicine, Rochester, New York), Emily Chu (11Department of Gynecology and Obstetrics, Emory University School of Medicine, Atlanta, Georgia), Kathryn Miele (1Division of Birth Defects and Infant Disorders, National Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, Atlanta, Georgia; 11Department of Gynecology and Obstetrics, Emory University School of Medicine, Atlanta, Georgia)
Categories: Article
Source: American journal of preventive medicine
Authors: Amy Board, Alana Vivolo-Kantor, Shin Y. Kim, Emmy L. Tran, Shawn A. Thomas, Mishka Terplan, Marcela C. Smid, Pilar M. Sanjuan, Tanner Wright, Autumn Davidson, Elisha M. Wachman, Kara M. Rood, Diane Morse, Emily Chu, Kathryn Miele
As perinatal drug overdoses continue to rise, reliable approaches are needed to monitor overdose trends during pregnancy and postpartum. This analysis aimed to determine the sensitivity, specificity, positive predictive value, and negative predictive value of ICD-9/10-CM codes for drug overdose events among people in the MATernaL and Infant clinical NetworK (MAT-LINK) with medication for opioid use disorder during pregnancy.
People included in this analysis had electronic health record documentation of medication for opioid use disorder and a known pregnancy outcome from January 1, 2014, through August 31, 2021. Data were analyzed during pregnancy through 1 year postpartum. The Centers for Disease Control and Prevention’s drug overdose case definitions were used to categorize overdose based on ICD-9/10-CM codes. These codes were compared to abstracted electronic health record data of any drug overdose. Analyses were conducted between May 2023 and May 2024.
Among 3,911 pregnancies with electronic health record−documented medication for opioid use disorder, the sensitivity of ICD-9/10-CM codes for capturing drug overdose during pregnancy was 32.7%, while specificity was 98.5%, positive predictive value was 23.4%, and negative predictive value was 99.0%. The sensitivity of ICD-9/10-CM codes for capturing drug overdose postpartum was 30.9%, while specificity was 98.4%, positive predictive value was 25.9%, and negative predictive value was 98.8%.
The sensitivity and positive predictive value of ICD-9/10-CM codes for capturing drug overdose compared with abstracted electronic health record data during the perinatal period was low in this cohort of people with medication for opioid use disorder during pregnancy, though the specificity and negative predictive value were high. Incorporating other data from electronic health records and outside the healthcare system might provide more comprehensive insights on nonfatal drug overdose in this population.
Opioid use disorder (OUD) during pregnancy and postpartum is a growing public health concern. From 1999 to 2014, the rates of OUD at delivery increased from 1.5 to 6.5 per 1,000 delivery hospitalizations in the U.S.^1^ OUD during pregnancy and postpartum is associated with a number of adverse health outcomes,^2^ including drug overdose, a leading cause of pregnancy-associated mortality.^3^ From 2018 to 2021, perinatal overdose mortality rates nearly doubled among those aged 10 to 44 years and tripled among those aged 35 to 44 years.^4^ Treatment with medication for OUD (MOUD), including methadone and buprenorphine, is the standard of care for pregnant people with OUD and has been found to reduce the likelihood of experiencing an overdose event during pregnancy or postpartum.^5–7^
To better understand all-cause mortality, including drug overdose, occurring within 1 year of the end of a pregnancy, Maternal Mortality Review Committees (MMRCs) in nearly every U.S. state, territory, and freely associated states have on-going review of deaths occurring during pregnancy or within 1 year of pregnancy.^8,9^ These multidisciplinary committees perform a detailed review of a broad array of records, including medical and social service records, to provide a deeper understanding of pregnancy-related mortality.^8^ MMRCs are focused on mortality and do not generally provide surveillance of nonfatal overdose.^8^ However, information on nonfatal overdoses in this population is critical for public health departments to provide resources and respond to emerging overdose trends.^10^
Electronic health records (EHRs) used in healthcare settings contain standardized diagnosis coding that is used primarily to confirm clinical diagnoses for billing purposes. However, these standardized codes (i.e., ICD-9-CM and ICD-10-CM) can assist in classifying overdoses treated by healthcare providers.^11,12^ The standardized ICD-9/10-CM codes from EHRs can also be leveraged in conjunction with unstandardized, free text within other medical record fields including the chief complaint to monitor nonfatal overdose.^13^ Together, these approaches are used to track nonfatal overdose trends in near real time and can detect regional differences to enhance overdose prevention programs.^14^ However, these approaches might perform differently with regard to the types of substances involved in a drug overdose. The Centers for Disease Control and Prevention (CDC)’s Drug Overdose Surveillance and Epidemiology (DOSE) system found that analyses using chief complaint and ICD-10-CM coded data identified more overdoses involving any opioids and any stimulants and fewer overdoses involving heroin specifically when compared to ICD-10-CM coded data alone.^14^
Limited data suggest that ICD-9/10-CM codes from EHRs might have low sensitivity but high positive predictive value (PPV) for identifying drug overdose in the general U.S. population.^15–18^ Data from EHRs could be mined to better understand trends in drug overdose morbidity and guide response efforts for subpopulations of interest. However, it is unknown how well ICD-9/10-CM codes approximate the true prevalence of drug overdose incidents for pregnant and postpartum people.
Using data from the MATernaL and Infant clinical NetworK (MAT-LINK), this analysis aimed to determine the sensitivity, specificity, PPV, and negative predictive value (NPV) of ICD-9/10-CM codes for overdose events compared to data abstracted from EHRs among people with documented MOUD during pregnancy, a population with limited data sources available for reliably estimating overdose prevalence.^19,20^
MAT-LINK conducts longitudinal surveillance of pregnant people−infant linked dyads using EHRs from geographically diverse U.S. clinical sites, including a cohort of people with documented MOUD during pregnancy. This analysis with data from 7 clinical sites included people with at least 1 EHR-documented MOUD during pregnancy and a known pregnancy outcome between January 1, 2014, and August 31, 2021. The 7 clinical sites included Boston Medical Center, Kaiser Permanente Northwest (Oregon and Washington), The Ohio State University, the University of New Mexico, the University of Rochester, the University of South Florida, and the University of Utah. Clinical sites used ICD-9/10-CM codes to identify documentation of OUD during pregnancy as well as pregnancy outcomes that occurred during the cohort period. A list of ICD-9/10-CM codes and more detailed information regarding case ascertainment has been described elsewhere.^21^ EHR-documented MOUD during pregnancy included methadone, buprenorphine with or without naloxone, and naltrexone; buprenorphine formulations used for the management of chronic pain were not considered MOUD for the purposes of this cohort. RxNorm and SNOMED CT codes were used to categorize information on MOUD during pregnancy.
To identify pregnancies with an overdose event, clinical sites electronically extracted ICD-9/10-CM codes and manually abstracted overdose-related data from available outpatient and inpatient records during each pregnancy, at delivery, and through 1 year postpartum (Figure 1). Extracted ICD-9/10-CM code data were collected via XML data extraction tools from clinical site EHRs or clinical data warehouses (CDWs). Abstracted data were obtained by pulling individual records for review by a manual abstractor at each clinical site, who then reviewed all available data in the EHR/CDW, including ICD-9/10-CM codes, laboratory test results, problem list, and clinical notes, as well as data from outside sources as available in the EHR. Based on the information reviewed from the EHR, abstractors submitted information to CDC via Research Electronic Data Capture (REDCap) to report whether or not an overdose occurred, the date of the overdose, and the substance(s) involved in the overdose. Sites selected from a list of 24 opioids and 12 non-opioid substances; if the substance was not included in the list, it was reported via a free text field. Clinical sites also conducted dual entry verification for 10% of their records to perform a data quality check and compare the abstracted data to what was expected based on clinical experiences. Additional details about the methods of abstraction and extraction in the MAT-LINK system have been described elsewhere.^21^ Because the surveillance period spanned several years, people who had multiple pregnancies during the cohort time period were represented more than once in the dataset. Multiple gestation pregnancies (i.e., twins, triplets) were only included once in the data.
Drug overdose was categorized in the extracted data using CDC’s DOSE overdose case definitions (Appendix Table).^11,22^ CDC’s chief complaint text definitions were not used in this analysis. Alcohol and cannabis T-codes were not included due to low counts of reported overdose events.
The ICD-9/10-CM code categories were compared to documentation of any drug overdose abstracted from the patient’s outpatient or inpatient records in the EHR, a more detailed data source which has been used as a comparison group in other studies examining the performance of ICD-9/10-CM codes.^15,17,18^ Because ICD-9/10-CM codes may underestimate overdose events and other EHR sources such as the chief complaint field might overestimate overdose events,^14^ using a manual abstractor familiar with the EHR system to examine all available sources of data in the EHR was considered the “gold standard” for this analysis. The total number of drug overdoses and rate per 1,000 pregnancies during pregnancy and postpartum were calculated, in addition to performing separate calculations to estimate the number and rate by the specific type of substances involved in the drug overdose. Sensitivity, specificity, PPV, and NPV were calculated using the formulas below for (1) evidence of any drug overdose and (2) drug overdose involving any opioids. Sensitivity, specificity, PPV, and NPV were calculated separately for the pregnancy period and the one-year postpartum period.
Sensitivity = True Positive/(True Positive + False Negative)
Specificity = True Negative/(True Negative + False Positive)
PPV = True Positive/(True Positive + False Positive)
NPV = True Negative/(True Negative + False Negative)
In the formulas above, “true positive” refers to an overdose captured in both abstracted and ICD-9/10-CM code data, “true negative” refers to no documentation of an overdose in both abstracted and ICD-9/10-CM code data, “false positive” refers to an overdose documented in ICD-9/10-CM code data but not in abstracted data, and “false negative” refers to an overdose documented in abstracted data but not in ICD-9/10-CM code data.
Data analyses were performed between May 2023 and May 2024 using Python v3.8.8 and SAS v9.4 (SAS Institute, Cary, NC). Because MAT-LINK data are protected under an Assurance of Confidentiality,^23^ cell counts of less than 5 were suppressed. This activity was determined to be non-research and exempt from undergoing a review from CDC’s IRB and was conducted consistent with applicable federal law and center policy (e.g., 45 CFR part 46.102.l.2; 21 CFR part 56; 42 USC §241.d; 5 USC §552a; 44 USC §3501 et seq). All MAT-LINK clinical sites underwent IRB review and received approval or exemption prior to data collection.
Out of 3,911 pregnancy episodes with MOUD, 55 had a documented drug overdose event during the pregnancy period (overdose rate 14 per 1,000 pregnancies) using EHR abstraction compared to 77 using ICD-9/10-CM codes (20 per 1,000 pregnancies) (Table 1). Postpartum, 68 had a documented drug overdose event using EHR abstraction (17 per 1,000 pregnancies) compared to 81 using ICD-9/10-CM codes (21 per 1,000 pregnancies). Nearly all recorded overdoses were nonfatal; the number of fatal overdoses were too small to report based on CDC’s Assurance of Confidentiality.
According to both abstracted and ICD-9/10-CM code data, opioids were the most common substance involved in a drug overdose. The second most common substance was sedatives according to abstracted data and “other substances” according to ICD-9/10-CM codes; of ICD-9/10-CM overdoses classified as involving “other substances,” 66.7% of pregnancy overdoses and 80.0% of postpartum overdoses had an ICD-9/10-CM code indicating an undetermined or otherwise unclassified substance.
The sensitivity of ICD-9/10-CM codes for capturing any drug overdose during pregnancy was 32.7%; specificity was 98.5%, PPV was 23.4%, and NPV was 99.0% (Table 2). The sensitivity of ICD-9/10-CM codes for capturing any drug overdose during the 1-year postpartum period was 30.9%, while specificity was 98.4%, PPV was 25.9%, and NPV was 98.8%. When restricted to opioid-involved overdoses, sensitivity increased during the postpartum period to 37.9%, yet remained similar during pregnancy (31.4%).
In this analysis of primarily nonfatal drug overdose events among people with MOUD during pregnancy, both sensitivity and PPV of ICD-9/10-CM codes for drug overdose were low when compared with abstracted EHR data. This suggests that ICD-9/10-CM codes might not reliably identify overdose events during pregnancy and the first year postpartum. In this analysis, relying on only ICD-9/10-CM codes resulted in a number of missed overdose events (false negatives) or incorrectly identified healthcare encounters as overdose events (false positives). While prior research has also noted low sensitivity of ICD-9/10-CM codes compared to abstracted EHR data in detecting overdose,^18,24^ a high PPV had previously been observed,^15,17,25^ which was not reflected in this analysis. Differences in this cohort (perinatal people receiving MOUD during pregnancy across 7 geographically diverse clinics) might partially explain this. For example, because this is a population at high risk of having experienced a drug overdose,^26,27^ the high number of false positives observed among ICD-9/10-CM codes in this analysis could be due to historical overdose codes that do not represent an acute event.^28^ Some documented reasons for this include the growing use of clinical documentation improvement programs to ensure all potential relevant codes related to the encounter are captured, potentially leading to the inclusion of codes corresponding to past conditions that were not present in the encounter; EHR functionality, including the problem list, which might carry forward previous diagnoses; and increasing reliance on clinicians to identify the ICD-9/10-CM codes associated with the encounter rather than medical billing professionals who are trained in the coding process.^28^ Such false positives therefore might not reflect the potentially protective effects of MOUD on experiencing an overdose event.^6,7^
In addition to opioids, sedatives such as benzodiazepines were frequently involved in overdose events. Similar patterns have been observed in other studies of nonfatal overdose data,^12,27^ although current trends in the general U.S. population indicate stimulants are the second leading substance type involved in an overdose death, following opioids.^29^ This analysis also noted a sizable number of observations with an ICD-9/10-CM code for “other and unspecified drug.” Given that ICD-9/10-CM codes are intended for billing and reimbursement, they may not capture detailed information about substances involved in an overdose. In addition, ICD-9/10-CM codes might not be specific enough to capture all substance types; in this analysis, information gleaned from abstracted EHR data fields such as laboratory test results and clinical notes allowed for the identification of more specific substances used. Therefore, harnessing data from additional EHR data fields, instead of relying solely on ICD-9/10-CM codes, might improve understanding of substance-specific overdose trends.
While abstracted EHR data is derived from additional sources beyond ICD-9/10-CM codes, data abstraction can be time-consuming and costly. Automated approaches could be explored to more accurately and efficiently examine overdose prevalence in this population. Alternatively, combining ICD-9/10-CM coded data with other data sources or strategies could enhance reliability for detecting opioid-involved overdoses specifically, such as incorporating chief complaint text keywords,^14^ pulling only those records with a first-listed diagnosis indicating an overdose,^17^ or incorporating information on naloxone administration.^25^ Prehospital data from first responders or Emergency Medical Services (EMS) is another source that could potentially be used to understand drug overdose in the perinatal population. EMS data can identify nonfatal overdoses that involve EMS and occur in a community setting among people who did not receive follow-up care in an emergency room or hospital setting.^30^ With increasing drug overdose mortality in the perinatal population,^4^ efforts to identify and validate potential data sources to monitor these trends can provide valuable information on opportunities to reduce overdose morbidity and mortality.
People in this analysis all had some engagement with the health care system as evidenced by at least 1 prescribed or dispensed MOUD. MOUD can reduce the risk of opioid overdose and is the standard of care for people who have OUD during pregnancy.^5^ Drug overdose rates in this analysis were slightly higher in the postpartum period than the pregnancy period, which is consistent with other studies that found that overdose mortality is highest from 43 days to 12 months after delivery.^31,32^ Therefore, pregnancy can be an opportune time to initiate MOUD and additional OUD management support that continues into the postpartum period, including naloxone provision and linkage to peer recovery and harm reduction services.
There were several noted limitations. First, billing code practices vary among clinicians and health systems, which may have contributed to differences between ICD-9/10-CM codes and abstracted data. Additionally, clinical sites might have differed in their approaches to data abstraction and underlying data sources available in their EHRs; however, it is unlikely that this would have led to any systematic biases in our analysis. Furthermore, the majority of sites used the same EHR system (Epic), which may have minimized differences among sites in the data system architecture used for abstraction. Because EHR documentation of lack of an overdose is inconsistent and typically includes overdoses treated in the emergency department rather than a community setting,^16^ it is possible that some overdose events were not captured in this analysis, which may have biased these results. This may have been more likely during the post-partum period when loss to follow-up due to relocation, care discontinuation, or death may have occurred. Additionally, as this cohort exclusively included individuals with MOUD reported during pregnancy, there is likely a bias toward ascertainment of opioid overdoses compared to other types of drug overdoses. Well-documented disparities exist in MOUD access and receipt during pregnancy, with people who are Black, Hispanic, and living in rural areas less likely to receive MOUD during pregnancy.^33–35^ Therefore, this sample and associated findings likely underrepresent the overdose experiences of these groups. Additionally, since drug overdoses often occur in a community setting and people do not always seek follow up care in a healthcare facility,^36^ there is no systematic way to capture nonfatal overdoses that did not get recorded in an EHR. Therefore, overdoses documented in the EHR may underestimate the true overdose rate in this population.
Furthermore, some substances involved in an overdose might not be identified via ICD-9/10-CM codes or might be assigned a non-poisoning ICD-9/10-CM code that was not included in this analysis.^36^ It should be noted that while this analysis contained some ICD-9-CM codes, the majority were ICD-10-CM codes, which were introduced on October 1, 2015. ICD-10-CM codes are more expansive and specific than ICD-9-CM codes.^37^ Previous studies that combined ICD-9-CM and ICD-10-CM codes found relatively high PPV of overdose codes (67.0%−81%) when compared to medical chart abstraction or survey data.^15,16^ Therefore, it is unlikely that the inclusion of ICD-9-CM codes along with the majority ICD-10-CM codes negatively influenced the estimates in this analysis.
Relying on ICD-9/10-CM codes alone may misclassify overdoses among pregnant and postpartum people. Incorporating other data sources from the EHR and integrating other sources outside of the healthcare system might provide a more comprehensive picture of nonfatal drug overdose in this population. As overdose morbidity and mortality continues to increase in the perinatal population, quickly and accurately identifying overdose events and monitoring trends across time is critical for effective overdose mitigation and response efforts.