Authors: Mark D. Neuman, Rui Feng, Aesha S. Shukla, Xiaoyan Han, Annamarie D. Horan, Karah Whatley, Marilyn M. Schapira, Edward R. Marcantonio, Richard P. Dutton
Categories: Original Investigation, Research, Online Only, Anesthesiology
Source: JAMA Network Open
This randomized clinical trial examines the efficacy of 3 clinician- and patient-facing strategies to reduce the use of benzodiazepines in anesthesia among older adults undergoing surgery.
Over 21 million surgical procedures are performed annually for US adults aged 65 years or older.^1,2^ Over half of these individuals receive a benzodiazepine, such as midazolam, during anesthesia care,^3^ typically with the intent of improving patient experience.^4,5^ Yet randomized clinical trials have demonstrated that benzodiazepines do not improve satisfaction with anesthesia compared with placebo.^6,7^ Current guidelines recommend against administering benzodiazepines to older patients due to potential adverse effects.^8,9^
Past studies on clinician-facing behavioral strategies to discourage benzodiazepine administration to older adults during anesthesia care used single-center, nonrandomized designs.^10,11^ Moreover, no study has examined patient-facing interventions. Generalizable evidence on effective patient and clinician interventions for reducing benzodiazepine administration to older patients is needed to inform quality improvement efforts.
We conducted a randomized clinical trial of a corporate quality improvement initiative (De-adopting Routine Preoperative Benzodiazepines for Older Surgical Patients [DROP-Benzo]) for a private medical group. The objective of this trial was to evaluate the effect of clinician peer comparison, patient informational mail, or a combination of these interventions compared with usual care on the rate of perioperative benzodiazepine administration to older patients. We hypothesized that patients receiving a study intervention would be less likely to receive a benzodiazepine during anesthesia care vs usual care.
DROP-Benzo, a 2 × 2 factorial, stepped-wedge cluster randomized clinical trial, was conducted in 22 divisions of a national anesthesia private practice (US Anesthesia Partners) between August 8, 2022, and May 28, 2023. The University of Pennsylvania Institutional Review Board approved the trial protocol and waived the informed consent requirement because the study posed no more than minimal risk to participants and could not be carried out practicably without such a waiver and the waiver was judged to not adversely affect the rights or welfare of the participants. The protocol and statistical analysis plan are provided in Supplement 1. The analysis plan was finalized prior to initiating data analysis. We followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.
During the study period, approximately 4200 anesthesiologists, certified registered nurse anesthetists (CRNAs), and certified anesthesiologist assistants across 22 practice divisions provided contracted anesthesia services to 415 hospitals, surgery centers, and physician offices (care locations) in 8 states (eTable 1 in Supplement 2). We planned the trial with 24 divisions encompassing 423 care locations; however, 2 divisions (8 care locations) left the practice before the start date and were not included in the study.
Patients were included if they were aged 65 years or older and undergoing a scheduled surgical or endoscopic procedure with general anesthesia; we excluded urgent or emergent cases because we could not reliably deliver the patient-facing interventions to this group. Additionally, we excluded patients who received a nerve block or who exclusively received monitored anesthesia since benzodiazepines may be appropriate for periprocedural sedation in these cases.
We randomly assigned patients in the 22 divisions to 1 of 4 treatment sequences (ie, 6 divisions per sequence) (Figure 1). Each sequence comprised a unique 5-period intervention implementation scheme (eFigure 1 in Supplement 2), resulting in 110 division-period clusters. Sequence assignments were made with stratified randomization across 6 strata comprising 4 divisions each using the fixed-block randomization algorithm. We defined the strata based on prior-year eligible case volumes within each division. At the time of study planning, the partner organization was in the process of establishing electronic medical record (EMR) data access for quality monitoring within 4 divisions. We assigned these divisions to a single stratum to permit future analyses of EMR data if these data were to become available.
Figure 1. Patient Screening, Enrollment, and Follow-Up
At the start of the trial, the study statistician (R.F.) communicated each division’s assigned treatment sequence via email to investigators at the partner organization (R.P.D. and A.S.S.), who then oversaw implementation of study procedures according to this assignment. We maintained the randomization algorithm, code, and original sequence on a secure server, which were accessible to the study data management team (R.F. and X.H.) and unknown to the principal investigator (M.D.N.) and remaining study personnel. We did not mask clinicians or patients to treatment assignment.
In the initial 10 weeks, no divisions received either intervention (clinician peer comparison or patient informational mail). We had initially planned an 8-week duration for this initial period, but an additional 2 weeks were permitted at the request of the partner organization to accommodate scheduling conflicts. Each intervention (peer comparison feedback or the patient informational mail) was subsequently initiated at a new group of practices at 8-week intervals. We rolled out interventions in an overlapping fashion (eFigure 1 in Supplement 2) such that each sequence had at least 1 period in which the assigned divisions received only 1 of the study interventions prior to adding the second intervention. Involved clinicians were notified of the project by email from practice leaders at the beginning of the trial, and relevant practice guidelines^8,9^ were made available via the practice website. At 63 care locations across 8 divisions, clinicians used electronic anesthesia record systems that allowed the quality management group at the practice to directly track benzodiazepine administration. At the remaining 352 care locations, benzodiazepine administration was tracked via required electronic self-reporting through the practice’s electronic billing and quality management platform, which was introduced between May and July 2022.
Anesthesiologists, CRNAs, and anesthesiologist assistants at care locations received monthly individual reports via their professional email address from corporate accounts, which were regularly used to communicate practicewide updates. These reports showed the clinician’s fraction of general anesthesia cases for patients aged 65 years or older to whom the clinician administered a benzodiazepine during anesthesia care. Each clinician was compared to all other clinicians in the practice to establish a uniform standard for improvement across the practice. Comparisons used a 3-month rolling mean of data from the practice’s electronic billing and quality management platform using the electronic anesthesia record system and clinician attestation. Following the recommendations for design of clinician peer comparison,^12,13,14^ we compared clinicians with performance below the group median scores to the median percentile of performance, whereas clinicians above this level were compared with the 10th percentile of performance. Clinicians at or above the 10th percentile were notified of their scores and encouraged to continue high performance (eFigures 2-4 in Supplement 2).
We designed electronic patient mail with input from researchers, practice leaders, and community members. Based on this input, the intervention consisted of an informational letter encouraging patients to engage in discussions with their anesthesia clinicians on the day of surgery regarding medication selection (eFigure 5 in Supplement 2). In units of patients assigned to receive this intervention, the mail was distributed via email 2 weeks prior to surgery, printed on corporate letterhead, and was signed by the practice chief quality officer (R.P.D.). Mail was available in both English and Spanish formats. We assessed patient engagement by the percentage of mail opened out of all the emails sent at 4 practice divisions.
The primary outcome was a binary variable indicating benzodiazepine administration by an anesthesia clinician at any time during anesthesia care. Outcome data were collected via an electronic anesthesia record system and clinician attestation. For patients undergoing general anesthesia, benzodiazepines are most often used as premedication prior to anesthesia induction; thus, we assumed this outcome would reflect preoperative benzodiazepine use. However, we did not explicitly distinguish between preinduction administration vs administration at other times during anesthesia care.
We measured patient satisfaction using 8 items from the Anesthesia Patient Satisfaction Questionnaire, version 2 (APSQ-2).^15^ These 8 items rate overall anesthesia experience and aspects of interactions with the anesthesia team on a scale of 1 to 5 (score 1-5, with higher scores indicating greater satisfaction). We dichotomized responses for analysis into favorable vs neutral or unfavorable based on selection of either of the 2 highest categories for each item. Satisfaction information was collected using an English or Spanish web-based APSQ-2 issued by text or email 3 days after surgery, with up to 3 weekly completion reminders sent to nonrespondents.
We collected from anesthesia records additional data on principal surgical procedure type, age, sex, American Society of Anesthesiologists (ASA) Physical Status classification (score I [indicating no systemic disease] to IV or V [indicating severe systemic disease that is a constant threat to life or moribund patient]), insurance type, care team structure, service location, and clinician tenure with the study practice. Since race and ethnicity are not captured in the anesthesia records, we collected this information directly from participants using a web-based questionnaire, which allowed us to conduct preplanned subgroup analyses by race and ethnicity. Race categories were American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White, and Other. Ethnicity categories were Hispanic or Latino and not Hispanic or Latino. For analysis, Native Hawaiian or Other Pacific Islander was grouped with the Other category due to low numbers. Individuals reporting more than 1 race were assigned to a separate category.
Baseline data were compared across groups using standardized differences. We considered standardized differences greater than 0.2 to be an important imbalance.
Clinicians reported safety outcomes in the practice’s uniform electronic adverse event reporting system. The principal investigator (M.D.N.) grouped reported events into 12 categories based on organ system and event type (cardiac, respiratory or airway, neurological, vascular, bleeding or transfusion, pain, nausea or vomiting, hyperthermia or hypothermia, hyperglycemia or hypoglycemia, procedural complications and medication errors, death, and other [eg, case cancellation, equipment malfunction, need to return to operating room, and need for intensive care unit admission]). Due to unavailability of EMR data for most patients in the trial, we were unable to analyze EMR data on safety outcomes.
We anticipated a total enrollment of approximately 225 000 eligible patients. We estimated a baseline benzodiazepine use rate of 45% based on prior research^3^ and available practice data. We projected that this sample would provide over 90% power for a 2–percentage point change in the primary outcome for each intervention alone, and a 4–percentage point change for the combined intervention, at a Bonferroni-adjusted significance of P = .02 and an intracenter correlation coefficient of 0.3.^16^
Analyses followed the intention-to-treat principle, wherein we analyzed data on all participants according to the treatment assigned via cluster randomization. We used mixed-effects logistic regression to estimate the treatment effect of clinician peer comparison, patient informational mail, or the combined intervention vs usual care (control) on the odds of receiving a benzodiazepine during anesthesia care. This model contained binary indicator variables for the treatment status of each unit within a given period for each of the 2 interventions and an interaction term capturing the joint effect of the interventions. The model included 2-level random effects, with 1 random effect for sequence assignment and the other for division nested within each sequence. To increase the precision of the treatment estimate, the model was adjusted for ASA Physical Status classification (III or higher [indicating severe disease] vs I or II [indicating no disease to mild disease]), surgical procedure type, age, sex, and study period by including them as fixed effects^17,18,19^ (eMethods in Supplement 2). Race and ethnicity were not included in the logistic model due to a high rate of missing data. Models addressed clustering via random effects.
We assessed for heterogeneity of treatment effects on the primary outcome by adding interaction terms to the main study model for (1) age (≥85 years vs ≤84 years); (2) sex, (3) race (Black, White, or other); (4) ethnicity (Hispanic vs non-Hispanic); (5) insurance type; (6) surgical procedure type; (7) whether the division fell above or below the practicewide 50th percentile of benzodiazepine administration; (8) clinician tenure in the practice (above or below the practicewide 50th percentile); (9) care model (solo anesthesiologist vs anesthesiologist-CRNA care team); and (10) service location (inpatient hospital or emergency department vs ambulatory surgery center, physician office, or hospital-based outpatient facility).
We used descriptive statistics to characterize patient satisfaction scores and adverse events across study groups. We compared rates of missing data across divisions and treatment arms. Study models used complete data; we planned additional sensitivity analyses if primary outcome data were missing in 10% or more of cases. Analyses used SAS 9.4 (SAS Institute Inc).
We enrolled 509 269 patients (255 871 females [50.2%], 253 398 males [49.8%]; mean [SD] age, 74 [7] years). Of these patients, 31.4% (159 674) received usual care, 16.0% (81 363) received the clinician peer comparison, 19.3% (98 520) received the patient informational mail, and 33.3% (169 712) received both interventions (Figure 1). At divisions where engagement with the patient informational mail was tracked, 42.0% of patients accessed the intervention.
Patient and surgery characteristics were similar across treatment arms (Table 1). Mean (SD) case duration was between 95 (82) minutes and 100 (86) minutes in each group. Distribution of ASA Physical Status classification, surgical procedure type, insurance type, clinician tenure in the study practice, care model, and service location were also similar across groups. Information on race and ethnicity was available for approximately 5% of all enrolled patients (eTable 2 in Supplement 2). Standardized differences were less than 0.2 for all covariates (eTable 3 in Supplement 2).
Primary outcome data were available for all enrolled patients, and 0 patients were lost to follow-up. Among patients assigned to usual care, 24.5% received a benzodiazepine during anesthesia care compared with 19.7% assigned to clinician peer comparison, 20.0% assigned to patient informational mail, and 19.7% assigned to both interventions (P < .001). Benzodiazepine administration declined over the course of the study in all arms (Figure 2).
Figure 2. Benzodiazepine Administration Over the Study Period by Month and Treatment Arm The percentage of patients receiving benzodiazepine decreased over time in each treatment arm.
The logistic mixed-effects model included 509 067 patients, 99.9% of all enrolled. Of these patients, 202 (0.04%) had missing data on ASA Physical Status classification and were not included in the model; data were complete for all other patients. After adjustment for study period and other covariates, none of the interventions compared with usual care was associated with lower odds of benzodiazepine administration (odds ratio [OR], 1.02 [95% CI, 0.98-1.07]; P = .35 for clinician peer comparison; OR, 1.01 [95% CI, 0.96-1.05]; P = .81 for patient informational mail; and OR, 1.11 [95% CI, 1.05-1.16]; P < .001 for the combined intervention) (Table 2). We obtained similar results with adjustment for study period only, without other covariates (eTable 4 in Supplement 2).
Subgroup analyses suggested a small decrease in benzodiazepine administration with the patient informational mail vs usual care among White patients (OR, 0.74; 95% CI, 0.56-0.92; P = .007), non-Hispanic or Latino patients (OR, 0.76; 95% CI, 0.62-0.95; P = .01), and patients who underwent otolaryngological or ophthalmological procedures (OR, 0.84; 95% CI, 0.72-0.97; P = .02). We did not observe decreases in benzodiazepine administration with the study interventions across other subgroups (eFigure 6 in Supplement 2).
A total of 39 326 patients (7.7%) answered at least 1 item on the postoperative satisfaction survey (APSQ-2). Among those with available information, patient satisfaction with anesthesia care did not differ across intervention groups and was favorable throughout the study period. A score of 4 or more was reported by at least 90% of patients across all items (Figure 3; eTable 5 in Supplement 2). Adverse events did not vary across treatment arms (eTable 6 in Supplement 2).
Figure 3. Patient Satisfaction With 8 Domains of Anesthesia Care Individual questionnaire items, coding, and response data are provided in eTable 5 in Supplement 2. Denominators for each item ranged from 29 389 (discussed risks and benefits) to 38 705 (favorable anesthesia experience) due to nonresponse. Anesthesia Patient Satisfaction Questionnaire, version 2 was used for assessment (score 1-5, with higher scores indicating greater satisfaction).
Among the 509 269 older adults included in the DROP-Benzo trial, clinician peer comparison, patient informational mail, or a combination of both did not reduce benzodiazepine administration compared with usual care. Patient satisfaction with anesthesia care was similar across groups and high overall.
Clinician peer comparison has previously been shown to result in guideline adherence in primary care^14^ and acute care settings.^20,21^ In the context of anesthesia, clinician peer comparison may improve adherence to recommended ventilator management strategies during general anesthesia care.^22^ Results of the present trial contrast with these findings and other evidence of the effectiveness of clinician peer comparison.^21^ This discrepancy suggests that clinician peer comparison may be less effective in discouraging benzodiazepine administration during anesthesia care than other aspects of care. A previous study found that introduction of an EMR-based best practice alert was associated with a decrease in benzodiazepine administration^10^; thus, the present findings may argue that clinician-facing interventions targeting benzodiazepine administration during anesthesia care may vary in their effectiveness depending on their level of integration into clinical workflows.^23^
In nonsurgical settings, patient empowerment interventions may be effective for promoting deprescribing of potentially inappropriate medications in older adults.^24,25^ We designed the patient informational mail with input from community members and clinical leaders; however, the simple mail received by patients may have been insufficient to alter benzodiazepine administration patterns. It is also possible that benzodiazepine administration during anesthesia care may be less amenable to patient empowerment interventions than other aspects of care.
We observed a reduction in benzodiazepine administration over the study period across all groups. This result may have been due to the practicewide email notification issued at the start of the trial or may have been an unintended consequence of the required self-reporting by clinicians of benzodiazepine administration within some divisions. Although we introduced this requirement 3 months prior to the start of the trial, it is possible that allowing a longer interval may have permitted greater washout of any effect on care patterns of required self-reporting of benzodiazepine administration. While unadjusted benzodiazepine administration rates were lower in each of the 3 intervention groups than in usual care, these differences did not persist after adjusting for the overall reduction in administration over time. Although benzodiazepine administration decreased over time, patient satisfaction with anesthesia care remained high throughout the study period, confirming past trial evidence that avoidance of benzodiazepines does not diminish patient satisfaction.^6,7^
Considered as a whole, trial findings suggest that clinician peer comparisons do not add a benefit to basic efforts to discourage routine benzodiazepine administration during anesthesia care, such as routine measurement of use and prioritization by practice leaders. Similarly, simple patient informational mail may have limited impact on benzodiazepine administration; the potential for other patient-targeted interventions to improve anesthesia care should be explored in future work. Additionally, the lack of a change in patient satisfaction confirms evidence from prior trials^6,7^ that reducing benzodiazepine administration during anesthesia care does not adversely affect satisfaction.
Advanced notification of clinicians regarding the intervention and practicewide data collection for the primary outcome may have introduced change in the practice independent of the study interventions. However, these design features were required by the standard operating procedures of the practice and to permit data collection across multiple EMRs and electronic anesthesia record systems used by participating clinicians. Additionally, the analytic approach we used allowed us to avoid spurious findings by accounting for time in the study models. It is possible that we may have obtained a different result had we allowed a longer interval between the introduction of clinician attestations and the start of data collection. Given these findings, we anticipate that even under such conditions, any impact of the interventions on practice would be small. At care locations wherein we measured engagement with the patient-facing intervention, participants opened the informational mail at a rate comparable to that reported in other studies.^26,27^ However, it is possible that increasing patient engagement with this intervention could produce different results.
In this DROP-Benzo randomized clinical trial of 3 corporate quality improvement interventions (clinician peer comparison, patient informational mail, and a combination of both), the interventions did not reduce benzodiazepine administration to older patients compared with usual care. Patient satisfaction with anesthesia care was high overall across treatment groups. There is a need to explore other patient-targeted interventions to improve anesthesia care.