Authors: Frederick Mun, Daniel Badin, Morgan Snow, Andrew B. Harris, Dawn M. LaPorte, Amiethab A. Aiyer
Categories: AOA Critical Issues in Education, 0020
Source: JBJS Open Access
In May 2022, the Association of American Medical Colleges (AAMC) published guidelines regarding interviews for the 2022-2023 residency application cycle. These guidelines recommended virtual interviews and discouraged “hybrid” interviewing. We conducted a survey of orthopaedic program directors (PDs) to understand their perspectives on these new guidelines and their plans for the upcoming cycle.
A 19-question multicenter, cross-sectional research survey was emailed to 98 PDs (38.8% response rate) through Qualtrics XM. Contact information was obtained from a public national database.
Most orthopaedic residency programs (60.5%) were planning on conducting in-person interviews before any AAMC and hospital guidelines, and most (65.8%) will likely be conducting virtual interviews post-guidelines. PDs voiced mixed opinions about virtual interviews (39.4% in favor vs. 47.4% against). PDs were also split on whether forgoing the AAMC guidance would be irresponsible for residency programs (47.4% believe it would be irresponsible vs. 44.8% believe it would not); however, a plurality are in favor of the AAMC's guidance (42.1%). Furthermore, PDs agreed that virtual interviews have disadvantages including favoring top-tier applicants, students from home institutions, and in-person rotators, making ranking applicants and learning about a program's culture more difficult. Most PDs (84.2%) felt that hybrid interviews would disadvantage applicants who would choose the virtual option.
AAMC guidance seems to be influencing how most orthopaedic surgery programs will conduct residency interviews for the 2022-2023 cycle. Most PDs agreed with the AAMC guidelines but voiced concerns regarding several disadvantages for all 3 proposed interview options (virtual, in-person, and hybrid). Our results indicate that the recent AAMC guidelines may have contributed to a shift in opinions among PDs but are not sufficient to create a consensus on the best practices for residency interviews. Our findings should encourage solutions focused on the deeper systemic issues within the orthopaedic application process in the post-coronavirus 2019 pandemic era.
On May 16, 2022, the Association of American Medical Colleges (AAMC) published guidance regarding interviews for the 2022-2023 residency application cycle. Among those guidelines, the AAMC strongly recommended that programs conduct virtual interviews and discouraged “hybrid” interviewing within the same program (e.g., offering both virtual and in-person options). Additional guidance suggested that programs share their interviewing plans clearly and early, review antibias and virtual interview best practices, and explore other potential interview methods. The AAMC believes that sharing these guidelines will promote transparency and predictability for all stakeholders and help specialty and training programs with their interviewing and recruiting decisions^1^.
It is unclear how these recommendations will be adopted by residency programs across the country, especially in light of the ever-increasing residency applications. In particular, orthopaedic surgery residency programs have among the highest ratio of applications to positions available of any specialty. In the 2021-2022 cycle, 1,737 applicants competed for 875 positions. Of those who matched into a position, 705 of 875 were first-time MD applicants, 111 of 875 were first-time DO applicants, and 19 of 875 were international medical graduates^2^. Tawfik et al. demonstrated that although applicants applied to more programs during the coronavirus 2019 (COVID-19) pandemic, they received fewer interview invitations than in previous years after virtual rotations, and applicants felt that virtual experiences did not adequately replicate the social factors that applicants found most important when ranking a program^3^. Thus, we sought to understand the perspectives of orthopaedic surgery program directors (PDs) on the AAMC guidelines on virtual interviewing for the 2022-2023 residency application cycle. We aimed for these data to inform applicants in planning their approach to the application process and to better understand how PDs plan to promote their residency programs during these unprecedented times.
This study was deemed exempt from review by our hospital's Institutional Review Board. We gathered publicly available contact information for PDs at all active Accreditation Council for Graduate Medical Education (ACGME) orthopaedic surgery programs through Doximity, Inc. (www.doximity.com) and the American Medical Association's Fellowship and Residency Electronic Interactive Database (FREIDA Online). Through this search, 161 orthopaedic surgery programs were identified. After excluding all emails that were not the personal email addresses of the PDs, we had 98 PD email addresses. A survey invitation email, including formal informed consent, was sent to each PD. Submission of the survey indicated the respondents' consent to participate, and all survey responses were collected anonymously.
This multicenter, cross-sectional research survey was developed, housed, and distributed through Qualtrics XM. Our survey questions used free response, single-answer multiple-choice, multiple-answer multiple-choice, and a Likert scale ranging from 0 (strongly disagree or least important) to 4 (strongly agree or most important) formats. Before distribution, the questions were pretested and tested with subsets of respondents resulting in a 19-item survey. The survey was distributed on May 23, 2022. Two follow-up emails were sent after the initial distribution. No funding was used. Our methods were based on a previous study that was conducted at our institution^4^.
Descriptive statistics were used to describe survey responses. All statistical analyses were performed using STATA version 15.0 (StataCorp, College Station, TX). To provide a more robust analysis, we combined those who selected “strongly agree” and “somewhat agree” into one cohort and “strongly disagree” and “somewhat disagree” into another cohort. The same was done for “significantly increased” and “somewhat increased” as well as “significantly decreased” and “somewhat decreased.”
We distributed our survey to PDs and received 38 of 98 responses, for a 38.8% response rate. The survey is available for review in Supplement 1 (http://links.lww.com/JBJSOA/A442). Most respondents were from university-based programs (27 of 33, 71.1%). Most were also planning to offer between 3 and 6 PGY-1 positions (34 of 38, 89.6%) in 2023. Program demographics are shown in Table I.
Most orthopaedic residency programs (25 of 38, 65.8%) will likely be conducting virtual interviews. Before any AAMC and sponsoring institution guidelines, most programs (23 of 38, 60.5%) were planning to conduct in-person interviews. Forty-seven percent of sponsoring institutions (18 of 38) have no recommendation or are recommending that residency programs conduct virtual interviews (13 of 38, 34.2%). These data are shown in Table II.
PDs share mixed opinions about virtual interviews (15 of 38, 39.4% in favor vs. 18 of 38, 47.4% against), and 42.1% (16 of 38) are in favor of the AAMC guidance on interviews. PDs also share mixed opinions regarding whether forgoing AAMC virtual interview guidance would be irresponsible for orthopaedic residency programs (18 of 38, 47.4% believe it would be irresponsible vs. 17 of 33, 44.8% believe it would not be irresponsible).
Most PDs feel that virtual interviews will make it difficult for applicants to learn about a residency program's culture (32 of 38, 84.2%) and for residency programs to rank applicants (25 of 38, 65.8%). Most PDs also believe that virtual interviews disproportionately advantage top-tier applicants (30 of 38, 78.9%), increase diversity of applicants interviewed (18 of 38, 47.4%), and will result in programs favoring students at their home institution and in-person rotators (31 of 38, 81.6%). Most PDs (32 of 38, 84.2%) feel that hybrid interviews disadvantage applicants who choose the virtual option. These data are shown in Table III.
In the 2022-2023 application cycle, most residency programs will be hosting additional, optional events, including virtual information sessions (34 of 38, 89.5%), virtual social events (16 of 38, 42.1%), and in-person social events (13 of 38, 34.2%). Most programs are planning to hold these additional events, even after learning of AAMC guidelines on virtual interviews (26 of 38, 68.4%). These data are shown in Table IV.
With the AAMC recommending virtual interviews for orthopaedic surgery residency programs during the 2022-2023 application cycle, it would be ideal if residency programs are largely in agreement with these recommendations and that consensus exists regarding virtual interviews and interviewing best practices. We conducted a survey of orthopaedic surgery PDs and found that most were planning on conducting in-person interviews pre-AAMC guidelines, with most now planning on conducting virtual interviews. Opinions on the AAMC guidelines were mixed, with opinions on both sides; however, most PDs agreed on several potential disadvantages for virtual interviews including favoring top-tier applicants, home students, and in-person rotators and making ranking applicants and learning about a program's culture more difficult. Our results indicate that the recent AAMC guidelines may not be universally favored by orthopaedic PDs, which highlights the need for unique solutions to the current challenges of the orthopaedic residency selection process.
Our results are similar to previous studies on PD perspectives regarding the efficacy of virtual residency interviews and virtual residency events. In a survey of 29 PDs and 99 applicants after the 2020-2021 application cycle, Brueggeman et al. determined that PDs largely felt that virtual interviews limited their ability to assess applicants' fit for their program, commitment to orthopaedics, and interpersonal skills, but also acknowledged that virtual interviews were more cost-efficient. At the time, most applicants (81%) and PDs (79%) expressed preference for in-person interviews despite some advantages to a virtual format^5^. This study may demonstrate a shift in orthopaedic PD opinions regarding virtual interviews. Although opinions continue to be mixed, compared with the findings of Brueggerman et al., a slightly greater number of PDs (39%) seem supportive of a virtual format, after having completed virtual interviews during the past 2 application cycles. Advantages of a virtual format, including time and cost efficiency, may certainly be appealing to some PDs; however, PDs in this study felt that virtual interviews had many disadvantages as well, including difficulty in learning about a program's culture, difficulty ranking applicants, and disproportionate favoring of top-tier applicants. Interestingly, the AAMC guidelines do not mention health safety as a consideration in its decision and refer to “solutions for a post-COVID-19 era.” Because many of the aforementioned challenges are likely to persist in the future, alternatives that can overcome these limitations are needed.
Eighty-four percent of PDs felt that hybrid interviewing would disadvantage applicants who choose the virtual option. Hybrid interviewing offers the flexibility to balance convenience and financial considerations, with the desire for in-person connection and evaluation; however, its limitations may be significant enough to preclude its use. After all, disadvantaging applicants who choose the virtual option may mean disadvantaging applicants with financial constraints, as noted in the AAMC guidelines^6^. Moreover, with highly competitive specialties such as orthopaedics, even small disadvantages may be detrimental to an applicant's prospects of matching. As such, although hybrid interviewing may seem like the “middle ground,” it may be a suboptimal solution to the current situation. Interestingly, the American Association of Colleges of Osteopathic Medicine (AACOM) endorsed participation in hybrid interviews and argued that virtual interviews alone may actually disadvantage osteopathic medical students and minority medical students^7^. The differences between the AAMC and AACOM highlight the controversial nature of this topic and the difficulty in finding an optimal solution.
Although this study focuses on the benefits, limitations, and controversy surrounding virtual interviews, we believe that our findings highlight deeper systemic issues within the orthopaedic residency application process. Problems such as a rapidly rising applicant pool, overapplying to programs, and interview hoarding have worsened, especially with the COVID-19 pandemic^2,3,5^. These systemic issues, which are especially amplified in the orthopaedic application process because of its competitive nature, may not be adequately overcome by transitioning to a virtual interview format alone. It is difficult to conjecture on possible solutions, but approximately 50,000 applicants per cycle apply to residency through the National Residency Matching Program, and steps must be taken to address these issues^8^. For example, in-person interviews may become more feasible if the problems of overapplication can be addressed by strict interviews or application caps, whereby each applicant may only apply to a fixed number of programs. This will allow each applicant to “safely” participate in fewer interviews, thereby incurring a more acceptable financial and logistic sacrifice, without lowering the chance of matching. However, these restrictions may be slow to implement because stakeholders such as the AAMC benefit financially from the current application structure. Perhaps, creating a separate match process similar to that in ophthalmology, urology, and plastic surgery may help orthopaedic residency programs attain more autonomy and regulation. The American Orthopaedic Association recently endorsed a preference signaling program that seeks to address some of these issues; however, the results of this are yet to be seen in the 2022-2023 cycle^9^. To our knowledge, these ideas have not been formally studied but may be needed with the shifts in the current era. Future efforts should be placed on evaluating and implementing such possibilities.
Our study must be interpreted within the context of its limitations. Our findings are based on an aggregate sample of 38 PDs (response 38.8%) and may not accurately reflect the opinions of all PDs. Most of our respondents (71.1%) were also from university-based programs, which are overrepresented in our study. Furthermore, survey responses may be limited by selection bias and response bias; PDs who feel strongly about virtual interviews were more likely to respond to our survey. As such, their opinions may subsequently be over-represented in our results. PDs have valuable insights around the residency selection process; however, their opinions may not accurately reflect the true nature of the selection process.
We found that AAMC recommendations for virtual interviews have influenced planning for orthopaedic residency interviews in 2022-2023 because most PDs were planning on holding in-person interviews pre-AAMC guidelines, with most of them now planning on holding virtual interviews. Most PDs voiced concerns regarding several disadvantages for all 3 proposed interview options (virtual, hybrid, and in-person). Virtual interviews were perceived to favor top-tier applicants, home students, and in-person rotators and to make ranking applicants and learning about a program's culture more difficult. Meanwhile, hybrid interviews were perceived to disadvantage applicants who would choose the virtual option. Our results indicate that orthopaedic PDs share mixed opinions on the recent AAMC guidelines and view all 3 interview methods as suboptimal. Our findings should invite solutions focused on the deeper systemic issues within the orthopaedic residency application process in the post-COVID-19 era.
Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A442). This content was not copyedited or verified by JBJS.
Daniel Badin, Email: dbadin1@jh.edu.
Morgan Snow, Email: msnow6@jhmi.edu.
Andrew B. Harris, Email: andrewbharris@jhmi.edu.
Dawn M. LaPorte, Email: dlaport1@jhmi.edu.
Amiethab A. Aiyer, Email: aaiyer2@jhmi.edu.