Advertisement

Implementation of a Specialty Society‒Sponsored Wilderness Medicine Fellowship Match

Published:December 15, 2022DOI:https://doi.org/10.1016/j.wem.2022.10.007

      Introduction

      Previously, wilderness medicine (WM) fellowships offered spots to applicants using an offer date. Due in part to increases in the number of WM fellowships and applicants, in 2021, the WM program directors (PDs) agreed to conduct the first WM fellowship match through the Wilderness Medical Society graduate medical education committee. This article outlines the process used and demonstrates its feasibility.

      Methods

      To create an independent matching process, a simulation was performed using imaginary programs and participants. Using the same algorithm utilized by the National Resident Matching Program, this process was completed manually and by computer to ensure accuracy.
      The PDs shared an email with the applicants they interviewed and submitted their names. Applicants registered for the match and generated a match list. The PDs then submitted a rank list of applicants they interviewed through a similar Google form.
      These lists were used to run the matching algorithm both manually and by computer. Any programs that did not “fill” or applicants who did not “match” were contacted to participate in a secondary match.
      Following the match, a survey was sent to PDs and participants for process improvement.

      Results

      The match filled 11 of 14 participating programs and 15 of 19 applicants. The results obtained via a computer algorithm were consistent with multiple human validations. The survey results were mostly positive, with 2 neutral responses and no negative responses.

      Conclusions

      The inaugural WM fellowship match was successful in matching the majority of programs and participants and was well-received by both directors and applicants.

      Keywords

      Introduction

      Wilderness Medicine (WM) fellowships previously filled positions using an offer-date set by fellowship directors each fall.
      • Spano S.J.
      Is it time for wilderness medicine fellowships to join the national resident matching program specialties matching service?.
      Applicant(s) were called in the order of preference by each director until all available positions were filled. Once called, applicants would have 30 min to accept or decline the offer. Limitations included the potential for verbal, nonbinding offers to influence candidates’ and directors’ actions on the offer day, as well as pressure on candidates to accept initial offers owing to the lack of knowledge regarding potential forthcoming offers. These concerns cast uncertainty on whether participants were placed in a disadvantageous position by the offer-date system.
      All WM fellowship directors participated in a survey to consider joining the National Resident Matching Program (NRMP) specialties matching service (SMS) match in the summer of 2019.
      • Spano S.J.
      Is it time for wilderness medicine fellowships to join the national resident matching program specialties matching service?.
      The majority supported WM joining the SMS; however, it did not reach the 75% threshold needed for SMS to be successful.
      National Resident Matching Program
      Establishing a fellowship match.
      The NRMP currently requires that 75% of the programs participate in the SMS in order to establish a new specialty match.
      National Resident Matching Program
      Establishing a fellowship match.
      Additionally, nearly half (46%) were unwilling to pay fees to participate in NRMP’s match.
      • Spano S.J.
      Is it time for wilderness medicine fellowships to join the national resident matching program specialties matching service?.
      There has been considerable growth in WM fellowship opportunities in the last few years, reflecting larger trends in fellowships outside of WM. There are currently 22 active programs listed on the Wilderness Medical Society (WMS) listing of graduate medical education (GME) fellowships, reflecting 70% growth in new programs since the 2019 NRMP SMS match survey. This is an augmented trend similar to that reported in a 2018 press release from NRMP president and chief executive officer Mona M. Signer, reporting national subspecialty training opportunities grew with a 16% increase in the number of applicants seeking fellowship positions through SMS since 2014.
      National Resident Matching Program
      Press Release: NRMP report shows 2018 appointment year fellowship matches at record high.
      Internal medicine fellowships during the COVID-19 pandemic surged in popularity, with an increase in applicants from 2020 to 2021 higher than all prior years studied for 7 of the 11 fellowships, with only gastroenterology showing a 0.3% decrease in applicants from 2020 to 2021.
      • Huppert L.A.
      • Santhosh L.
      • Babik J.M.
      Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years.
      The 2021 medical specialties matching program, comprising primarily internal medicine subspecialties, was the largest on record, with 9% more applicants and 12% more programs that submitted rank order lists over 2020.
      National Resident Matching Program
      Press Release: NRMP medical specialties matching program matched more than 7,000 applicants into advanced training positions.
      Formal fellowship matching services are not exclusively managed by the NRMP; other businesses, the military, and professional societies host matching services. The San Francisco match currently provides fellowship matching services to 22 subspecialties.
      SF Match.
      The military does not use a computer-generated match list; the selection committee arranges negotiated pairings between programs and applicants, with the ability to place an applicant in a program they did not rank.
      American Medical Association
      Career planning resource; learning about early match programs.
      The American Urological Association, in conjunction with the Society of Academic Urologists, has overseen the urology residency match program for >35 y, which includes fellowship matches.
      American Urological Association
      Urology and specialty matches.
      It is the authors’ opinion that the benefits to a specialty society taking on a centralized match for the field are a presumed increase in engaged society members, both pre- and postmatch, and retaining members who will be future leaders in the field. By serving as a first point of contact for fellows entering the field, there is an opportunity for the society to leave a positive impression on these individuals. Although there are numerous specialty societies with WM sections or workgroups, there is currently only one specialty society dedicated solely to WM, the WMS. As a specialty society, the WMS is unique in that it includes multiple types of medical training within the healthcare continuum. This includes programs for nurse practitioners (NPs), physician assistants (PAs), and physicians in multiple clinical specialties, currently including emergency medicine and family medicine. This design allows continued promotion of the development of WM fellowship in other specialties.
      One recognized barrier to a centralized match was cost. The NRMP assesses fees to both applicants and programs, and these fees and limited program administrative funding were often cited by program directors (PDs) as reasons for opposing a centralized match. The activities of specialty societies are directed and enacted by voluntary service of its engaged members with the support of dedicated staff, and these groups could therefore conduct a match process without assessing additional fees. It was deemed possible by consensus of all fellowship directors to participate in a specialty society‒hosted match for the academic year of 2022 and 2023. This is the report on perceived success and limitations for applicants, PDs, and support staff of the sponsoring society on the impact and sustainability of a WMS-sponsored GME fellowship match after its inaugural year.
      The timeline for initiation of the WM match from decision to implementation was several years, starting with the informal yet consistently recurring discussion among WM fellowship directors to participate in a standardized match process. The first step was the survey regarding consideration to join the NRMP SMS match (that did not reach the threshold) in the summer of 2019.
      • Spano S.J.
      Is it time for wilderness medicine fellowships to join the national resident matching program specialties matching service?.
      In turn, the original informal match process continued, and in fall 2020, all WM fellowship directors unanimously agreed to move forward with a centralized match process after proper and thorough blinded testing of the algorithm process was successfully carried out.
      The objectives of this article are to outline and report the results of the process used by the WMS to implement a fellowship match that would ensure applicants and programs achieve the best possible match while avoiding placing additional financial burdens on programs or applicants.

      Methods

      The WMS GME committee agreed in June 2021 to conduct a simulated trial/test match. The objective was to have a blinded, computer-based algorithm complete the matching process, similar to the NRMP but housed entirely within the WMS. This was done using a variation of the basic computer science “stable match” algorithm, which is similar to the NRMP match algorithm. A publicly available Python Git repository using a series of Google Apps Script (GAS) codes was used to automate the process.

      Python Git Repository; publicly available. Available at: https://github.com/J-DM/Roth-Peranson. Accessed July 2021.

      Both the applicant and the PD had a mirrored workflow consisting of the following 3 interdependent components/tasks: 1) registration form; 2) rank list form; and 3) rank list confirmation form.
      The process started once the applicant completed a registration form that was publicly available. Next, the GAS assigned a unique ID (UUID) for each applicant that was utilized as a check in lieu of a login/password, and an email was sent with a unique rank list submission form. This form consisted of a dropdown selection with all programs prefilled as options. The applicant then completed the rank list form, and the GAS automatically sent a copy of the submitted rank list to the applicant’s email. This was used to confirm accuracy, and applicants then “certified” this list. Once certified by the applicant, the rank list confirmation form was complete.
      As just described, the PD workflow mirrored that of the applicant with the same components. First, a registration form was sent to all PDs, and the GAS assigned a UUID for each PD, which was used as a check in lieu of a login/password. Then, an email was sent with the unique rank list submission form. The PD form differed from the applicant form in that the selection input was in free-text format, as opposed to the dropdown with all programs prefilled as options for the applicant. The PD then completed the rank list form, and the GAS sent a copy of the rank list to the PD’s email in order to “certify” the list.
      To verify the accuracy of this computer-based process, the simulated test match raw data were also sent to the 8 participating members of the WMS GME match subcommittee on July 7, 2021. They were instructed to run each simulated test rank list (17 imaginary applicants) by hand using the NRMP match algorithm.
      National Resident Matching Program
      How the NRMP Matching Algorithm Works.
      The results obtained by the subcommittee members were then cross-checked with the results obtained by running the same test rank lists through the computer algorithm. All results were consistent and identical. The unmatched test applicants/programs then participated in a subsequent test secondary match, formerly known as “scramble,” to determine whether that process would be effective in placing applicants and filling positions. This process consisted of allowing unmatched programs and unmatched applicants to communicate with one another and conduct mock interviews. These unmatched programs and applicants then submitted new rank lists based on these interviews, and the same match algorithm was used once again. The simulated secondary match process was successful in filling all positions. All simulated test match results were cross-checked and verified by the committee at large, proving that the computer algorithm was secure and reliable.
      After completion of the simulated trial/test match, all PDs and WMS GME committee members agreed to participate in the proposed inaugural WM match via email or telephone verification. The deadline of October 25, 2021 was set for the submission of all rank lists to the WMS staff member by all participating applicants and programs. Individual emails were sent to both the programs and applicants to encourage timely completion of the process. On October 28, 2021, at 0900 PST/1200 EST, initial emails were sent simultaneously to each applicant and each PD to inform them of either a successful match or eligibility for the secondary match.
      All programs had access to the unmatched applicants report (list of unmatched applicants), and all unmatched applicants had access to the unfilled programs list. The secondary match was opened up later that day at 1300 PST/1600 EST for unmatched applicants
      • Huppert L.A.
      • Santhosh L.
      • Babik J.M.
      Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years.
      and unfilled programs
      National Resident Matching Program
      Establishing a fellowship match.
      to begin communicating with each other; the proposed submission date of the secondary match rank list was October 30, 2021, with subsequent results confidentially emailed to participating parties the following day, on October 31, 2021. None of the unmatched applicants
      • Huppert L.A.
      • Santhosh L.
      • Babik J.M.
      Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years.
      or unfilled programs
      National Resident Matching Program
      Establishing a fellowship match.
      chose to participate in the secondary match. Raw data from the rank lists were then sent to committee members, and 1 member put it into the format required for Python script while the remainder performed the match algorithm by hand. Manual data cleaning was required for the PD rank lists because they submitted choices as free text since there are theoretically an unlimited number of potential applicants. This cleaning consisted of standardizing the formatting of names by removing MD or DO (n=4), removing a nickname in quotes between the first and last name (n=1), and using a shortened first name instead of a formal first name (n=1). This was required because the script could not recognize that these small variations in the names were not the same person, while a human easily could. The committee members sent hand-calculated match results, which were compared with Python script output just as was done during the simulated match, and the results were again consistent. Identifying results were then publicly released on the WMS website, and emails were generated and sent to all applicants/PDs on match day. A brief survey was sent to PDs and participants to gather feedback for process improvement.

      Results

      Prior to the completion of the match process, 3 programs withdrew after registering for the match and did not submit a rank list of applicants. All 3 programs cited either hiring internal candidates or not offering a position as reasons for withdrawing. A total of 13 programs and 15 applicants completed the match process. After the first round, 11 of 15 applicants had matched, and 2 programs and 4 candidates had the opportunity to complete the secondary match. During the secondary match period, the unfilled programs
      National Resident Matching Program
      Establishing a fellowship match.
      withdrew, obviating the need for the secondary match. Of the participating applicants, 42% (n=8) matched into their first-choice program, and 43% (n=6) programs matched their first-choice applicant.
      From the survey, 15 responses were recorded, 6 of which were applicants and 9 of which were PDs. From this feedback, 13 positive comments were noted and 2 comments that were neutral were noted. Positive comments included “Smooth process,” “Loved it, never look back,” “It was helpful; I had no issues.” Neutral comments included “Rank list email only show number of choices. Forward WMS emails back to the gmail address” and “I almost forgot to certify my list. Can that be a part of the initial submission process? Or a website that can be adjusted in real time and certified instead of a Google form?” No negative comments were noted.

      Discussion

      After several years of growth in both the number of WM fellowships and applicants, the WMS hosted a match process that successfully matched the majority of applicants and programs. While most PDs had supported some form of a centralized matching process in 2019, there were still a considerable number of concerns preventing it from moving forward, such as the fees assessed by the NRMP, penalties for withdrawing, and a timeline that was not feasible with current interview schedules. Many PDs noted in 2021 that they had a significant increase in the number of applicants in 2020 and cited this as motivation for moving away from the previous telephone-based offer system and pursuing a match for the subsequent year.
      The decision to proceed with a match process hosted by a specialty society (WMS) and not the SMS addressed several concerns among program leadership. The field of WM represents many medical specialties, and programs are now offering spots to physicians trained in emergency medicine, family medicine, and internal medicine, as well as PAs and NPs. This match process was able to match applicants into family medicine as well as emergency medicine programs, which would not have been possible using the SMS, which would have limited the programs included in the process to a single clinical specialty. This process is, therefore, also adaptable to the evolving needs of the specialty of WM and is flexible enough to be more inclusive of trainees and programs with different requirements and could include PAs, NPs, and physicians with different clinical specialties in the future. Because an applicant cannot be matched to a program that did not place their name on the rank list, the process does not need to be run separately for different types of programs or applicants.
      One of the concerns raised with a matching process for WM fellowships, specifically with the SMS, was the requirement that 75% of programs participate. As non-Accreditation Council for Graduate Medical Education accredited programs, WM fellowships typically hire fellows as junior faculty and can be subject to more variability in the number of positions available owing to funding and staffing. For a relatively small subspecialty, most programs have a limited number of faculty, and limitations on faculty time and availability have necessitated programs altering the number of positions offered, including not taking a new fellow in years past. The PDs raised concerns that the SMS would make it difficult to remain flexible when offering positions in the match, concerned that it would impact the ability of other programs to participate in the process or prevent their program from participating in the future. During this first match process, 3 programs withdrew prior to the rank list certification without disrupting the process and will be able to participate in the future without penalty, thus allaying those concerns.
      Another reason for not participating in a match previously cited by the PDs was the fee associated with the NRMP SMS. By hosting the match through the WMS, we were able to avoid any additional fees to the programs or applicants. The work of designing, testing, and implementing the process was completed by volunteer members of the GME committee, and administrative support was provided at no additional cost by the WMS and took approximately 4 mo.

      Limitations

      The limitations of this study were that the survey had a small number of responses and did not ask quantitative questions regarding the experience. The feedback covers only 1 y of this process. Not all programs participated, meaning some programs may have made offers outside of the match process. Because WM fellowships are not required to be accredited, there is no enforcement for programs to participate in a match process. There were also fewer applicants this year than the year prior, as the sudden surge was likely influenced by the job market in 2020, similar to the case with both the subspecialties of infectious diseases and interventional radiology, which occurred in the 1980s and 90s.
      • Niederle M.
      • Roth A.E.
      The gastroenterology fellowship match: how it failed and why it could succeed once again.
      It is unclear if this process will still be useful if the number of applicants dramatically decreases.
      Although the computer algorithm performed well during the match process, with its accuracy validated by multiple manual checks, it requires manual data cleaning of the rank lists by transferring the names from the Google forms to the program, introducing the possibility of human error. That was mitigated by cross-checking the computer results manually, but it is unclear whether that validation would be feasible or reliable if the number of programs and applicants grows.
      The secondary match was not utilized, so it was not able to be evaluated. Feedback from programs and applicants most commonly cited the short timeline (less than a week) to interview and submit additional rank lists as the reason for not completing the secondary match. Additionally, with so few programs available nationally, applicants likely applied to all the programs of interest, and the failure to match into an open position did not increase their interest in programs not previously applied to.
      Our analysis of this process is limited by the paucity of published work on the establishment of fellowship match processes. Many of our references were web-based because the information regarding the NRMP and American Urological Association match process are available only through their websites, and they do not produce static publications that describe the match algorithm or processes used.
      If the field of WM continues to grow and there is an increase in the number of applicants and fellowship programs, the workload may exceed the time and energy available from volunteer committee members and WMS staff without imposing fees to support the administrative efforts. It is likely that even if fees are required in the future, owing to the significant volunteer engagement and service from the WMS, costs would still be much lower than using the SMS.

      Conclusions

      This study demonstrated a successful match process through an independent specialty society using a computer algorithm similar to the NRMP at no cost to the applicants or programs. As the field of WM is rapidly evolving, ongoing surveillance will be needed to ensure the process remains relevant and best suited to the needs of both WM fellowship programs and applicants. Further development may be needed for a secondary match as the number of programs and applicants changes. Currently, a secondary match process is not of benefit.

      Acknowledgments

      The authors thank Gabi Glass and the members of the WMS GME committee for their support in establishing the match process.
      Author Contributions: data acquisition (CAD, SL, TH, AG, SS); drafting and critical revision of the manuscript (CAD, SL, TH, AG, SS); approval of final manuscript (CAD, SL, TH, AG, SS).
      Financial/Material Support: None
      Disclosures: SS reports travel expense for WMS Conference Planning Committee Winter – 2022 Conference, travel expense of Board meeting of California American College of Emergency Physicians, Honoraria for Quincy Conservation and Wilderness Medicine Conference 2022, and medical staff leadership contract of Community Medical Centers, Fresno, CA. SL reports financial support from Wilderness Medical Society and Virginia ACEP and serves on the Wilderness Medical Society, Blue Ridge Adventure Medicine Roanoke Outside Advisory Board, and Roanoke Greenway Commission.

      References

        • Spano S.J.
        Is it time for wilderness medicine fellowships to join the national resident matching program specialties matching service?.
        Wilderness Environ Med. 2021; 32: 12-18
        • National Resident Matching Program
        Establishing a fellowship match.
        (Available at:)
        • National Resident Matching Program
        Press Release: NRMP report shows 2018 appointment year fellowship matches at record high.
        (Available at:)
        • Huppert L.A.
        • Santhosh L.
        • Babik J.M.
        Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years.
        JAMA Netw Open. 2021; 4: e218199
        • National Resident Matching Program
        Press Release: NRMP medical specialties matching program matched more than 7,000 applicants into advanced training positions.
        (Available at:)
      1. SF Match.
        (Available at:)
        https://sfmatch.org
        Date accessed: April 5, 2022
        • American Medical Association
        Career planning resource; learning about early match programs.
        (Available at:)
        • American Urological Association
        Urology and specialty matches.
        (Available at:)
      2. Python Git Repository; publicly available. Available at: https://github.com/J-DM/Roth-Peranson. Accessed July 2021.

        • National Resident Matching Program
        How the NRMP Matching Algorithm Works.
        (Available at:)
        • Niederle M.
        • Roth A.E.
        The gastroenterology fellowship match: how it failed and why it could succeed once again.
        Gastroenterology. 2004; 127: 658-666