Citing a lack of interest among applicants and program directors, the Association of American Medical Colleges (AAMC) announced that a video interview program it had been piloting for emergency medicine residency programs would not continue past the 2020 application cycle.
The standardized video interview (SVI) was touted as a supplementary tool in the selection of applicants to interview. Since the 2018 Electronic Residency Application Service (ERAS) cycle, the program aimed to measure knowledge of professional behaviors and interpersonal skills and communication, two of the core-competency requirements for success in residency laid out by the Accreditation Council for Graduate Medical Education (ACGME).
After the pilot, the AAMC had hoped to widen the implementation of the project.
“Based on the data collected to date, the AAMC concluded that the SVI is a reliable, valid assessment of behavioral competencies that does not disadvantage individuals or groups,” according to an evaluation summary on the AAMC website. “The decision not to offer the SVI for the ERAS 2021 application cycle is based on lack of interest among the emergency medicine community in continuing to use and research the SVI, as well as an assessment of operational factors necessary for successful expansion of the program.”
Opposition from applicants, programs
A letter signed by stakeholders in emergency medicine residency programs called on the AAMC to discontinue the program.
“We understand that the number of applications per applicant to emergency medicine programs has doubled over the past decade, and that this has increased pressure on residency programs to screen which students to invite for in-person interviews,” says the letter, signed by resident and faculty leaders of organizations such as the American Academy of Emergency Medicine, American College of Emergency Physicians, Council of Residency Directors in Emergency Medicine and Emergency Medicine Residents’ Association.
“However, after three years of piloting the SVI, reviewing the data, and hearing from the members of our community, we respectfully oppose further study or use of the SVI.”
That letter lays out three key reasons for the program’s dissolution: lack of evidence to support the SVI as an assessment tool, uncertainty around the cost of the program, and student perceptions.
To support the third point, the letter cites a June 18, 2019, Academic Medicine report, “Applicant Reactions to the AAMC Standardized Video Interview During the 2018 Application Cycle.” The survey determined that “less than one-quarter of applicants agreed that the SVI gave them an opportunity to describe their interpersonal and communication skills or knowledge of professional behavior, and only half agreed that they were able to answer SVI questions based upon past experiences.”
At the 2017 AMA Interim Meeting, the House of Delegates directed the AMA to, among other things, advocate delaying expansion of the SVI to other specialties “until data demonstrates the Association of American Medical Colleges’ stated goal of predicting resident performance.”
Talking to the camera
The video interview consists of six questions and applicants have three minutes to answer each one. Applicants are asked to draw on personal experiences in answering the questions, which are designed to probe their professionalism and interpersonal skills. For example: How have you handled dealing with some of your more challenging patients? What specific actions did you take? What was the outcome?
These answers are then scored by AAMC-trained raters on a scale of one (rudimentary) to five (exemplary).
“The SVI was intended to address the fact that program directors report a lack of adequate tools to assess an applicant’s interpersonal and communications skills and professionalism,” the letter from emergency medicine stakeholders says. But after three years, the emergency medicine letter says, “the SVI has not been demonstrated to be the tool to fill this gap.” The letter argues the “evidence shows that evaluation of professional and interpersonal communication skills by emergency medicine faculty during in-person interviews significantly differs from SVI scores.”
