AMA's Moving Medicine video series amplifies physician voices and highlights developments and achievements throughout medicine.
Featured topic and speakers
To access more than 12,000 AGGME-accredited programs, visit FREIDA™, the AMA Residency & Fellowship Database®.
Daniel Dent, MD, a professor of surgery and medical education at the UT Health San Antonio Long School of Medicine, discusses what USMLE Step 1 and COMLEX Level 1 going pass/fail means for program directors and applicants with American Medical Association Chief Experience Officer Todd Unger.
Speaker
- Daniel Dent, MD, professor of surgery and medical education, UT Health San Antonio Long School of Medicine
Transcript
Unger: Hello, this is the American Medical Association's Moving Medicine video and podcast. Today we're talking about the USMLE Step 1 and COMLEX Level 1 going pass/fail, and what that means for program directors and applicants. I'm joined today by Dr. Daniel Dent, a professor of surgery and medical education at the UT Health San Antonio Long School of Medicine in San Antonio, Texas. I'm Todd Unger, AMA's chief experience officer in Chicago.
Thanks so much for joining us, Dr. Dent. Lots to talk about here. Big interest among medical students—this change.
Why don't we just start with some basics? When did Step 1 officially move to pass/fail?
Dr. Dent: So, it officially moved to pass/fail on January 26 of this year. So, the current MS2—now becoming MS3 class, as they took it—if they wanted to take it particularly early, they could still get a numerical score. But at the timeline where most students across country take it, it would be pass/fail for that class.
Unger: And give us a little bit of background on the factors that drove this change.
Dr. Dent: So, I think there were a number of factors. And I probably should have mentioned in the first one—COMLEX for the osteopathic schools went pass/fail on May 10 of this year—again, likely affecting the same class of students—the applicants for the fall of '23 to match in '24.
Factors that drove this were a combination of things. People worry about teaching to the test versus teaching a more holistic version of medicine. The stress of getting a particular score on a test was a concern. And the fact that you spend, basically, two years in a classroom just cramming for a singular exam, as opposed to, potentially, to people looking at your performance across your coursework over two years as a way to evaluate you was viewed as problematic, frankly, to look at a singular number. And so, I think there are a variety of factors that went into this decision.
I should also point out that it's a complex situation where diversity, equity and inclusion are concerned because there's data that shows that certain racial and ethnic groups don't do as well on standardized tests, relative to their actual knowledge. And so, it creates an opportunity for discrepancies there. But at the same time, it also could have an anti-DEI effect with regard to students that are not mainstream students from allopathic U.S. medical schools. And so, the osteopathic students and the international medical graduates get a chance to distinguish themselves by their test scores and that may be a move in the wrong direction in terms of DEI issues.
Unger: We're going to talk in more detail about that idea of distinguishing oneself, given this change, in a minute. I'm curious, based on what you've heard from those that are going to be affected by this change—what do folks think about this new approach?
Dr. Dent: Well, I thought about this after I was made aware this would be one of the questions. And I've not heard anything good from anybody. But I also think it's a selection bias because people—I only hear what people complain about, for starters. And the other thing is I don't think there's any MS1 or MS2 out there that's saying, “Oh, thank God, my life is so much stressful—less stressful.”
I don't know that, if you didn't go through the previous iteration—I don't think you appreciate that there is maybe significantly less stress in your life now because you're not getting a pressure to get a 250 on Step 1, because first and second year of med school will come with plenty of stress, and it's hard work. And so, I don't know that people are really celebrating the lack of a numerical test score.
But at the same time, I think there probably is—some of the advantages people saw in making this change are coming to fruition. But I don't know that we're hearing about them.
Unger: I'm sure change of any kind is difficult for folks. It's always, I guess, a little sense of uncertainty around it. When you start to play this out in your mind about the future for those that this will impact, how do you—how do you see it? How do you see this change impacting the applicants in the coming year and programs?
Subscribe and succeed in medical school
Get tips and insider advice from the AMA on succeeding in medical school—delivered to your inbox.
Dr. Dent: Well, as I've talked to program directors—and I am a leader in the surgery program directors' organization but I also am on an organization of program director associations and so I speak to program directors from other specialties as well. It's very mixed and I think it's going to be interesting to see how program directors respond to this. And I think what we'll see in that first match, without a Step 1, is basically iteration 1.0. And it'll be a growth exercise over, probably, three to five years of adaptation for how the program directors look at applicants.
And I'm concerned that, in the first iteration, the easiest move is just going to be default to Step 2 score, and—while maybe looking at some other aspects, looking at grades, looking at clinical rotation grades, maybe clinical rotation shelf exam scores, things like that to get a composite of—what is this person's medical knowledge and their capacity to attain medical knowledge because that's really what we're trying to measure with a Step 1 score. But I think they'll start looking at other aspects of that.
I also think other aspects of the application will become more important—the letters of recommendation. And specifically, do they speak to your medical knowledge? Because right now, they mostly speak to, do you work hard and do you get things done? But they don't speak to your medical knowledge all that often, as I read through thousands of letters of recommendation every year. And so, hopefully, we'll start seeing some changes there.
Unger: Well, Dr. Dent, you just gave a really, really great piece of feedback. And when I go into this topic, we hear so much from M1s about, especially in this year—about how you distinguish yourself in the residency application process without that score.
Now, one thing you just said was, make sure you have letters of recommendations that speak to your medical knowledge. And that's an important thing for folks to keep in mind when they ask for those in this realm of distinguishing yourself. What kind of pieces of advice do you give to medical students that they should be thinking about now, as they prepare for that future?
Dr. Dent: So I think you need to be able to demonstrate good acquisition of medical knowledge. And your MS3 rotations, in particular, and your shelf scores on them are something that program directors may look at as a substitute for a measure for medical knowledge instead of your Step 1 score.
And realizing that medical knowledge on your MS3 rotations is really what plays into Step 2 score. But I still think most clinical program directors are going to care more about how you do as a clinician than they do about how you did in some basic science classes. They really care about clinical medicine. And so, if you can show that you can do well on exams about clinical medicine, that, I think, will substitute, in many program directors' eyes, for a Step 1 score because you're going to be taking tests on clinical medicine for the rest of your life. And in their program, you're going to have to pass a board exam on clinical medicine in that specialty. And so, program directors do want to see that.
They also want to see—like I previously mentioned—the letters of recommendation is crucial because, ultimately, what you're looking for is somebody you want on your team at 2:00 in the morning, taking care of a sick person. And having some medical knowledge as part of that but it is, by no means, the whole picture.
Unger: When you're looking at potential applicants, how about extracurriculars, leadership positions, distinguishing experiences? Where does that fall into the mix?
Dr. Dent: So, it definitely can be something that we view as a very positive thing. For example, the person that has 50 volunteer experiences that were all one-hour long—that, to us, is not as valuable as someone who committed to a volunteer organization and did it on their spring break and every summer and ended up in a leadership role in that volunteer organization. And so being selected by your peers and being committed to something through those leadership—and being rewarded with that through a leadership role is something we value highly.
The other thing we value highly in surgery—team sports or team events, orchestra, band, cheerleading, whatever. Things that where you work in a team and you have success as a member of a team is highly valued. And work experiences—and, again, like the volunteer experiences, the person that worked 20 different jobs, to us, makes us wonder why they had to move 20 times, whereas the person who came back to be, eventually, after five summers, the manager at Chili's because they waited tables or they started out cleaning the place, and then they waited tables, and then they became a bartender and then, their last summer, they were a manager—that type of experience is something we value highly because we see a commitment to an organization and an institution and somebody who, through their hard work and how they're viewed by their coworkers, ends up earning themselves a leadership role.
Unger: How about participation in organized medicine or health policy advocacy, things like that?
Dr. Dent: Absolutely. And again, any of that is viewed positively. I want to make that clear. But significant commitment to it is viewed much more positively. And so, if you're the AMA rep for your school or you've done other things within the AMA and have worked with—to do some local volunteer work for the AMA and your community and those sorts of things, whether it's AMA or your state medical association—that is definitely viewed positively as well.
But again, there's the people that show up so they can check a box and put it on their CV. And then there's the people that show up and pay attention and commit. And that's really who we're looking for.
Unger: Well, that is great advice. I want to follow up on something you spoke about earlier around equity issues. I think there are potentially concerns about how this change might impact IMGs or applicants from lesser-known medical schools, who, I guess, could have hoped that an outstanding numerical score would have helped them get noticed for an interview.
Do you think that this change might put folks like that at a disadvantage? And what advice would you have for them?
Dr. Dent: Well, I think it has that potential. And if you come from a well-known medical school with letters of recommendation from well-known people and there's nothing else numerical to distinguish you from somebody from a lesser-known school with letters from lesser-known people, that's a challenge. And that was one of the concerns people had about taking away the numerical score of Step 1.
However, ultimately, I guess you can distinguish yourself via Step 2. Many schools have class rank. Many don't. And schools that are not just pass/fail but have honors, high pass, pass, or A, B, C type grading—you can distinguish yourself with those grades.
There are ways to distinguish yourself other than Step 1. And I think Step 2 is probably the default at this point but there are some other ways that you can do it. I worry if Step 2 goes away, that that would be a real challenge for people that are at a disadvantage based on what school they go to, with their ability to overcome that.
Unger: Well, I understand that in addition to the main MyERAS application, this year's applicants for both MD and DO programs, including IMGs, are also eligible to complete an optional supplemental application in participating specialties. Can you tell us more about this type of supplemental application and what it would tell a program director, and how your institution used it in general surgery?
Dr. Dent: Yes, so, if I have the number right, I believe 16 specialties participated in this last year with the supplemental application. So, it wasn't everybody but it was half or more of the specialties. And what it does is it gives the program a chance to look for things that would make you a good fit in their program because, while there are more residency positions than there are U.S. grads, by 8,000 spots, there are not more residency positions than there are applicants when you count the international medical graduates added into the applicant pool.
And so, as a result, we're—I first heard about this—I don't know—five years ago, program directors is talking about, what are we going to do about the problem of too many applicants? That was never viewed as a problem before. But the reason it's a problem is not because there's too many good people. It's because people are applying to so many programs that we—for example, in San Antonio—get applications from people in the Northeast that we know are extremely unlikely to leave the Northeast if they can have the option of staying in the Northeast for their training or residency training.
So what we look for in the supplemental app is something that suggests that maybe they would leave Boston or DC or Philadelphia to come to San Antonio for their training. Something that connects them with our Hispanic culture, some volunteer work that they've done related to that sort of thing, the fact that they speak Spanish and other things they can put on their supplemental application that would make us then say, while we mostly realize that the people we're going to match are largely going to be from Texas and surrounding states, we might interview the person from Seattle or Boston or Chicago if we see something in their supplemental app that suggests they actually might rank us and be happy and want to come to San Antonio.
Unger: Very interesting. This is obviously a big deal of great interest to medical students out there. Any final thoughts or advice that you'd like to give them?
Dr. Dent: That's a great question. I think that, in general, you show up, you work hard, you try your best. You do your best possible on the third-year rotations, in particular. And then listen to the guidance at your school about your application and what you might be competitive for because nothing, to me, is as heartbreaking as someone who got their hopes up and wouldn't listen to the guidance that they got that—you're unlikely to get into orthopedic surgery with your resumé, your CV and maybe you should look at another specialty. And then they don't do it because they're a perfectly good—I'm sorry, a perfectly good applicant for a number of specialties even though they might not get into one of the uber-competitive ones. And if they recognize that, they can match and they can have a very happy and very productive career and professional life.
But when they choose not to, that's when we struggle as a medical school when we look back at, how did our students do in the match? Some of those situations are among the more frustrating, not just for us as educators but for the individual as well. And so, I would encourage students to work with their faculty and their dean's office and their guidance to create a good match strategy because, on Match Day, you're going to look around and you're going to see somebody that deserved better. And it's a numerical process that, on the whole, serves everyone well but every once in a while, an individual falls through the cracks. And your job is to not be that individual. And if you listen to your mentorship and the guidance on your campus, you can significantly reduce those odds.
Unger: Dr. Dent, that's just great advice, and it's been such an interesting conversation, I'm sure really valuable to medical students and programs out there. Thanks again for being here today and sharing your perspective with us.
The AMA has a lot of resources to help applicants through the residency application process. You can begin by visiting freida.ama-assn.org. And "FREIDA" is F-R-E-I-D-A. That's freida.ama-assn.org, to access FREIDA, our residency and fellowship database, that has information on more than 12,000 ACGME-accredited programs, as well as resources like our Road to Residency video series.
We'll be back soon with another Moving Medicine video and podcast. In the meantime, you can see all of our episodes at ama-assn.org/podcasts. Thanks for joining us today and please take care.
Disclaimer: The viewpoints expressed in this video are those of the participants and/or do not necessarily reflect the views and policies of the AMA.
