AMA's Moving Medicine video series amplifies physician voices and highlights developments and achievements throughout medicine.
Featured topic and speakers
In today’s episode of Moving Medicine, AMA Chief Experience Officer Todd Unger talks with Jesse Rafel, MD, at the Institute for Innovations in Medical Education at the NYU Grossman School of Medicine in New York about changes to the USMLE® Step 1 Exam and what medical students need to know.
Want to learn more? Register for AMA's upcoming webinar “Navigating changes to Step 1” on Oct. 27, 2021, 7–8 p.m. Central.
Speaker
- Jesse Rafel, MD, Institute for Innovations in Medical Education, NYU Grossman School of Medicine
Transcript
Unger: Hello, this is the American Medical Association's Moving Medicine video and podcast. Today I'm joined by Dr. Jesse Rafel at the Institute for Innovations in Medical Education at the NYU Grossman School of Medicine in New York. We're talking about changes to the USMLE Step 1 exam and what medical students need to know. I'm Todd Unger, AMA's chief experience officer in Chicago. Dr. Rafel, thanks for joining us today. Why don't we just start by talking about what the changes are to the Step 1 exam?
Dr. Rafel: Well, thanks for having me, Todd. Great to be here. Yeah, some big changes. January 26, 2022, individuals who take the exam on that date or afterwards, it will no longer be reporting a score. It will be scored pass/fail, which is a big deal. In addition, there are some more minor changes, like reducing the number of attempts from six to four as a maximum number of attempts for any USMLE exam, again, going live around the same time.
Unger: So, generations of medical students before this, have relied on that test score, as they applied to residency programs. I know that pass/fail change is something that both educators and students have been advocating for. Why is that the case?
Dr. Rafel: Yeah, I think we've seen the USMLE exams, which are licensure exams, meant to make a determination about licensure, be used for secondary uses, like screening for residency application in the process, being used at a single measure for screening applicants. Those sort of unintended uses have led to all kinds of downstream impact, that is really unfortunate. Led to a lot of stress around studying to get scores well above the passing threshold, which right now is 194. So, those unintended implications of that scored exam led the NBME and the USMLE to make this change and I really think it would be a net positive, although, as we will talk about likely, some growing pains here in the transition period.
Unger: I guess, with any change, there's adaptation, a little bit of confusion. Obviously, really important for medical students to understand these changes to Step 1 right now. Can you talk about why that's so important?
Dr. Rafel: Yeah, I think right now, it is a time of transition and so there's going to be more talk of this, particularly in the coming year, as this change is made about how do we handle individuals who may have taken it, graded or scored, I should say, before that deadline and pass/fail after and how is that going to be handled? These are active conversations happening in the education community. Students should expect to hear more about that. They should work through their member AMA medical student sections to advocate for transparency in this process.
So, it is a timely ... it is the time to be talking about this stuff. On the other hand, I think there ... my hope is that there's a lot of positive that comes out of this, that it drives the community towards a better approach to how we use these types of scores. So, I'm excited for this change but I totally, totally recognize having not been too far away from my medical school years, the uncertainty can bring a lot of angst for this transition. I
Unger: I like the way, and I, for whatever reason, haven't heard anybody talk about it before in terms of unintended uses. I guess those unintended uses have been going on for quite a while. So, I'm interested in talking about the implications here. So, let's just start kind of in general, what are the implications for a pass/fail Step 1 for students?
Dr. Rafel: So, the biggest implication and the intended consequence of that, the reason part, in large part that this change was made is that hopefully by this being pass/fail, students will not feel the need any longer to devote huge amounts of time for studying to get very high test scores, well, above that passing threshold. On Step 1, we hope that this is leads to reduction in the stress and anxiety around studying for this exam, which is somewhat different than what you practice in clinical medicine. It is an exam somewhat separate from clinical medicine. So, I really do hope that that is a key consequence of reducing stress, focusing on the coursework you have, whatever your medical school is offering and really being fully engaged in your coursework, which is something we saw that the exam itself was taking away from.
I do think there will be other consequences and that's going to be driven by how programs react to this change, and being entirely frank with our audience, nobody knows exactly how programs are going to react because each residency program is a little different but I think we have some ideas around.
Unger: So, I'm a person who's on the patient-end of this thing. Don't we want them to score really high on this exam?
Dr. Rafel: Yeah. So, a great question and the thing about the USMLE that although the scores were much maligned, no one will debate that passing it is a very important threshold for patient safety. So, nothing about the standards, and that gets reevaluated every year about what that passing threshold should be, nothing about that has changed. So, patients should know that the standards for becoming a doctor have not changed one bit. The standards for getting licensed in the U.S. have not changed and I think it will, in fact, drive folks to focus on other areas besides test- taking that are really important for being a doctor, like communication, that might not be tested so much on Step 1.
Unger: What you mentioned, basically, we don't know how programs are going to react to that. So let's talk a little bit about one of those unintended uses, so to speak, is about residency programs. How's that going to change their selection process?
Dr. Rafel: Yeah. We saw a phenomenon over the course of years and it's a multifactorial phenomenon but residency programs receive many applications for the number of positions they have. When faced with that deluge of applications, using shortcuts to screen individuals out who they think might not succeed. One of those shortcuts was using the Step1 score and that drove some of that mania around getting really high scores. With that taken away, I think there are, this could go different ways for different programs. My hope is that programs will embrace holistic review, which is really taking a look at the whole applicant. Yes, spending some extra time to do the process and really consider considering other metrics they have. Yes.
Step 2 CK will remain scored, which is the other part of the USMLE series, but also considering their other attributes and experiences and who they are as a person, and how that aligns with a given program. So, that's my hope. I also really implore programs if they do start to use Step 2 CK in a more substantial way in this cycle, to recognize that there's imprecision in the scores. So, two individuals that score less than 16 points different do not statistically differ on Step 2 CK. So, we see programs making decisions with minor differences and really, I think that is and again, an unintended use of those examinations scores.
Unger: In my whole career, how hard a time I've had getting people to understand statistical significance in scores like that. That's tough. When you're on the other end of this, you're a student, you don't have ... Now the score, let's say, if you had a fabulous score, that's obviously something that you could showcase in your application. What is a student to do to distinguish their application in this process now?
Dr. Rafel: It's a great question and I totally get it, and it is a competitive process. Let's not pretend it is not. Step 2 CK, I do think will take on newfound importance. I don't want folks to take on the same mania that it revolved around Step 1, but I think doing well on Step 2 CK is going to be an important measure for most programs. But, when you survey residency program directors and say, "What was it about applicants who got a look by you and actually ranked to potentially match at your program? What was it about them that really stood out?" It's not USMLE number one, two, three, four or five. It's perceived commitment to my specialty.
It's the genuine interest in our program. It's evidence of professionalism and communication skills. So, I'd really implore applicants to remember that there's a lot of other things that programs are looking for and if you can try to convey to them your genuine interest in that specialty and that program, your genuine ability to communicate with patients and show professionalism out when you're caring for patients, those things can far outweigh even a score on Step 2 CK, which will remain scored, that maybe isn't where you wanted it to be.
Unger: When we talk to medical students, especially through our Medical Student Section, about this issue, one of the things that's come up a number of times is just the kind of leadership experience that you can get through an organization like the AMA, learning about policy making, getting involved in that, being active in profession. Is that something that programs will take a look at?
Dr. Rafel: Absolutely and I can't speak for every program. Every program has a different process, so I don't want to pretend like I know each of their processes but having done resident selection myself at one large program, I can say that it is important to show evidence that you are able to function as a leader, as a team member, as a follower, at times and sometimes that happens in the clinical environment. Other times it could happen in an advocacy environment like AMA MSS offers. So, I just really encourage students who I talk to, to follow what their passion is. Show that sincerity of passion, your passion. Don't fake it, follow what you really want to do and if that is advocacy, that's a fantastic way to show—hey, I'm passionate about this issue—and that issue could even be these sorts of changes that are happening in our community.
Unger: Dr. Rafel, we just finished recognizing International Medical Graduate Recognition Week and we talked a lot about that through the process about, the obstacles that IMGs face. Is this going to be another change that's going to have an obstacle effect for IMGs?
Dr. Rafel: I think it's a great question and it's unfortunately, another unknown. IMGs are a key part of our application pool and our workforce in the U.S., and so there's no doubt about it that they're key in all of this. I think it is a concern, that was one entryway into the profession of medicine in the U.S. for IMGs because it was standardized. I will come back to Step 2 CK again, I think, whereas for U.S. medical grads, Step 2 CK may have been more important to some degree at some programs. I think for IMGs being realistic will be quite important as a way to show standardized performance compared to peers.
Unger: Well, in the spirit of helping folks understand these changes and implications, the AMA is hosting a webinar called Navigating Changes to Step 1 for students on October 27, during which you and another panelist are going to dive into these changes and let students know what they need to be thinking about. Can you just highlight what will be covered and why students should attend?
Dr. Rafel: Yeah, thanks for that. Thanks for that pitch. We're really excited to have that. We'll get have a little more time to go into some of the deeper nuances here, and most important, answer questions that folks can submit and try to get down to the nitty gritty of how folks are navigating this right now and some of the uncertainty. Try to address uncertainty where we can. Call out the uncertainty where it still exists and I think that'll help the community also provide feedback to students moving forward on areas where they can hopefully get more transparency for those students. So, if you're a student listening, I encourage you to join that. If you're not a student and you're a faculty member, I really encourage you to send that out to your medical student listserv as an opportunity for them to hear more about these changes, and hopefully we can dispel some myths and also present it in a factual way about what's what is changing and what is not changing.
Unger: I'm going to echo what you has said. This is going to be an incredibly important webinar. You can register for it at ama-assn.org/step-1-changes. That again, that's October 27. Do not miss it because it's going to be really interesting and I know everyone out there is worried, as usual, about anything that changes. That wraps up our episode today. Dr. Rafel, thank you so much for being here and sharing this information and for being part of this upcoming webinar. Don't miss another episode of our video or podcast, so make sure to click subscribe on our YouTube channel or on Apple, Spotify or wherever you listen to your podcasts. Find all of our videos and podcasts at ama-assn.org/podcasts. Thanks for joining us.
Disclaimer: The viewpoints expressed in this video are those of the participants and/or do not necessarily reflect the views and policies of the AMA.
