Digital

AI scribes for clinicians: How ambient listening in medicine works and future AI use cases

. 15 MIN READ

AMA Update covers a range of health care topics affecting the lives of physicians, residents, medical students and patients. From private practice and health system leaders to scientists and public health officials, hear from the experts in medicine on COVID-19, medical education, advocacy issues, burnout, vaccines and more.

What is an ambient scribe? How does scribe AI work? Are AI scribes worth it? What are the medical issues with AI?

Brian Hoberman, MD, executive vice president of information technology and chief information officer at The Permanente Federation and chief information officer at The Permanente Medical Group in Northern California. Dr. Hoberman discusses the implementation and impact of ambient scribe technology and shares insights on how technology enhances clinician-patient interaction by reducing administrative burdens and improving documentation accuracy. Also covering the challenges of adapting technology to medical specialties, the future potential of AI in health care and the importance of responsible AI usage. AMA Chief Experience Officer Todd Unger hosts.

  • The AMA is your powerful ally, focused on addressing the issues important to you, so you can focus on what matters most—patients. We will meet this challenge together. Join us.
  • Learn more about our AMA advocacy priorities, including:
    • Reforming Medicare payment
    • Fighting scope creep
    • Fixing prior authorization
    • Reducing physician burnout
    • Making technology work for physicians

Speaker

  • Brian Hoberman, MD, executive vice president of information technology and chief information officer, The Permanente Federation; chief information officer, The Permanente Medical Group

AMA Health System Program

Providing enterprise solutions to equip your leadership, physicians and care teams with resources to advance your programs while being recognized as a leader. 

Unger: Hello and welcome to the AMA Update video and podcast. Today we're talking about ambient scribe technology and its impact on one health system. Our guest today is Dr. Brian Hoberman, executive vice president of information technology and chief information officer at the Permanente Federation and chief information officer at the Permanente Medical Group in Northern California. I'm Todd Unger, AMA's chief experience officer in Chicago. Dr. Hoberman, welcome. 

Dr. Hoberman: Thank you very much. It's very nice to be here. 

Unger: Well, I'm really looking forward to hearing about your experience with ambient scribing, but why don't we just start off by talking a little bit about your background. And it sounds like you have lots of different roles at Permanente, so tell us a little bit about you. 

Dr. Hoberman: Sure. Thank you. So I'm a practicing hospitalist, and have been for almost 30 years. I got into health care technology a little bit before the EMR era and ended up leading the deployment of the EMR for our hospitals in KP, and then later led the―and still do―lead the EMR across KP. 

And over the last few years, I took on some additional responsibilities in technology. And today, I serve as the CIO for the Permanente Medical Group and also for the Permanente Federation, which essentially means leading technology across all of Kaiser Permanente for the physician side. 

Unger: Well, you've seen, I guess, the gamut of technology rollouts then, given what your background is, and the latest with ambient scribing. Let's just start with the basics about how this technology works and why did you decide to start using it? 

Dr. Hoberman: Yeah. Well, the way it works is, it's that wonderful line about a technology sufficiently advanced it's indistinguishable from magic. So it feels like magic to the docs and often to their patients. And it essentially is a recording in the room of the conversation between the patient and the clinician. Sometimes the patient's family members are part of that recording. 

And then after a bit of processing, it turns it into a recognizable and often very accurate and readable clinical note, and also can turn into decent after-visit discharge or post-visit instructions for patients. So it accomplishes both of those things. What docs love about it is that it increases the patient-doc direct connection, because you don't have a keyboard and a screen in between them, and it really does save docs time. 

Unger: So tell us a little bit about the trial that has been undergoing with ambient scribing and what you learned from it. 

Dr. Hoberman: Sure. We're past trial at this point. We're fully deployed across all of Kaiser Permanente. But about a year ago, in the Northern California region, our physician CEO saw the opportunity. And we, at that time, started a pilot with a vendor that ultimately was not the permanent vendor that we ended up using. And we rolled it out initially to a handful of docs, got feedback, and then we rolled it out more broadly and discovered that this ambient listening technology really worked. So that led to, across all of Kaiser Permanente, a decision to go with an enterprise approach. And at that time, we looked at vendor options, and we started rolling out an enterprise vendor, and that is completely rolled out today. 

Unger: That's amazing. And it sounds like that's pretty fast from kind of pilot to rollout. Is that right? 

Dr. Hoberman: Yeah. It is fast. It's fast. But what makes the ability to go fast, that people accept it, they like it, they want it and it works. The old rule of thumb, software must work―this works. 

Unger: And it's interesting because, of course, what you point out, a lot of the burdens that exist without ambient technology I can see why there would be such a positive reception to that. You're still in the early stages with this. Where do you see room for improvement or where are you going to go with it? 

Dr. Hoberman: That's a great question. So in the early stages, what we see are pockets where it's not tuned to the specialty. And so the conversation with the vendor for us is about, here's what feedback we're hearing from those specialists. And of course, we have the specialists directly in those conversations. And the vendor then makes adjustments to the model that become specialty-specific. 

The early, early days were concerns about things like hallucinations and did it get it wrong, and was it too verbose? And that kind of stuff. And while we have paid extremely close attention to that and will continue to do so, that actually hasn't turned out to be the barrier. The barrier was actually usability fine-tuned to specialty workflows. 

Unger: Tell us a little bit more about what's the disconnects on the specialty side. 

Dr. Hoberman: It's the nature of the work they do. So if you're an oncologist, your oncological history is going to carry over from note to note. And it wasn't necessarily grabbing that prior information. And so it's doing some work along with the EMR vendor to say, OK, let me go find that templated information that you always move forward, and let me add to that aggregate information as you move your note forward. 

There's also some understanding of drug names and other things like that. If you're a mental health provider, it's collecting a history and documenting the history that's really fine-tuned for a mental health note as opposed to an internal medicine note, which certainly may touch on behavioral issues, but there are differences. And those differences are subtle enough, as you can imagine, that it's an opportunity for the vendor to do some work. 

Unger: Well, what's interesting about your response too is it really does bring to the fore the role that AI is playing in the ambient scribing arena here. And what a powerful tool it can be to reduce administrative burdens on physicians. What do you see in the future? How else can it help? 

Dr. Hoberman: Yeah. That's a fun question to think about. When you think about the future future, I mean, it's incredible. I think that a couple generations from now physicians will practice differently in ways that we might be able to try to imagine today, but it's really hard to pin down. 

I don't think the technology is going to get worse. You've heard the cliché that the AI of today is the worst AI we will ever experience. But this notion of what its job is, is undefined. Today, the job is clearly augmentation. And it's augmentation of the kind of work that, for docs or other people in health care, is sometimes tedious, burdensome and time consuming, and it allows people to turn their attention to those things that are more―the value that a human can contribute to this thing as opposed to something like just banging out a note as you're listening to somebody. 

Now, the doc, of course, sees the note, reviews the note, revises the note. And as you heard from the examples I gave of people who aren't satisfied with it, are not in every case saying, hey, this works for me right now. But over time, it's just going to get better and better. So that's the ambient side of AI. There are plenty of other sides of AI too, that are going to develop over time. And the big EMR vendors are working on those things. And if you like, I'll give you an example. 

Unger: I'd love that. 

Dr. Hoberman: So I had a patient actually this weekend who was on the higher end of the complexity bell curve. And she was new to me. And there will come a time when instead of doing what I did, which was spend a very significant amount of time reading through specialist notes―and there were multiple specialists involved, and I had to synthesize a lot of information. 

Ambient, which is pretty good at reading things and reads faster than I do, can synthesize it in a way where I can get a summary of that patient that actually may help me be a better doctor by appreciating more pertinence to the story than I was able to appreciate. And why, in this case, was it so difficult? Because this patient was brand new to our organization. And so the understanding of who she was wasn't carrying forward. I had to synthesize that understanding. 

Unger: That's a great example. And what a nice benefit to the patient of just feeling more seen and more known, even though she was new to you. 

Dr. Hoberman: Yeah. 

Unger: When you look to the future and you're thinking about using AI, I mean, scribing, are there any kind of examples or things you're coming across that you're thinking about in terms of using this in a responsible way? 

Dr. Hoberman: Yeah. Whenever I talk about AI, I honestly just use two slides. One slide says "AI," and the second slide says "our AI." So you really can't talk about AI without thinking about being responsible about AI. And what does responsible mean? Fundamentally, I don't think it means something different than being responsible with pharmaceuticals or responsible with device implants. There's an inherent responsibility that physicians have towards doing the right thing for their patients and doing no harm. 

And so then we get into, well, what is doing no harm and doing the right thing mean? Well, for one thing, in the case of, like, ambient it means asking permission. Is it OK with you if we use this tool? Here's why I would like to use it. Is it OK with you? It means protecting privacy at the level that is expected of health care organizations. It's making sure that there's no built-in bias and built-in inequities and that kind of stuff in it. 

And then it gets to―and there are probably other considerations too that I'm not going to touch on today that if I had something in front of me, I might say this and this too. But the other one that's always top of mind for me is, does the software work? Does it do the job? We don't prescribe drugs that don't do the job. Sometimes the drugs have side effects and we understand them. But that's the trade-off that we make using our judgments. We have to make sure too that when we're using technology, that it's purpose-built to do the job that we need to do and that we understand what the risks are when we're using it. 

Unger: Now, a lot of times when we talk about advances like this, we're talking about the technology. And I'm often reminded by leaders like you that that's only one piece of the puzzle. Obviously, there are the people and the processes and the systems and just the general operational environment. Any learnings on that front of how you're knitting all those things together with the technology? 

Dr. Hoberman: Yeah. That's a great question too. So everybody who's paying attention to the world has heard of AI and has seen examples of AI. And some people may have touched it themselves. Some people may have played with it. Some people may not have, but they are aware of it. I mean, I just saw a segment on 60 Minutes a couple days ago where OpenAI is collaborating with Khan Academy on using AI in the classroom to educate young children. It's ubiquitous, and the applications are everywhere. 

And yet, many, many people who work with us in health across the full scope of the kinds of folks that we work with are not necessarily fluent in what AI is. And there's this open question about, what does it mean to be AI fluent? It's similar, in my opinion, to, what does it mean to be cyber secure? How do I know what's phishing? How do I know what not to click on? How do I behave responsibly with computers? 

And so I think it's an evolving understanding, but it's incumbent on us as leaders to think about that and to try to make sure that the education that we provide to our colleagues and our workforce isn't just coming from the media. Then when we get―and by the way, my organization, over the last couple of years, has done formal education on these things, and will continue to do so. But it's evolving so quickly, there's a different tale to tell, really, every year as we look at it. 

On the people side of things, not everybody instantly adopts technology change. I don't know if that's a shock, but it turns out that―and I can relate to that because I don't adopt kitchen gadgets readily. And yet I have other people saying, this thing is fantastic―you should use it. So everybody has their own pace of comfort with adoption of change and that kind of thing. And as leaders and as people who are trying to support care delivery, we have to be really empathetic to that and try to make sure that we reach people as they are and we work with them as they are. And that means that we're going to be working with a lot of diversity. 

Unger: Just a final question and going back to something you mentioned in your background. I mean, you've been involved in technology projects from the world of the AEMR and through what you're working on right now with AI. Are you seeing more of a focus on reducing burdens for physicians as opposed to maybe creating them in the first place? 

Dr. Hoberman: Yeah. I think about reducing burdens every day, and the people I work with do as well. But you think about more than that, like what problems do we want to solve in health care, right? We want to improve access. We want to improve quality. Reducing burdens is part of―reducing burnout and increasing morale and increasing satisfaction, attracting people to the field. 

So there are many, many things that you think about where you ask the question, how can technology assist with these sorts of things? And certainly, as somebody who's a veteran of the EMR era, I heard more than once that some of the stuff that we implemented made things harder for them. But it's the classic dilemma of, there's so much clicking and all this other stuff, but never take it away, because the power of the thing. 

And now we come along and we say, OK, we have a new generation of technology that can focus on some of those problems that previously were insolvable. And what else can it do? And that actually makes me really enjoy coming to work every day and thinking about how we work on that. 

Unger: Well, that's very inspiring and a great place for us to end today. Thank you, Dr. Hoberman, so much for joining us. I know there are so many practices that are thinking about AI right now, and so hearing about your experience is going to be really helpful. 

If you found this discussion valuable, you can support more programming like it by becoming an AMA member at ama-assn.org/join. We'll be back soon with another AMA Update. Be sure to subscribe for new episodes and find all our videos and podcasts at ama-assn.org/podcasts. Thanks for joining us today. Please take care. 


Disclaimer: The viewpoints expressed in this video are those of the participants and/or do not necessarily reflect the views and policies of the AMA.

Subscribe to AMA Update

Get videos with expert opinions from the AMA on the most important health care topics affecting physicians, residents, medical students and patients—delivered to your inbox.

AMA Update podcast logo

FEATURED STORIES