AMA Update covers a range of health care topics affecting the lives of physicians, residents, medical students and patients. From private practice and health system leaders to scientists and public health officials, hear from the experts in medicine on COVID-19, medical education, advocacy issues, burnout, vaccines and more.
Featured topic and speakers
Benefits of AI in health care: How is AI used by doctors? What is augmented intelligence? How are LLMs used in medicine? What is ambient AI for medical documentation?
Our guest is Vincent Liu, MD, MS, chief data officer, at The Permanente Medical Group. AMA Chief Experience Officer Todd Unger hosts.
- Learn more about the AMA Health System Program.
- Wondering what is the difference between augmented intelligence and artificial intelligence in health care? Check out this resource.
- The AMA is your powerful ally in patient care. Join now.
Speaker
-
Vincent Liu, MD, MS, chief data officer, The Permanente Medical Group
Transcript
Unger: Hello and welcome to the AMA Update video and podcast. These days, it feels like everybody is thinking about ways that they can use AI in their practices. Today, we're going to be talking about what it takes to bring an AI initiative to life. My guest today is Dr. Vincent Liu, chief data officer of The Permanente Medical Group at Kaiser Permanente, joining us from Santa Clara, California. I'm Todd Unger, AMA's chief experience officer, in Chicago. Dr. Liu, it's great to have you back.
Dr. Liu: Thank you. It's always a pleasure, Todd.
Unger: Well, the last time that we talked, you spoke about some of the AI programs that you have in place at Kaiser Permanente. Why don't we just start, for background, with a brief recap of the ways—at least the most important ones—that you're using AI right now.
Dr. Liu: Yeah, we're doing a lot of work in AI- and machine learning-based clinical decision support. So that's stitching together complex pieces of data from across the EHR and other places to try to make information more accessible to our clinicians, help them improve the way they think about risk stratification and they approach their patients and our populations.
We are also looking into large language models. A lot of innovation and excitement around the use of things like ambient AI. Those are devices that listen to clinicians and patients as they talk and summarize that transcript, and thus reduce the clinical burden of documentation that is overwhelming a lot of our physicians.
Unger: I had a chance to visit The Permanente Medical Group about a year ago. And one thing that really struck me was just how involved—deeply involved—in technology issues the physicians are, which of course makes so much sense. How does it feel to be able to help lead in that arena?
Dr. Liu: It's amazing. I would say physicians are our greatest asset in this medical group. They are on the front lines and they are leading as well. And when you have that kind of structure, it offers tremendous opportunities to understand what are the enhancements that technology can bring to the everyday work that we have, to our objectives to achieve the quadruple aim—whether that's patient outcomes, population management, health care costs or provider wellness—all of those are more important than they've ever been. And having physicians so deeply integrated into the decision making around this technology and how it should be deployed I think is critically important.
Unger: It's so great to hear it starting from there because, as we know, not all technology initiatives in health care began that way. Let's talk about AI. So much potential. Obviously still new. And a lot of practices are trying to figure out where to start. I'm curious, when you think about the AI initiatives that you outlined, other ones that are in the pipeline, how do you prioritize the ones that you want to pursue?
Dr. Liu: I think that's a great question. A lot of it has to do with, again, the quadruple aim. It's just something in which there's an opportunity to leverage the best-in-class technology in order to improve one of our objectives. And I'll use ambient AI as an example because it's fresh in my mind. We recently published some of our experience from The Permanente Medical Group, in which we've deployed this now across all physicians in Northern California to assist in their conversations with patients. And what we've heard is overwhelmingly positive feedback in terms of its ability to reduce the clerical burden of documentation, its ability to improve the engagement and conversation between patients and their providers.
So it's just one step in trying to overcome some of, again, that clerical documentation or administrative burden, which frees up doctors to do what they love to do—which is communicate with patients and interact with patients.
I'll mention another thing, Todd, because I think last time I was on, we talked a little bit about our Aim High Program, where we were funding five external health systems to deploy AI in randomized trials to do rigorous real-world evaluation. So that's another way we look at what's shovel ready in terms of the joining of the technology as well as the use case, where there's a compelling need, if the technology improves the way that we care for patients or improves their outcomes, we need evidence of that so that we can really support that kind of work.
Unger: I'm going to add "shovel ready" to my lexicon there. I love that, the way you stated that.
One of the things—and I think this is consistent in the leaders I've talked to at The Permanente Medical Group—is that it's not just about technology. It's about people and it's about operations. When you think about a program, let's say, that you've prioritized or you're trying to implement, what's the team that you're bringing together to make something like that happen?
Dr. Liu: Yeah, I like to use the phrase "augmented intelligence" because augmented intelligence places the people—whether that's patients, clinicians, or community—rather than algorithms at its center. So the team, really, for us, the core competencies for this, is in three domains. We call it our three-legged stool. The first is clinical integration. So that's everything about, how will this thing actually be deployed in a workflow, understanding all aspects of that, and ensuring or working our best to ensure that this will be seamless, hopefully efficient, and ultimately effective and safe.
The second is technology enablement. That's a challenge these days—getting the data to the right place, then running the algorithm, then feeding the data back, and presenting it in a way that's accessible and understandable by our clinicians. That's really important. So that's with our information technology partners, data and analytic partners, enterprise architecture and infrastructure—all of what makes that precious stream of data flow so that it enhances the work that we do.
And then, ultimately—and I think this is the new part—is data scientists. It's all about, what is the method? How do we rigorously test and evaluate performance metrics of those methods? And then, ultimately, how do we evaluate this to know that the technology, integrated with our clinical workforce and our data scientists, ultimately produce something that's valuable for patient health or provider wellness?
Unger: It's so interesting. Physicians aside, that sounds a lot like the people on my team, and just the ability to combine the data science to the operations to make sure that what you learn actually informs the strategy going forward and to make that testing environment part of the operation. Of course, new technology, new processes like that, require a lot of training. How do you train your teams to use these new tools? How do you roll them out?
Dr. Liu: Yeah, I mean, I think it's important to note that we've been doing quality or performance improvement for a long time. So the fact that we're bringing in cancer screening or hypertension control or blood—other types of population health—we've been rolling those out to our clinicians and our teams for a long time. So it's built upon that foundation of, how do we effectively educate, how do we maintain, and how do we reach our performance targets with respect to the way we think about a large population of patients whose health we want to improve?
Secondly, even before AI, we had a long history of technology implementation. So technology has taken over even before AI in terms of EHR and then the other channels of communication, which are really important to improve coordination across these complex teams. So it's really built on that foundation.
I think what distinguishes AI from what's come before are uncertainties about, how do the methods work, and more importantly, where are their failings? And so I think that's a lot of the education, is, what is a neural network or a deep learning, or what is a regression of this or that type? And where should we apply that with confidence? And then where should we be cautious about what this output is telling us?
As of today, clinicians and humans are still in every loop, making decisions about what these data are telling them. We do think that it provides them a little bit more insight than what they had before. But, again, we teach them to take care of the patient in front of you and to integrate a diverse stream of data, other features that come from their experience and their knowledge, in order to provide expert care to this patient.
Unger: Absolutely. In some respects, the accomplishment will be having them not even notice that it's there. It's just that, as you said before, it's the augmentation and the ability to return the physician back to doing what they want to do. I'm curious—you've got a lot of stuff under your belt already—what's next for you? What's the next big AI move?
Dr. Liu: I mean, there's so much excitement about large language models that I think that's going to be a big focus. How do we identify the best use cases, the ones which maximally, again, help us to reach the quadruple aim? I think anything that reduces physician burden, administrative or clerical burden, has got to be top of mind for every health system, every medical group. There's a ton of other things which are more traditional that we've been doing for some time—clinical decision support, computer vision. I think we're going to see a bounty of those things really start to get integrated in that seamless way you described.
So it's the balance of making sure our pipeline is robust for bringing through clinical decision support, computer vision applications, then, really thinking hard about what the optimal use is of things like large language models are. And that's probably taken up most of my time in terms of how we operationalize these technologies.
Unger: Well, Dr. Liu, I can't wait to hear more about what you're up to. And we'll check back in with you in the future. In the meantime, thanks for joining us today, and we'll talk to you soon.
To support the AMA's efforts to make technology an asset for physicians and not a burden—that's what this has all been about—you can become an AMA member at ama-assn.org/join.
That wraps up today's episode. And we'll be back soon with another AMA Update. Be sure to subscribe for new episodes and find all our videos and podcasts at ama-assn.org/podcasts. Thanks for joining us today. Please take care.
Disclaimer: The viewpoints expressed in this video are those of the participants and/or do not necessarily reflect the views and policies of the AMA.
Subscribe to AMA Update
Get videos with expert opinions from the AMA on the most important health care topics affecting physicians, residents, medical students and patients—delivered to your inbox.
