Trust Is Not a Feature: The New Rules of Clinical AI Governance and Validation Becker's Healthcare Podcast ยท 29 Sep 2026 Source: https://podcast.show/beckershealthcarepodcast/episode/155568759/ Transcript: https://podcasts.opennash.com/shows/beckers-healthcare-podcast/trust-is-not-a-feature-the-new-rules-of-clinical-ai-governance-and-validation-5375e1/ (via OpenNash Podcasts, machine-generated) Well, hello, everyone. This is Erica Spicer-Mason with Becker's Healthcare. Thank you so much for tuning into the Becker's Healthcare podcast series. Today, we're going to talk about clinical AI governance. And joining me for this conversation today, we have Dr. Sheila Bond, the head of clinical AI at Wolters Kluwer Health. Dr. Bond, welcome. Thank you so much for being here today. Thank you. Pleasure to be here. Well, Dr. Bond, it's so great to have you with us today. And I thought to get us started, it would be great if you could just share a little bit more about yourself, your path in healthcare, and how does your work as head of clinical AI at Wolters Kluwer look day to day? What does it involve? Sure. It's been, I mean, one, it's just such an exciting era for medicine. I've been in medicine for 22 years now. So taking care of patients either at Mass General or Brigham for that time period. And for the past 14 years or so, alongside of that, I've been in clinical decision support, usually at the intersection of content and technology. So have really thought deeply about how the words that we create in medicine, whether that's through evidence-based medicine reference or tool, or how we teach students on the wards, making sure that the information that we give to the next generation is the right information for patient care and always encouraging them to think critically. So this has been just a really exciting era for me. I think it's just a shift in how we're going to create medical information for generations to come. So I feel really honored to simply shepherd that in. And the teams that I work with are amazing in terms of the dedication and passion that they have for this mission. As the head of clinical AI, my role is really to help them create the vision of what's next for how we deliver information, whether that's through technology or through the words that we write on the page. It's also getting clinicians and engineers together to create that vision that takes both of their combined expertise to figure out how to use generative AI to remain trustworthy and to create not only the system that creates information and presents it in a new way, but also how we validate it, how we govern it, and how we hold ourselves accountable. And it's exciting from start to finish, and it probably taps into every element of every skill that I've accumulated over the years. Yeah. Dr. Bond, I appreciate the overview, by the way, and what an exciting role you've stepped into, as you mentioned, the fact that it's combining so much of what you've already done in healthcare and is navigating a lot of uncertainty ahead. It sounds like a very exciting place to be. And I want to go a little bit deeper on something that you mentioned earlier in your response. And that was this real dedication that I know your teams at Wolters Kluwer Health have when it comes to clinical AI and making sure that the content that is surfaced at the point of care is trusted. Trust comes up constantly in conversations when we're talking about clinical AI, but it can definitely mean different things to different people, different stakeholders in the care journey. So from your point of view, when a clinician or a health system says that they trust a tool, what do you think that's really resting on? That's a great question. I think the era has really come back around to trust and what it means specifically. So that's one of the questions that I hear a lot these days. It's not a feeling, right? And I think the way we have defined it is one, there's governance. There are people who oversee the system, how it's designed and know how it's designed. So they can A, create what needs to be created and B, measure outcomes and measure how it functions and display that to others. So the fundamental tenets of trust for us are one, having a governed system. Two, at this point in time, generative AI is still early in its history. So we make sure that it is grounded in content that is worthy of our trust already. And I think that's one thing that people forget is that there has to be a source of truth, right? Genitive AI is very good at assembling words in the right order and having them sound great, but making sure that they are actually tied to a source that is trustworthy itself is important. And then the validation and the data that we produce is also, I think, really core to trust, right? Does the system perform as expected? How do we know? How do we measure that? And are we willing to share it with you? And then ultimately, you know, who is accountable for this. When something goes wrong, who do you call? How do you discuss it? How do people act in that circumstance? It is a new technology, so it won't be perfect, but we do have ways and means of introducing things into medicine when they're new. And I think those are the central tenets, right? That there is accountability, governance, measurements, and the ability to act and improve. And I think the other thing that is really important is that technology took hold of this early on. And a lot of the measurements that you see out there are really coming from the technical world. But for us in healthcare, the end user is in healthcare. And usually the information that is acted upon is for a patient. So I think we all collectively, as a community who is bringing this together, needs to remember that that's the end game in this. And the measurements are not just technical measurements of progress for an AI system, But the measurements that we want to put forth really demonstrate that this is enough to act upon and the right guardrails and safety mechanisms are in place for use for with the patient as the endpoint in mind. it's a really helpful framework dr bond and i appreciate you elaborating on those key pillars that kind of are building this this concept of trust and not just a concept but something that is measurable actionable and i want to go a little bit deeper too on governance you had mentioned governance a few times now and this is another term that can be used kind of broadly just like the concept of trust. So inside an organization, what does AI governance look like when it's working well? I'd love to know, you know, from your, where you stand, who tends to own it and how do decisions actually get made in a way that's effective? That's a wonderful question as well. And it really forces you. And I think a healthy way to break down what quality is so that other people can understand it, see it and measure it. So it's a good forcing function. Our governance is really cross-disciplinary. I think our clinical and our clinical AI team set the standard for what good is. Everybody, right, needs to understand that so that we can measure it. That includes our product teams and our technology teams. And all of us, right, are responsible for understanding the industry out there. So technology does need to understand a bit of healthcare and healthcare does need to understand a bit of technology. So I think for the most part, you know, in terms of the organization working well, It's all of us collectively understanding the same thing, having the same vision and predefining what good is and how we're going to measure that, building the system and then building the tools to make those measurements, looking at them all together and understanding whether we've met our goals or not. And if we have or if we haven't, is there anything unexpected that we didn't foresee that has come up? And then I think one of the most challenging things is sort of shifting modes, is shifting from sort of creating into measuring. And then before release, technology is known for fast release cycles. But once you get into the elements of understanding and managing risk, it's a different framework, meaning you work until that risk is gone or otherwise mitigated or settled in some other way. And I think that's sort of a mark of the era is, A, we all have to be able to share a vision, communicate to one another, because there's no one domain, so to speak, that's going to be able to do this themselves. It's going to take an alliance between healthcare and technology. technology. So it is that ability to break something down so that folks you don't usually work with can understand it, to measure it, to measure it to a pre-definition, and then to look at it and say, have we done enough? And if we haven't, we no longer work to a time, right? You work until that risk is gone. Yeah. Yeah. Dr. Bond, I appreciate you defining governance. You know, we are fresh off of our IT and revenue cycle conference at Becker's. And I have to say that the way that you just described it aligns so closely with what I know a lot of organizations are striving for. But to your point, it is challenging to put into action. And also, there's no one way to do governance. So it seems to certainly be a challenge for most organizations. And I also want to follow up on this idea of risk here and talk a little bit more about validation. I know validation tends to get framed as something that happens before a new tool or technology goes live. But how do you think about it once a tool is already in clinical use? What's hardest to measure at that stage? Great question as well. I think that speaks to sort of the accountability pillar that I talked about earlier. It's not only what you do before you design your system and how you measure it when it's at the gate, but that post-monitoring surveillance is very important. So looking at whatever your system is designed to do, looking at how it fares in the real world, at this point in time, I think for most clinical AI systems, it's understanding the types of questions that will come in, what your system's output is. And we're always really sure that we've got expert eyes on the sampling of what is out there in the industry, just because of how precious it is. The hardest thing, of course, is that this is a technology that isn't going to be perfect for a while. So measuring how you do in the real world isn't an easy matter. And even so with some of our traditional medical references, you can write something to perfection in that state, but still someone might not understand it well enough. Right. So the hardest part is, you know, the gap between, you know, doing everything you possibly can to ensure that you're delivering what should be worthy of trust. But then, you know, the actions that someone takes or how someone interprets that information is, you know, always one of the things that will keep you up at night. That's sad. Right. I think, you know, when you're talking about governance, it probably, you know, there's the industry level governance. And then there is also, you know, what we are responsible for as a community of medicine and as an industry, which I think is to educate our users on what AI literacy is and how to use these tools, both individually. And then, you know, that's now part of medical school education as well, coming more and more AI literate. So I think those things go hand in hand with the governance of a system. And as always, it's just really heartening to be part of the community of medicine and see that they do respond to that call. Right. I've been all over the world talking about clinical AI and whether it's, you know, in Europe or Asia or the United States. I think everyone is starting to speak the same language and taking on the responsibility that the community of medicine does. So it's been really heartening to see folks step up in the way that they usually do and heartening to see that across the world, there's the layer of medicine and how much people care, no matter what region they're from, that we get this right. I appreciate the global perspective there, Dr. Bond. And it's so interesting, too, to hear how medical curricula are already changing to support AI in clinical use. And kind of in that vein, I'm curious whether we're talking about clinical AI use or governance or even validation. What questions are you hearing from clinical and IT leaders now that maybe you weren't hearing even just one or two years ago? I think the questions are more pointed and precise now. I think we're sort of beyond the wow factor and we're beyond the usability factor and sort of the fun factor and the novelty factor of this. And we're into how are we actually going to integrate it and what does it take to get it right and what does it take to scale? So, you know, as you talked about at the beginning of the interview and trust, people are now asking not just for the who and how or the system design, but really the specific data and starting to drill down into what really matters. So I've been, one, we've been working on that foundationally for so long. I'm really thrilled to hear those questions coming back from the industry itself and from academic medical centers as to what really the foundation should look like and what should you measure to make sure that whatever we build now is going to be the foundation for the future. So I'm really happy to hear that people are educating themselves, looking at the data, reviewing the data, right? There are chats on LinkedIn about benchmarks in medicine and how best we should measure things. So I'm just very glad to see that those questions are coming up and people are asking for what precisely one should measure. And in that regard, I think for us, one of the most important things is to make sure, and this is really interesting because, as I said, I was just in Asia and Singapore and China, and we were all coming to the same point about what of all of this should be measured as we're starting to grow and to scale. and making sure that when this technology is creating words that will impact patient care, that there is no unsupported leaps, right? And how do we start to quantify inference, right, is sort of this point of intersection that we were at across, you know, three different countries. So it was very interesting. It's not like a benchmark or a safety score or something gimmicky, But the intersection that we were at was, for years, we have, decades really, in the field of evidence-based medicine, made sure that we have a system for analyzing and appraising the medical literature to make sure it's the right information to act on for patient care. And we have that strong foundation. So we now don't want to create a new foundation based on ease of use or just information that is plausible or unsupported. And that technology can so easily do that. What we came around to, I think, in discussion was the ability to quantify and measure whether your system is taking a leap and making a recommendation for patient care or you being disciplined. And, you know, even though it might not be as personally pleasing, right, to get a more challenging answer to your question, that may be the right thing to do. And that's what we want to measure. Yeah. Well, it's been great hearing Dr. Bond about this increased curiosity and more targeted questions that you're hearing from clinicians and leaders alike over just the last two years, really. But as you look ahead, when you consider where clinical AI may be heading and even just three to five years from now, is there anything on your mind that we haven't yet touched on that you wanted to highlight or any final thoughts you'd like to leave our listeners with? Sure. Probably two things. I think first, I can't underscore enough. I said it probably throughout this interview, but right now we're foundation building again. And making sure that we get this foundation right is just mission critical for us because I think it really will be the foundation for the healthcare information that we use in EHRs, in ambient systems, and for many, many, many things to grow. So don't rush into it. Make sure that foundation is worthy of the decisions that it's going to support. That's number one. And number two is the whole reason to do that is that this opportunity for health care is absolutely enormous, right? When you think about the power of gender to me, it's not really to summarize and retrieve things. It really is to transform things so that people can understand them better. And throughout my career in medicine and 22 years, it really is the most important thing is to make sure that your patients understand, right? Every patient journey is unique. And I remember starting in medicine, it was an entirely different field. You used to take care of your patient in the outpatient setting, in the inpatient setting. Sometimes you would meet them in the emergency department and walk them up to their room. And you knew who they were holistically as a person. so you knew how to connect with them or how to communicate with them. And in those little moments, you might tell them what would happen next. And that was such a huge part of taking care of a person. And now as medicine has changed and grown, someone else does those things. Someone else takes a patient to the ED. Someone else walks them from point A to point B. And that ability to explain is lost in some ways. or that moment-to-moment care where you're helping someone understand what is next is loss. And I think the really enormous opportunity that we have for healthcare workflow is to bring that back in, right? So folks these days talk about workflow like it's an efficiency thing or like workflow is a destination and everything in it is fixed. But I think the worst thing we could do for healthcare in many ways is to stuff more information into that old workflow, right? That workflow was designed for the past. And if I'm to sort of re-envision a new workflow right at the center of it is probably using this technology so that other people can understand what's happening to them again and for its transformative potential. So the reason we invest so much in our foundation building is so that we can realize that, right? So that we can take what is trusted information, so that we can take information that we know doesn't have any unsupported or crazy leaps in it and then change it, right? So that someone else can understand in a way that is helpful for them. And to me, that's sort of the ultimate. You know, when you think about medical information, it's sort of locked in writing. And that's a really elite form of communication that takes a lot of years, right? To understand and for medical information, even more so, right? So if we can use our brains and our expertise and everything we learned to sort of unlock that, I think the future is, is bright. Well, Dr. Bond, it's been such a pleasure talking to you. Your passion for your work comes, comes through so clearly just as we're speaking to one another. So I want to thank you for, for bringing that to our conversation and also helping our listeners see what is possible when that foundation is gotten right from the beginning. So again, appreciate your time today and sharing your thought leadership with us. Very much appreciated. Thank you. Always a pleasure to be here. Wonderful questions. And I could talk about this forever. So thank you. Maybe we'll do a part two. Well, thank you again, Dr. Bond. And of course, we also want to thank our podcast sponsor today, Wolters Kluwer. Listeners, be sure to tune into more podcasts from Becker's Healthcare by visiting our podcast page at beckershospitalreview.com. Thank you.