Overcoming Skepticism and Driving AI Adoption - with Umesh Rustogi of Microsoft

Welcome everyone to the AI in Business Podcast. Today's guest is Umesh Rustogi, General Manager of Dragon for Nursing at Microsoft Health and Life Sciences. Umesh joins Matthew Demello, Editorial Director at Emerj, to explain why many healthcare AI initiatives fail to scale and what leading organizations are doing differently to achieve real adoption, reducing nurse documentation burden, improving accuracy and compliance, and turning AI investments into measurable operational and patient care gains. But first, are you driving AI transformation at your organization, or maybe you're guiding critical decisions on AI investments, strategy, or deployment?

If so, the AI in Business Podcast wants to hear from you. Each year, Emerj Artificial Intelligence Research features hundreds of executive thought leaders. Everyone from the CIO of Goldman Sachs to the head of AI at Raytheon and AI pioneers like Joshua Benjo, with nearly a million annual listeners, AI in business is the go-to destination for enterprise leaders navigating real-world AI adoption. You don't need to be an engineer or a technical expert. If you're involved in AI implementation, decision making, or strategy within your company, this is your opportunity to share your insights with a global audience of your peers.

If you believe you can help other leaders move the needle on AI ROI, visit Emerj dot com and fill out our thought leader submission form. That's Emerj dot com, click on Be an Expert. You can also click the link in the description of today's show on your preferred podcast platform. That's Emerj dot com slash expert one. Again, that's E-M-E-R-J dot com slash expert one. Without further ado, here's our conversation with Matthew and Umesh. Listen to hear Umesh's proven approach to turning AI pilots into real adoption and measurable ROI in clinical workflows.

Umesh, welcome to the program. It's a great pleasure to have

Thank you, Matthew. Really excited to be back again.

Absolutely. We're seeing AI tools move from pilot projects to everyday clinical use. Healthcare leaders face new questions about trust, interoperability, and culture change. We're seeing Microsoft's partnerships provide a model for scaling innovation that serves both clinicians and patients, and a big reason why we wanted to have you on the show to talk about this with a much closer view of what's going on. What approaches are you seeing return results where health systems start weaving AI into everyday nursing workflows?

And how does partnering closely with clinicians help make that transition smoother?

That's a great question, Matthew. Our early customers adopting Dragon Copilot for nurses actually reported outcomes that include reduction in documentation time, reduction in documentation latency, improvement in the quality and completeness of documentation, uh, like, for example, capturing of the invisible care and reduction in incremental overtime, and most importantly, reduction in cognitive load for nurses, right? So here are some of the approaches that enable that wide adoption and the uh all the outcomes that I just talked about.

So I would say, uh, I mean, first and foremost is the fact that we co-created Dragon Copilot for nurses with nurses, right? So we recognized very early on that the nursing workflows are unique, they are different than the physician workflows. It's a very mobile, fast-paced, uh, environment, structured documentation heavy. So we had to build a solution which was purpose-built for nurses versus just repurposing the technology that was used for physicians as an example, right? So from day one, uh, we worked with the frontline nurses, the nurse informatics team, the nurse leaders to make sure uh that the uh functionality, uh that we are building and deploying in Microsoft Dragon Copilot for nurses was actually suitable for nurses or fine-tuned for

nurses. I would say the second uh approach uh that worked is our uh focus on usability and the direct integration into the existing EHR workflows, right? Which allowed nurses to document care simply by just speaking aloud. Wi- uh about their patient interaction, and then AI then transforms these conversations into

structured flow sheet documentation, which nurses can quickly review and approve before it enters into the EHR, right? So we were laser focused uh on that uh usability and making sure that uh there was no friction in the workflow. Third, I would say, is our little bit obsession about improving accuracy, right? So it was evident that the accuracy, if the accuracy of the solution was not good enough, then nurses will have to spend lot more time editing or correcting the results versus just reviewing and accepting it, right?

And we would not see the adoption if the accuracy was not good. So over a course of time, I mean, during the early previews, we worked very hard on making sure that the accuracy improves. And the nurses, uh, basically now, uh, have the output of the system, which basically works for them. Uh, fourth uh thing that jumps out, I would say, is change management, right? So organizations that see uh strong adoption, uh, they treat uh, I mean, AI as a change management journey, not just a one-time thing.

So nurse leaders, the informatics team, uh, they, for example, would use uh adoption analytics available through our Dragon Admin Center or work with our adoption consultants to track adoption trends within their uh nurse population. How are the nurses using it? Uh, who's using, who's not, what are the friction points? And then based on that, they would uh adjust the solution if needed, or tailor the training, uh, if required. And then lastly, I would say, I mean, all this uh is kind of strengthened by, I mean, Microsoft's uh broader focus on interoperability, our emphasis on responsible AI, uh, ensuring safety and trust, transparency, explai- explainability, all these things are top of mind for us.

And then, I mean, uh, last but not the least, ensure that we are always compliant with the healthcare regulations. Absolutely. And like like the profession very professional tech leader that you are, you you stayed focused in that last answer about what the capabilities are that we're driving in these spaces and what what the outcomes are. Always the f- always the first uh questions, always the first answers. Underneath that, of course, we know, is the human element.

How are we the change management element? How are we dealing culturally as an organization, uh, with these shifts? What approaches are you seeing best support the culture shift required for AI to become part of day-to-day nursing practice? And if we can, uh, maybe put a definition on on or or articulate what that culture shift looks like.

Absolutely. Uh, and another excellent question, uh, Matthew. So how to make sure that there is actually widespread adoption of this AI technology and it gets embedded into the day-to-day nursing practice at these organizations? I believe it requires two things, right? One, of course, was, uh, a technology that is built with a deep understanding of the end-user nursing workflows, right? We just talked about all the different things, but then it also requires a very sustained organization, uh, change management effort, right?

It's much more than just handing over the tools, uh, as this technology does require new ways of doing things for the end users. So we learned a lot about what some of these good change management practices are, uh, through our early collaboration with many of these leading healthcare organizations like Mercy, like Advocate Health and so on. Uh, we also found out that individual nurses can have variations in their workflows depending on their roles, whether they are a CNA, PCT, or a registered nurse, and different nurses can have different level of comfort level with technology, right?

So, uh, focusing on, uh, some kind of a edu- education curriculum that helps each and everyone learn this technology and integrate into their daily practice is important. So the healthcare organizations that did it well actually provided support for education. They provided like protected education time to enable these nurses to spend some time and then simulate and learn, right? Because this is something new. They encouraged care out loud practices that okay as you are conversing with the patient care out loud about okay what you are seeing what you are observing and so on.

Uh and then uh uh this uh simulation uh time or this uh practice time really helped get the users get past that initial uh hesitation and inhibition that they had and get them start using the solution. The another thing which also helped uh within all these organizations is sharing the success stories from one another, right? So it's not just about throwing out the technology to the users and let them use it, but kind of engaging with these users on an ongoing basis, right?

Creating local champions within individual nursing units, right? So that the users can come together, they can share each other's experiences, uh, what worked, what didn't work, not just what didn't work, but also what worked, right? Because sharing these success stories is important. So, for example, we heard from a user at Mercy that, uh, I mean, like many of the nurses said that by narrating their care, by caring out loud and having that care being pulled into, uh, the EHR as a structured documentation was great for that because it improved their quality of their documentation, right?

And as they talked about this with other fellow nurse users, other users got the confidence of okay, hey, yes, this technology works, and this is the right way to use it. So they learn from each other. Uh, and then I would say that, uh, I mean, ultimately, the uh, organizations that thrive with adoption, they basically, I mean, treat AI deployment not as a one-time thing, but as a cultural evolution, right? Kind of rooted in continuous support and trust and transparency.

Absolutely. Uh, you you had mentioned the individual element in that last answer, and something that the this audience knows for uh absolutely is that, uh, one of the benefits of AI, or one of the distinct capabilities of even determin a lot of deterministic AI is is a sense of personalization that you can build into these systems on the back end. And something we continually forget, especially in talking about patient or for other industries, customer experience workflows, is that works both ways.

Of you can personalize the experience for the patient, for the customer, also for the human agent on the employ- employer side of the line, or the the the business side of the line, the organization side of the line, to make it their job easier. Wondering how you're thinking about balancing that personalization, maybe for, uh, the nurses? Where where's that appropriate versus, you know, where you wanna educate human beings to best fit into this overall system where ultimately patient experiences are are the main metric.

No, so that's a that's a great question again. And and I think, as you rightly pointed out, all these systems have a certain deg- I would say, a balance between what is best practice or this is how the solution ought to be used. Second is every organization will have some differences in their workflows and so on. So it's almost like configured for that specific organization. And then the third level, as you said, is the actual user-level personalization, where, beyond what the organization mandates, there are bells and whistles that the end user can tweak because this is how they like to best use the system.

A very naive or a simple example of this could be a certain group of users basically want the recording to start immediately the moment they select the patient in the application versus somebody who would say, no, no, no, I select the patient, but then I explicitly say start recording, right? So this is a very naive or simple example of how user personalization is important in applica- in or or these kind of capabilities are important in solutions like these to make sure that we can balance the needs of an individual with the needs of an organization.

Absolutely, absolutely. You you've been mentioning throughout the program so far the the partnerships with health systems. Tell us about those partnerships and what they revealed about what it takes to bring AI into different clinical environments in a way that works for staff and fits real workflows.

Yeah, so I mean, when we started, uh, this whole initiative, we basically started working with about ten of these leading healthcare organizations. And I would say a few things, uh, stand out from that, uh, from their engagements, right? So, one was it was very clear, uh, that a solution like these need to be able to adapt to unique institutional workflows of that healthcare organization. So, um, identifying these, uh, admin owners early, uh, kind of validating integration with their EHR systems, uh, confirming that, uh, what kind of, uh, unique flow sheet schemas their organizations have, what kind of taxonomy they have, because these are differences across one organization to another.

And then, uh, ensuring that the AI continues to work at high accuracy with all those unique requirements across these different organizations, this was important, and this led us to build various kind of self-service AI assessment or AI customization tools that the nursing informatics team at these organizations could basically use to tweak the system as needed, right? So, the need for, uh, incorporating or supporting these unique institutional workflows, I would say, was one of the things which stood out.

Another thing is the value in validating whatever we are rolling out, uh, in a kind of a diverse care setting before we scale, right? So, the our most, I mean, while we were doing these implementations with with various organizations and then designing these workflows with nursing and nursing informatics team, we learned from them, incorporated it before we said, okay, now the solution is generally available and roll it out to other customers. And we will continue to do that, right, as we expand the solution into new nursing service lines, uh, as we expand the solutions into new regions, as we expand the solution into new kind of AI, uh, AI actions, uh, and experiences, we would always validate it in a

more controlled setting before we scale. And that's why, uh, I mean, all the engagements that we have with these customers is more like long-standing collaboration versus a one one-off. Another thing I would say is, uh, the responsible AI feedback, right? So, how Microsoft integrates feedback loop from this real world usage of the solution into ongoing product evolution and improvement. We have capabilities within the solution where, I mean, the end users, the nurses, can actually say, okay, hey, this is I was expecting the output to be this, but this is what I got.

And then behind the scene, we have, uh, capabilities, including a clinical integrity team, which would actually review this feedback and say if something needs to be tweaked within the solution to make this better, right? So, this ongoing feedback loop, uh, is super important. And then I think the last, uh, thing I wanted to call out was these structured change management models, right, which cover the onboarding, education, continuous improvement that helps organizations see, uh, stronger adoption curves.

So, learning from these partnerships, learning what works, what doesn't work, capturing it as best practices in change management, and then sharing it out with our new customers as best practices. So, all these things help us not only improve the product, but also build these change management best practices that help organizations make sure that there there is a widespread adoption, uh, of these technologies within their organizations.

Absolutely. And and create that learning loop as as it gets called. Yeah, absolutely. I think maybe where the rubber's hitting the the road there just in terms of challenges for leaders. And I know you've brought up accuracy throughout this entire conversation. But maybe while especially while we've got you here for the next couple of minutes, it it m might be worth kind of pulling apart, especially when you're looking at metrics or you places where you wanna drive accuracy.

Of course, everybody wants the impossible 100% and or or the the proverbial nineteen, uh, you know, 99.9, nine nine. But of course, accuracy and ceilings don't necessarily go that way based on the workflow, based on, uh, on what you're trying to accomplish, et cetera. Any advice, uh, for leaders, maybe where we don't know what a a reasonable accuracy ceiling might be in a particular area about how to go about that back and forth process of driving that accuracy and and accepting, you know, a ceiling where you can detect it for for what it is or or what maybe the standard for evidence of that accuracy ceiling might be.

That's a that's a great question. So I would say it's a it's a combination of these different aspects, uh, Matthew. So, number one is giving the tools to these organizations, right, uh, that they can use to tune accuracy, right? So, for example, uh, many of these flow sheet schemas often have certain, I would say, they're not amenable, uh, or they have, uh, certain, if I may use the term, ambiguity as to which is the right choice.

Because these flow sheet schemas in these organizations have evolved over years, right? So we provide tools that can help the organization identify where some of these challenges, potential challenges may be that they can use to then correct their flow sheet schema or enhance their flow sheet schemas if needed. Uh, the second, uh, I would say, is, uh, again, in the change management, right? So I was talking about care out loud practices. Obviously, the AI can only capture things that were talked about.

It cannot capture things which were not talked about, right? So how do you ensure that more and more portion of your flow sheet schema is actually captured in the by by AI would basically require that you practice those care out loud practices. And then the, uh, last thing, uh, I would say, is this responsible AI feedback loop that I was talking about, uh, things that may not work today, right? I mean, as part of the continuous innovation, we continue to work on the improving the accuracy of those things so that they work in future.

A lot of really gr- strong advice there, I know, our our executive audience is going to take home. Something we've been mentioning to this audience a lot, especially over the last three years since the generative AI explosion, is that we're kind of in this fog of adoption, I I more or less, uh, call it, where at the beginning of the mass adoption process, we don't w we have a strong idea of maybe what the finish line looks like or what life will be like once, uh, as I like to say, calling back to the era of internet adoption, what it looks like when the whole neighborhood has Wi-Fi, so to speak.

But i it I think we're coming out of a sort of a fog of adoption right now in the healthcare and AI space. I can speak e- especially for content creation here at Emerj, that has looked a lot different than what we planned on. We we are finding that thanks to LLMs, the product is better, but not e- every, uh, workflow is necessarily taking a shorter amount of time. In fact, a lot of workflows are taking longer because you have these more advanced tools in places that are giving you more information and more accuracy.

And you need to as a human being, you know, a carbon-based life life form reading these things with your eyes, you need a little bit more time to really, uh, process that information. And that's not necessarily w- what the purveyors of this technology were saying this this how this hype cycle would go when they start. Same thing, uh, same thing happened with cloud back when we had the hype cycle for that around around ten years ago.

I'm I'm interested in what this project tells you as someone really driving it about the next phase of AI in healthcare beyond nursing, uh, and what leadership mindset is really necessary to guide teams through that shift.

So, if you look at, I mean, Microsoft broader vision in AI, I I would say we are basically looking to harness the power of AI to empower clinicians and nurses, improve the patient experiences, improve health outcomes, and in general, advance health equity, right? So, that's the the broader vision. And I believe in the next phase of AI in healthcare, we'll we'll basically start seeing an evolution from task automation to more agentic intelligence that collaborates with clinicians, right?

Coordinates care, and then kind of helps accelerate breakthrough broadly within the entire, uh, healthy ecosystem. I believe the next chapter will basically be defined by this agent and humans working together, teaming up, where, uh, whether it's clinician, researchers, uh, operational staffs, they're supported by these agents to amplify their impact, right? Reduce the burnout and deliver better health outcomes of whatever their roles may be. So, we will start, uh, seeing more and more of these. I mean, one such one such example could be, let's say, somebody like a nurse today ends up spending a lot of time in, uh, once a patient has been discharged, right, following up with the patient, whether the patient is adhering to the schedule medicine schedule

that was recommended at discharge. And it's important to make sure that they adhere to that, uh, schedule so that they don't actually have a relapse and then, uh, readmitted, right? So, something like this post-discharge medical adherence follow-up, I mean, could the agents actually help the nurse in doing some of these follow-ups? So, these are some examples, uh, where we will basically see the nurse and the agents kind of team up to create more productivity and more outcome and more throughput for these, uh, clinical staff, as compared to what we are able to see today, right?

So so I think, uh, I mean, in general, as I said, so our vision, uh, is to, uh, empower clinician and nurses and then improve this patient experiences and the health outcomes. What kind of a leadership mindset is necessary to guide the teams through this shift, right? So I think this technology will keep on advancing, keep on evolving, right? And but this transformation and change ultimately comes down to people. So we believe that organizations will succeed when their leadership at these organizations will will invest in continuous learning within their orgs and then foster this environment of innovation and experimentation.

It's important. I mean, not every new technology will work necessarily the first time, right? So there has to be that mindset to, uh, to experiment with it, right? Prove it out and then scale it. And then support the various, uh, clinician and nurses through each of these changes. Because just like the example that we were talking about with the current rollout of Dragon Copilot for nurses, all these future technologies will require some kind of, uh, change management on part of those users and organizations as well.

Absolutely. And and just for the the larger umbrella answer to the question, which is we're seeing, uh, a larger shift from tax automation, uh, from task automation. Tax automation is different, forgive me. That's that's my last job in podcast talking. Task automation to agentic, and I think that's that's very much a chorus we're hearing in a lot of different industries right now. And as we make that shift, you know, across the global economy, uh, insights like these are gonna be so important.

So we'll have to have you back as that shift gets closer and closer on the horizon. Umesh, thank you so much for being with us on today's show. It's been a great pleasure.

Thanks a lot, Matthew.

Wrapping up today's episode, I think there are three key takeaways for healthcare and enterprise leaders from our conversation with Umesh, General Manager of Dragon for Nursing. First, AI adoption succeeds when technology is purpose-built for end-user workflows, not just deployed as a generic tool. Second, measurable ROI requires integrating AI into daily operations with strong change management, training, and feedback loops. Finally, scaling AI beyond pilots depends on balancing organizational standards with user-level personalization to build trust, accuracy, and sustained adoption.

Interested in putting your AI product in front of household names in the Fortune 500? Connect directly with enterprise leaders at market-leading companies. Emerj can position your brand where enterprise decision-makers turn for insight, research, and guidance. Visit Emerj dot com slash sponsor for more information. Again, that's E-M-E-R-J dot com slash S-P-O-N-S-O-R. If you've enjoyed or benefited from the insights of today's episode, consider leaving us a review on Apple Podcasts and let us know what you learned, found helpful, or liked most about this show.

Also, don't forget to follow us on X, formerly known as Twitter, at Emerj, and that's spelled E-M-E-R-J, as well as our LinkedIn page. I'm your host, Emily Smith, producer here at Emerj AI Research, on behalf of Daniel Fagella, our CEO and head of research, as well as the rest of the team here at Emerj, thanks so much for joining us today, and we'll catch you next time on the AI in Business podcast.