What It Actually Takes to Trust an AI System Running on Its Own - with Yogesh Bhagat of CVS Health
The AI in Business Podcast · 3 Nov 2026
Source: https://podcast.emerj.com/what-it-actually-takes-to-trust-an-ai-system-running-on-its-own-with-yogesh-bhagat-of-cvs-health
Transcript: https://podcasts.opennash.com/shows/ai-in-business/what-it-actually-takes-to-trust-an-ai-system-running-on-its-own-with-yogesh-bhag-cff0bc/ (via OpenNash Podcasts, machine-generated)

Welcome everyone to the Emerge AI in Business podcast. Today's guest is Yogesh Bhagat, Vice President of Digital and Technology at CVS Health. Yogesh joins us to discuss what it takes for customer-facing A I agents in healthcare to move from pilots to autonomous operations, and why no agent should launch with autonomy. It has to earn it by giving consistent answers, however, a question is asked. Just a quick note for our audience that the views and opinions expressed by Yogesh on today's program are his own and do not reflect those of CVS Health or its leadership.

If you offer A I products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title, at the right type of organization, and, of course, readiness. Emerge attracts V P and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional I T. To learn the exact strategies we use to help leading A I brands and startups connect with their ideal enterprise A I buyers, visit emerj.com/AD1.

That's E M E R J dot com slash A D one. Now, the conversation with Yogesh. Yogesh, welcome to the show.

Good morning, Shalina. Happy to be here. So excited to talk about my experience about A I.

Yeah, and I think today is going to be a practical conversation, and we're going to start it quite practical as well. Because I think so many people will have some kind of phone story, I V R story, right? You phone in and it says, press one for this and press two for that. And eventually you're shouting at the phone, saying, I want to speak to a human. And we've seen teams switch out things to replace it.

We have seen. So many teams having to go through these processes and some succeed, some fail. And this is something that you've been through more than once now. So, when you're building A I that sits right in front of customers, where does the real friction show up for us before that system even goes live?

That's an interesting and loaded question. Friction has a different meaning for different kind of industries. For me, in healthcare, friction is not about a user experience issue just. It is about access to care, affordability, and patient safety. Now, coming to the point of friction before even something goes live, I think there are two fundamental things which we need, where we face friction. One is governance model, and second is run cost. So let's take a simple example of pharmacy management system.

We receive a prescription, we edit it, we go through a verification process, fill the bottle, dispense it to the customer. Now, this involves hundreds of tasks. Every task has a different level of a complexity. But when you are trying to build an A I agent around it, the level of governance doesn't change. You need people from security, you need biasness validation, you need to have observability, then you need to make sure that answers are consistent. It's more like a deterministic, not the probabilistic approach.

So all those things creates a governance model. And the natural tendency for us is, hey, let's bring experts from every field, and every agent which is going to go live is going to be go through this governance cycle. That is called governance by process and which will slow down and which will create friction for you. What we have done is, hey, we don't need our engineers to learn how to build observability. We don't want every engineer to focus on security.

We don't want every engineer to focus on guardrails. First thing we said is we need to go governance by rigor. And, by what it means, we actually build an enterprise A I development platform, which brings security, guardrails and observability by default for creating any agent. Now, pretty much you have taken lot of tasks which humans would have done, being done by some automated process and it is validated and it is tested. So you validate the platform, not every agent which goes.

So that's the first step of friction, which you can remove by having the right tools and building the right processes around it. Now, second step is if you look at these processes in any system, complexity we talked about. Few complexities like reading a prescription from a fax might be easy to automate, create an autonomous agent. Whereas agents where you're going to touch patient's eligibility, patient copay amount, or dosage, those are difficult because that directly impacts health.

So what we decided is we are going to go in a very different approach. Nobody will have a right to launch an autonomous agent. You have to launch an agent and earn the respect. How do you earn the respect? You need to demonstrate that every action you are taking or every response you are giving is consistent, even if asked in a different way. And you need to have a way to measure that. Once you are able to do that, you have proven that you are no more a probabilistic agent, you can run autonomous.

So that's our guardrail. I'll give an example like T O B check. You can do it once we validate that it is always doing it. We remove the larger model out of the context, judge model out of a context, we remove human in the loop, and we know that it is always working fine. So these are the small things which runs in governance. Second, I talked about a run cost. Let's understand. All of us build any use case for having a R O I work for us.

I'll say everyone tells me M I T has published a report where 95% of A I project fails. But that's a theory. How do you attack that problem? You need to think while solving any business problem, there are things you are going to automate, you are going to run safety check with models, you are going to run judge models for result validation. So it is going to cost you. If you don't think about that cost upfront, you can't solve it.

And how we did it is when we looked at our I V R calls, which is million plus call every day. How do we solve it? We hosted twenty five S L M s in our data center, working in conjunction with the L L M s. What it helped us, it reduced our cost by five to six x. In addition, it also helped us to meet the sub second response time, which any I V R system would require.

So I think these two are the main friction points before you even run the things. And if you don't care about it, you will might move fast, but that fast will have a roadblock very soon.

That makes sense. And thinking about, you know, with the governance, you said that testing is going to earn us the right to automate. But how much of the friction comes from testing? Because, like you mentioned, not everyone's going to ask the same question in the same way. The date of birth question is a great example. If you ask me for date of birth, some people will answer it date, month, year, others will say month, date, year.

So is there a lot of friction that comes in with that testing?

It's a great question, and I say, I think testing itself, or while you're building an agent, no longer a testing is a traditional way. Nothing fails in the same way. If you go in a traditional software, it fails in the same way every time. But agents doesn't fail same way. It's a probabilistic system, you can get a failure from multiple points. So what you need to do is build an agentic A I testing approach as part of your development life cycle itself.

What we have done is every agent we build, our developer platform also builds you an agent to test it. I'll give you an example. We say, hey, agent, you are an O A P S person, old age person, and you have X number of basically healthcare issues. And you are asking a question to an I V R. What kind of a questions will you ask? And when will you get frustrated? How will you show your frustration?

So agent is generating those test cases and we solve that problem before it goes into a production. Second step is about judge models. What you do is your main models, which are generally a smaller models, are giving an answer. And then you are cross validating the output just by a larger models. So as you see, as you see maturity, these judge models will go out of life because your smaller models would have learned everything. That's how I think you get reduce the testing effort and overall, you earn the right.

Wow, so, and that was exactly what I was gonna ask next, because I want us to get into the point of, okay, you know, we've tasted, it's taken real life calls, um, you know, how do we know whether it works the way we should? And and you've already started answering that. And when we're saying the big models and the smaller models, and, you know, taking out the large ones and then leaving the small are we also thinking cost there?

I'm assuming that bigger models mean more money.

Frontier models, if I would have run frontier model, then I would have not been able to make it live. It would have only be a pilot project, would have never gone live. But I think in case of I V R, it is beyond cost. Because in I V R, as we are talking even if I don't speak for another five seconds, you will feel like network is not working and you will hang up on me. So you need a sub-second response time, and that sub-second response time is where smaller models comes.

Even sometimes you will replace few things with M L models. So that's a learning which you built and you remove use larger model only for reinforcement learning or feedback loop. So how we do is everything we do have two reviews. One is a judge model and second is a human in loop, which is an escalation path for us. These two reviewers do not just act as a security check for us. They act as a feedback loops so that we can train our smaller models constantly and it is improving constantly.

This is where you are measuring their deterministic nature. And that's how, step by step, you'll remove larger models into the real from real environment, and you'll only use later on for reinforcement learning.

How do we know that the the caller is not feeling that switch to a larger model? Is that something that happens during the call, or is it something that we sort out way before we get to that point?

So there is a clear escalation path which you need to build. And like example, I'll tell you, we have caller path. Way to know that if there is combination of frustration, combination of a repeat of a question, and different scenarios, which led lower model know that if the question is getting repeated twice, then I need to escalate to a larger model and validate my response. And if that also says it's not what response is right, then there is an escalation path to a human in loop.

Biggest thing here is you need to make sure that there is no biasness while you are doing it. Because this is about access to care. So it's not just about escalation path, but also about ensuring there is no biasness in this escalation path.

Yeah, no, absolutely. And when we're saying there needs to be a human in the loop and there's a feedback loop going back to the human, who's the human in charge of? Is it going to be engineers? Is it the contact center floor? Is it compliance? Who gets to be involved in that?

Human in loop is the agents who are finally picking up the customer calls. What questions were asked from them? What were their responses? Why were we not able to answer? And that recording also goes as a feedback loop. So in this case, it's a real human who is solving final escalation path.

Great. So it goes back to, I would say, contact center flows, and I guess then certain scenarios would be used for training purposes as well. And giving feedback to the engineers as well as to where we need to maybe up governance or change the way it interacts. Am I getting the right idea?

Absolutely, because once you take that transaction, you put it into a reinforcement model of models. Engineers look at it. Do they really need to change the rag, or do they need to change some kind of a prompt? What is the real need? And that's where engineer comes into a picture.

Uh, makes a lot of sense. So we're getting to the part of the conversation where we get to tell our leaders thinking about replacing I V Rs and thinking about this entire process. They need to know where to start, they need to know what to look out for. So, for a leader about to hand a major customer-facing process to A I for the very first time, what would you let them or want them to have in place before they trust it to run on its own?

What do they need to get to make sure that it they earn the right of automation?

I'll say first, we already talked enterprise A I development platform, which allows all engineers, your average engineer to best engineer, to build A I agents in one consistent way. Without that's a table sticks, you need to build that. Okay, second, within that platform, you also need to build a testing approach. Deterministic testing approach doesn't work. We already talked about it. I will not repeat it. Third is ability to replay the transaction or observability. That's a crucial piece.

When you are talking with millions of customers, any one escalation comes. You need to know out of hundred agent, which agent answered, which particular task was deterministic in nature, where the path went wrong. And if you don't know how to do that, and it takes many, many hours to just find out one use case, then you are not ready for a go live. So you should be able to find out gap in your process in seconds, not in even minutes.

So that's a key piece. Then you need to know customer experience. Are you working for customer facing system where it is a sub-millisecond responses required, like example, I V R versus a chatbot? Chatbot is a place where you can get two seconds. Because you can say I'm triangulating, I'm doing this. People are so used to chat G P T Claude that they take they are okay with the time. So if you know that, then you need to pick right models which can answer or meet your things.

Or you can have engineering solutions like async way of working or whatever you like. There are different ways to solve that problem. Then economics. I will stress this part a lot of time, because everybody is tilted. I will apply one Claude and have open A I one model frontier model and the problem will be solved. But it will not be solved in an economical way. That's a crucial key to success. If you have not solved about that problem, then you are going to face issue after pilot.

So that's a core, I believe.

Would you advise our listeners to start with their main business problem or their main friction point? Or would you advise them to start on a narrower path? Like allow things like a customer phoning for store hours or just to follow up on whether a refill is already done, like start with narrower things before you expand, get the testing done on those first.

It is an interesting thing. We, it's a very debatable piece. We started the same thing. We were replacing the old legacy I V R, which used to say press one. It's a tree structure. And we said, hey, should we replace one? And then do step by step versus all at once. And what we learned is experience is so much broken between if you try to move from conversational A I to a s old I V R systems.

They will not be able to communicate in the same way. The tone which you want to set for consumers, you'll not be able to measure. So our approach was go for full blown, but go with a s smaller set of a customers. Learn from a smaller set of a customer, give them a complete experience, and then build confidence, build your learning models, build the feedback loop in that with smaller set of customers, improve your containment numbers later on.

First focus on experience, because what mistake you can get into is day, when you launch with a let's say 2% of your customers, you want the containment of that A I system to be better than the traditional I V R. That's a default every business user wants. No, that's not what you should focus on. First, focus on learning, observability, how you are going to improve the customer experience. Then automatically you will see the containment and satisfaction of a customer will increase.

So my suggestion will always go with the go small, but with full A I experience, otherwise it is difficult to change behavior of a customer.

I like that. I think before we wrap up, I think I just want to recap for our audience that I think what stood out for me was when you said no one has the right to go autonomous. We need to earn it and the testing and everything that goes on behind it is super, super important, like you've just highlighted. How do we know that it works? We need to make sure that we do have that continuous feedback loop that we are continuing to guide the agent and get it to a point where we know that we can trust it.

And then, like you just said, start small, start with a small sample of customers, take them through the entire journey, learn through that. It's part of your testing process. And that's how, like you said, how we'll gain confidence and confidence goes into trust and that's ultimately what we're paying for. Yogesh, anything that you'd like to leave our audience with?

I would leave it with one thing. Autonomy is not earned, not launch. You need to really earn it in that way. Safety layer should be so cheap that nobody get tempted to replace it. So make sure your economics play in your hand and safety is always first.

Exactly, and we'll leave it just there. Thank you so much, Yogesh.

Thank you.

Wrapping up today's episode, let's look at the three key takeaways from our conversation with Yogesh. First, autonomy should be earned rather than launched, with agents running on their own only after they give consistent, measurable responses, however, a question is asked. Second, governance scales when security, guardrails, and observability are validated once at a platform level rather than reviewed agent by agent. And finally, run costs have to be engineered from day one, because pairing smaller models with larger judge models and human escalation keeps safety affordable and responses fast enough for the customer-facing channels.

If you offer A I products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title, at the right type of organization, and, of course, readiness. Emerge attracts V P and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional I T. To learn the exact strategies we use to help leading A I brands and startups connect with their ideal enterprise A I buyers, visit emerj.com/AD1.

That's E M E R J dot com slash A D one. For further executive level analysis and to join our network of leaders delivering workflow impact with A I, visit emerge dot com. On behalf of the team at Emerge, we'll see you on the next episode.