# What It Actually Takes to Trust an AI System Running on Its Own - with Yogesh Bhagat of CVS Health

- **Show:** The AI in Business Podcast
- **Published:** 3 Nov 2026
- **Source:** https://podcast.emerj.com/what-it-actually-takes-to-trust-an-ai-system-running-on-its-own-with-yogesh-bhagat-of-cvs-health
- **Transcript:** https://podcasts.opennash.com/shows/ai-in-business/what-it-actually-takes-to-trust-an-ai-system-running-on-its-own-with-yogesh-bhag-cff0bc/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Yogesh Bhagat of CVS Health argues that a customer-facing AI agent should never launch with autonomy. It has to earn that by giving consistent, measurable answers however a question is phrased. Governance should be applied once to a shared development platform rather than agent by agent, and run costs must be planned from the start so that safety stays affordable.

The main tension is between moving fast and staying safe in healthcare, where errors affect access to care. His answer is layered. Small models handle most calls quickly and cheaply, larger judge models and human agents check the results, and those checks feed back into training. He also advises starting with a small group of customers who get the full AI experience, instead of chasing containment numbers early.

## Key insights

1. **Validate the platform, not every agent** CVS built an enterprise AI development platform that supplies security, guardrails and observability by default. Because the platform is validated once, individual agents skip a separate round of expert reviews, which Bhagat calls governance by rigor rather than by process.
2. **Autonomy is earned through consistent answers** An agent must show it gives the same answer however the question is asked, and teams need a way to measure that. Once it does, the larger judge model and the human in the loop can be removed for that task, such as a date-of-birth check.
3. **Small models cut IVR costs five to six times** For its IVR, which handles over a million calls a day, CVS hosted 25 small language models in its own data center alongside larger models. This cut costs five to six times and met the sub-second response time that phone callers expect.
4. **AI agents should be tested by other agents** Traditional software fails the same way each time, but agents fail in many ways. CVS's platform builds a test agent that plays, for example, an older patient with health issues, and it generates questions and frustration to find failures before launch.
5. **Start small, but give the full AI experience** Replacing an old phone menu step by step breaks the experience, so Bhagat favors launching the full conversational system to a small share of customers, such as 2%. He says to focus first on learning and observability, since containment and satisfaction should then improve.

## Questions this episode answers

### Why shouldn't a company let an AI agent run on its own from day one?

Bhagat says autonomy has to be earned. In healthcare, mistakes touch patient safety, so an agent should first prove it gives consistent answers however a question is asked, and only then lose its human and model oversight.

### What does 'governance by rigor' mean, and how is it different from governance by process?

Governance by process sends every new agent through reviews by security, bias and other experts, which slows everything down. Governance by rigor builds security, guardrails and observability into a shared platform, so only the platform needs validating.

### Why use small AI models instead of the biggest ones for phone calls?

Callers hang up if a phone system pauses for several seconds, so it needs sub-second responses, and small models are faster. Bhagat also says running frontier models would have kept the project at pilot stage because of cost; the small models cut costs five to six times.

### What is a judge model, and when does a human get involved?

A judge model is a larger AI model that checks the answers a smaller model gives. If the caller seems frustrated or repeats a question, the call escalates to a larger model, and if that fails it goes to a human agent, whose handling is fed back into training.

### How should a business start when replacing a phone menu system with AI?

Bhagat recommends launching the complete AI experience to a small group of customers instead of replacing menu options one at a time. Focus first on learning and customer experience, and containment (calls resolved without a human) should improve afterward.

## Facts

- **Yogesh Bhagat** [0:06:50]: "We hosted twenty five S L M s in our data center, working in conjunction with the L L M s. What it helped us, it reduced our cost by five to six x."
- **Yogesh Bhagat** [0:06:50]: "when we looked at our I V R calls, which is million plus call every day."
- **Yogesh Bhagat** [0:06:50]: "In addition, it also helped us to meet the sub second response time, which any I V R system would require."
- **Yogesh Bhagat** [0:04:04]: "we actually build an enterprise A I development platform, which brings security, guardrails and observability by default for creating any agent."
- **Yogesh Bhagat** [0:04:04]: "So you validate the platform, not every agent which goes."
- **Yogesh Bhagat** [0:05:13]: "Nobody will have a right to launch an autonomous agent. You have to launch an agent and earn the respect."
- **Yogesh Bhagat** [0:05:49]: "We remove the larger model out of the context, judge model out of a context, we remove human in the loop, and we know that it is always working fine."
- **Yogesh Bhagat** [0:06:18]: "I'll say everyone tells me M I T has published a report where 95% of A I project fails."
- **Yogesh Bhagat** [0:08:23]: "What we have done is every agent we build, our developer platform also builds you an agent to test it."
- **Yogesh Bhagat** [0:08:23]: "We say, hey, agent, you are an O A P S person, old age person, and you have X number of basically healthcare issues."
- **Yogesh Bhagat** [0:07:53]: "Nothing fails in the same way. If you go in a traditional software, it fails in the same way every time."
- **Yogesh Bhagat** [0:08:50]: "your main models, which are generally a smaller models, are giving an answer. And then you are cross validating the output just by a larger models."
- **Yogesh Bhagat** [0:08:50]: "As you see maturity, these judge models will go out of life because your smaller models would have learned everything."
- **Yogesh Bhagat** [0:11:09]: "if the question is getting repeated twice, then I need to escalate to a larger model and validate my response."
- **Yogesh Bhagat** [0:12:13]: "Human in loop is the agents who are finally picking up the customer calls."
- **Yogesh Bhagat** [0:14:38]: "Chatbot is a place where you can get two seconds."
- **Yogesh Bhagat** [0:14:09]: "You need to know out of hundred agent, which agent answered, which particular task was deterministic in nature, where the path went wrong."
- **Yogesh Bhagat** [0:16:09]: "We were replacing the old legacy I V R, which used to say press one. It's a tree structure."
- **Yogesh Bhagat** [0:17:08]: "when you launch with a let's say 2% of your customers, you want the containment of that A I system to be better than the traditional I V R."
- **Yogesh Bhagat** [0:09:46]: "Because in I V R, as we are talking even if I don't speak for another five seconds, you will feel like network is not working and you will hang up on me."
- **Yogesh Bhagat** [0:09:46]: "Frontier models, if I would have run frontier model, then I would have not been able to make it live. It would have only be a pilot project, would have never gone live."

## Transcript

**[0:00:12] Shalina:** Welcome everyone to the Emerge AI in Business podcast. Today's guest is Yogesh Bhagat, Vice President of Digital and Technology at CVS Health. Yogesh joins us to discuss what it takes for customer-facing A I agents in healthcare to move from pilots to autonomous operations, and why no agent should launch with autonomy. It has to earn it by giving consistent answers, however, a question is asked. Just a quick note for our audience that the views and opinions expressed by Yogesh on today's program are his own and do not reflect those of CVS Health or its leadership.

**[0:00:42] Shalina:** If you offer A I products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title, at the right type of organization, and, of course, readiness. Emerge attracts V P and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional I T. To learn the exact strategies we use to help leading A I brands and startups connect with their ideal enterprise A I buyers, visit emerj.com/AD1.

**[0:01:07] Shalina:** That's E M E R J dot com slash A D one. Now, the conversation with Yogesh. Yogesh, welcome to the show.

**[0:01:33] Yogesh Bhagat:** Good morning, Shalina. Happy to be here. So excited to talk about my experience about A I.

**[0:01:39] Shalina:** Yeah, and I think today is going to be a practical conversation, and we're going to start it quite practical as well. Because I think so many people will have some kind of phone story, I V R story, right? You phone in and it says, press one for this and press two for that. And eventually you're shouting at the phone, saying, I want to speak to a human. And we've seen teams switch out things to replace it.

**[0:02:02] Shalina:** We have seen. So many teams having to go through these processes and some succeed, some fail. And this is something that you've been through more than once now. So, when you're building A I that sits right in front of customers, where does the real friction show up for us before that system even goes live?

**[0:02:18] Yogesh Bhagat:** That's an interesting and loaded question. Friction has a different meaning for different kind of industries. For me, in healthcare, friction is not about a user experience issue just. It is about access to care, affordability, and patient safety. Now, coming to the point of friction before even something goes live, I think there are two fundamental things which we need, where we face friction. One is governance model, and second is run cost. So let's take a simple example of pharmacy management system.

**[0:02:51] Yogesh Bhagat:** We receive a prescription, we edit it, we go through a verification process, fill the bottle, dispense it to the customer. Now, this involves hundreds of tasks. Every task has a different level of a complexity. But when you are trying to build an A I agent around it, the level of governance doesn't change. You need people from security, you need biasness validation, you need to have observability, then you need to make sure that answers are consistent. It's more like a deterministic, not the probabilistic approach.

**[0:03:28] Yogesh Bhagat:** So all those things creates a governance model. And the natural tendency for us is, hey, let's bring experts from every field, and every agent which is going to go live is going to be go through this governance cycle. That is called governance by process and which will slow down and which will create friction for you. What we have done is, hey, we don't need our engineers to learn how to build observability. We don't want every engineer to focus on security.

**[0:04:04] Yogesh Bhagat:** We don't want every engineer to focus on guardrails. First thing we said is we need to go governance by rigor. And, by what it means, we actually build an enterprise A I development platform, which brings security, guardrails and observability by default for creating any agent. Now, pretty much you have taken lot of tasks which humans would have done, being done by some automated process and it is validated and it is tested. So you validate the platform, not every agent which goes.

**[0:04:39] Yogesh Bhagat:** So that's the first step of friction, which you can remove by having the right tools and building the right processes around it. Now, second step is if you look at these processes in any system, complexity we talked about. Few complexities like reading a prescription from a fax might be easy to automate, create an autonomous agent. Whereas agents where you're going to touch patient's eligibility, patient copay amount, or dosage, those are difficult because that directly impacts health.

**[0:05:13] Yogesh Bhagat:** So what we decided is we are going to go in a very different approach. Nobody will have a right to launch an autonomous agent. You have to launch an agent and earn the respect. How do you earn the respect? You need to demonstrate that every action you are taking or every response you are giving is consistent, even if asked in a different way. And you need to have a way to measure that. Once you are able to do that, you have proven that you are no more a probabilistic agent, you can run autonomous.

**[0:05:49] Yogesh Bhagat:** So that's our guardrail. I'll give an example like T O B check. You can do it once we validate that it is always doing it. We remove the larger model out of the context, judge model out of a context, we remove human in the loop, and we know that it is always working fine. So these are the small things which runs in governance. Second, I talked about a run cost. Let's understand. All of us build any use case for having a R O I work for us.

**[0:06:18] Yogesh Bhagat:** I'll say everyone tells me M I T has published a report where 95% of A I project fails. But that's a theory. How do you attack that problem? You need to think while solving any business problem, there are things you are going to automate, you are going to run safety check with models, you are going to run judge models for result validation. So it is going to cost you. If you don't think about that cost upfront, you can't solve it.

**[0:06:50] Yogesh Bhagat:** And how we did it is when we looked at our I V R calls, which is million plus call every day. How do we solve it? We hosted twenty five S L M s in our data center, working in conjunction with the L L M s. What it helped us, it reduced our cost by five to six x. In addition, it also helped us to meet the sub second response time, which any I V R system would require.

**[0:07:18] Yogesh Bhagat:** So I think these two are the main friction points before you even run the things. And if you don't care about it, you will might move fast, but that fast will have a roadblock very soon.

**[0:07:29] Shalina:** That makes sense. And thinking about, you know, with the governance, you said that testing is going to earn us the right to automate. But how much of the friction comes from testing? Because, like you mentioned, not everyone's going to ask the same question in the same way. The date of birth question is a great example. If you ask me for date of birth, some people will answer it date, month, year, others will say month, date, year.

**[0:07:51] Shalina:** So is there a lot of friction that comes in with that testing?

**[0:07:53] Yogesh Bhagat:** It's a great question, and I say, I think testing itself, or while you're building an agent, no longer a testing is a traditional way. Nothing fails in the same way. If you go in a traditional software, it fails in the same way every time. But agents doesn't fail same way. It's a probabilistic system, you can get a failure from multiple points. So what you need to do is build an agentic A I testing approach as part of your development life cycle itself.

**[0:08:23] Yogesh Bhagat:** What we have done is every agent we build, our developer platform also builds you an agent to test it. I'll give you an example. We say, hey, agent, you are an O A P S person, old age person, and you have X number of basically healthcare issues. And you are asking a question to an I V R. What kind of a questions will you ask? And when will you get frustrated? How will you show your frustration?

**[0:08:50] Yogesh Bhagat:** So agent is generating those test cases and we solve that problem before it goes into a production. Second step is about judge models. What you do is your main models, which are generally a smaller models, are giving an answer. And then you are cross validating the output just by a larger models. So as you see, as you see maturity, these judge models will go out of life because your smaller models would have learned everything. That's how I think you get reduce the testing effort and overall, you earn the right.

**[0:09:22] Shalina:** Wow, so, and that was exactly what I was gonna ask next, because I want us to get into the point of, okay, you know, we've tasted, it's taken real life calls, um, you know, how do we know whether it works the way we should? And and you've already started answering that. And when we're saying the big models and the smaller models, and, you know, taking out the large ones and then leaving the small are we also thinking cost there?

**[0:09:43] Shalina:** I'm assuming that bigger models mean more money.

**[0:09:46] Yogesh Bhagat:** Frontier models, if I would have run frontier model, then I would have not been able to make it live. It would have only be a pilot project, would have never gone live. But I think in case of I V R, it is beyond cost. Because in I V R, as we are talking even if I don't speak for another five seconds, you will feel like network is not working and you will hang up on me. So you need a sub-second response time, and that sub-second response time is where smaller models comes.

**[0:10:14] Yogesh Bhagat:** Even sometimes you will replace few things with M L models. So that's a learning which you built and you remove use larger model only for reinforcement learning or feedback loop. So how we do is everything we do have two reviews. One is a judge model and second is a human in loop, which is an escalation path for us. These two reviewers do not just act as a security check for us. They act as a feedback loops so that we can train our smaller models constantly and it is improving constantly.

**[0:10:47] Yogesh Bhagat:** This is where you are measuring their deterministic nature. And that's how, step by step, you'll remove larger models into the real from real environment, and you'll only use later on for reinforcement learning.

**[0:10:59] Shalina:** How do we know that the the caller is not feeling that switch to a larger model? Is that something that happens during the call, or is it something that we sort out way before we get to that point?

**[0:11:09] Yogesh Bhagat:** So there is a clear escalation path which you need to build. And like example, I'll tell you, we have caller path. Way to know that if there is combination of frustration, combination of a repeat of a question, and different scenarios, which led lower model know that if the question is getting repeated twice, then I need to escalate to a larger model and validate my response. And if that also says it's not what response is right, then there is an escalation path to a human in loop.

**[0:11:42] Yogesh Bhagat:** Biggest thing here is you need to make sure that there is no biasness while you are doing it. Because this is about access to care. So it's not just about escalation path, but also about ensuring there is no biasness in this escalation path.

**[0:12:00] Shalina:** Yeah, no, absolutely. And when we're saying there needs to be a human in the loop and there's a feedback loop going back to the human, who's the human in charge of? Is it going to be engineers? Is it the contact center floor? Is it compliance? Who gets to be involved in that?

**[0:12:13] Yogesh Bhagat:** Human in loop is the agents who are finally picking up the customer calls. What questions were asked from them? What were their responses? Why were we not able to answer? And that recording also goes as a feedback loop. So in this case, it's a real human who is solving final escalation path.

**[0:12:32] Shalina:** Great. So it goes back to, I would say, contact center flows, and I guess then certain scenarios would be used for training purposes as well. And giving feedback to the engineers as well as to where we need to maybe up governance or change the way it interacts. Am I getting the right idea?

**[0:12:49] Yogesh Bhagat:** Absolutely, because once you take that transaction, you put it into a reinforcement model of models. Engineers look at it. Do they really need to change the rag, or do they need to change some kind of a prompt? What is the real need? And that's where engineer comes into a picture.

**[0:13:06] Shalina:** Uh, makes a lot of sense. So we're getting to the part of the conversation where we get to tell our leaders thinking about replacing I V Rs and thinking about this entire process. They need to know where to start, they need to know what to look out for. So, for a leader about to hand a major customer-facing process to A I for the very first time, what would you let them or want them to have in place before they trust it to run on its own?

**[0:13:30] Shalina:** What do they need to get to make sure that it they earn the right of automation?

**[0:13:34] Yogesh Bhagat:** I'll say first, we already talked enterprise A I development platform, which allows all engineers, your average engineer to best engineer, to build A I agents in one consistent way. Without that's a table sticks, you need to build that. Okay, second, within that platform, you also need to build a testing approach. Deterministic testing approach doesn't work. We already talked about it. I will not repeat it. Third is ability to replay the transaction or observability. That's a crucial piece.

**[0:14:09] Yogesh Bhagat:** When you are talking with millions of customers, any one escalation comes. You need to know out of hundred agent, which agent answered, which particular task was deterministic in nature, where the path went wrong. And if you don't know how to do that, and it takes many, many hours to just find out one use case, then you are not ready for a go live. So you should be able to find out gap in your process in seconds, not in even minutes.

**[0:14:38] Yogesh Bhagat:** So that's a key piece. Then you need to know customer experience. Are you working for customer facing system where it is a sub-millisecond responses required, like example, I V R versus a chatbot? Chatbot is a place where you can get two seconds. Because you can say I'm triangulating, I'm doing this. People are so used to chat G P T Claude that they take they are okay with the time. So if you know that, then you need to pick right models which can answer or meet your things.

**[0:15:12] Yogesh Bhagat:** Or you can have engineering solutions like async way of working or whatever you like. There are different ways to solve that problem. Then economics. I will stress this part a lot of time, because everybody is tilted. I will apply one Claude and have open A I one model frontier model and the problem will be solved. But it will not be solved in an economical way. That's a crucial key to success. If you have not solved about that problem, then you are going to face issue after pilot.

**[0:15:45] Yogesh Bhagat:** So that's a core, I believe.

**[0:15:48] Shalina:** Would you advise our listeners to start with their main business problem or their main friction point? Or would you advise them to start on a narrower path? Like allow things like a customer phoning for store hours or just to follow up on whether a refill is already done, like start with narrower things before you expand, get the testing done on those first.

**[0:16:09] Yogesh Bhagat:** It is an interesting thing. We, it's a very debatable piece. We started the same thing. We were replacing the old legacy I V R, which used to say press one. It's a tree structure. And we said, hey, should we replace one? And then do step by step versus all at once. And what we learned is experience is so much broken between if you try to move from conversational A I to a s old I V R systems.

**[0:16:38] Yogesh Bhagat:** They will not be able to communicate in the same way. The tone which you want to set for consumers, you'll not be able to measure. So our approach was go for full blown, but go with a s smaller set of a customers. Learn from a smaller set of a customer, give them a complete experience, and then build confidence, build your learning models, build the feedback loop in that with smaller set of customers, improve your containment numbers later on.

**[0:17:08] Yogesh Bhagat:** First focus on experience, because what mistake you can get into is day, when you launch with a let's say 2% of your customers, you want the containment of that A I system to be better than the traditional I V R. That's a default every business user wants. No, that's not what you should focus on. First, focus on learning, observability, how you are going to improve the customer experience. Then automatically you will see the containment and satisfaction of a customer will increase.

**[0:17:39] Yogesh Bhagat:** So my suggestion will always go with the go small, but with full A I experience, otherwise it is difficult to change behavior of a customer.

**[0:17:48] Shalina:** I like that. I think before we wrap up, I think I just want to recap for our audience that I think what stood out for me was when you said no one has the right to go autonomous. We need to earn it and the testing and everything that goes on behind it is super, super important, like you've just highlighted. How do we know that it works? We need to make sure that we do have that continuous feedback loop that we are continuing to guide the agent and get it to a point where we know that we can trust it.

**[0:18:15] Shalina:** And then, like you just said, start small, start with a small sample of customers, take them through the entire journey, learn through that. It's part of your testing process. And that's how, like you said, how we'll gain confidence and confidence goes into trust and that's ultimately what we're paying for. Yogesh, anything that you'd like to leave our audience with?

**[0:18:33] Yogesh Bhagat:** I would leave it with one thing. Autonomy is not earned, not launch. You need to really earn it in that way. Safety layer should be so cheap that nobody get tempted to replace it. So make sure your economics play in your hand and safety is always first.

**[0:18:54] Shalina:** Exactly, and we'll leave it just there. Thank you so much, Yogesh.

**[0:18:57] Yogesh Bhagat:** Thank you.

**[0:19:12] Shalina:** Wrapping up today's episode, let's look at the three key takeaways from our conversation with Yogesh. First, autonomy should be earned rather than launched, with agents running on their own only after they give consistent, measurable responses, however, a question is asked. Second, governance scales when security, guardrails, and observability are validated once at a platform level rather than reviewed agent by agent. And finally, run costs have to be engineered from day one, because pairing smaller models with larger judge models and human escalation keeps safety affordable and responses fast enough for the customer-facing channels.

**[0:19:46] Shalina:** If you offer A I products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title, at the right type of organization, and, of course, readiness. Emerge attracts V P and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional I T. To learn the exact strategies we use to help leading A I brands and startups connect with their ideal enterprise A I buyers, visit emerj.com/AD1.

**[0:20:10] Shalina:** That's E M E R J dot com slash A D one. For further executive level analysis and to join our network of leaders delivering workflow impact with A I, visit emerge dot com. On behalf of the team at Emerge, we'll see you on the next episode.