# Managing AI Agents at Scale Across BFSI Operations - with Yoav Naveh of Reindeer AI - **Show:** The AI in Business Podcast - **Published:** 3 Jul 2026 - **Source:** https://podcast.emerj.com/managing-ai-agents-at-scale-across-bfsi-operations-with-yoav-naveh-of-reindeer-ai - **Transcript:** https://podcasts.opennash.com/shows/ai-in-business/managing-ai-agents-at-scale-across-bfsi-operations-with-yoav-naveh-of-reindeer-a-2ef53d/ - **Provider:** OpenNash Podcasts (machine-generated) ## In brief Banks and other financial firms have largely moved past AI experiments, and the hard part now is control rather than capability. Yoav Naveh of Reindeer AI argues that agents will only work in regulated operations if they can admit what they don't know, ask experts for help, learn from the answers, and be managed like software that is tested before each release. The central tension is between how easy it is to build an impressive agent and how hard it is to run one reliably. Demos break on exceptions, policy changes and staff turnover. Naveh says leaders should name someone to own the strategy and build only what gives a real competitive edge. He warns that slow adopters risk falling behind quickly. ## Key insights 1. **Demos fail when real-world exceptions arrive** Naveh says it is now easy to build a proof of concept (a small trial) over a weekend. Real operations bring edge cases and changing policies and regulations, which he links to the widely quoted claim that 90% of such trials fail. 2. **The key agent skill is asking for help** Instead of stopping or making up an answer, an agent should message a subject matter expert and ask targeted questions. Naveh calls this the 'perfect human in the loop' because the agent also collects data on how to solve the case. 3. **A second loop lets agents self-heal** One loop handles each case with an expert. A second loop gathers all that feedback and has an agent update its own knowledge, so the same exception does not keep recurring. 4. **Agent updates need software-style release discipline** Naveh says changes to an agent's knowledge need an approver, plus A/B testing and regression checks before rollout. He says the market is still working out who in the business owns that role. 5. **Build only where your data is unique** Naveh says engineers should not spend time on generic work like accounts payable, which is not a competitive advantage. He says in-house builds leave nobody to maintain them and create technical debt when their builder leaves. ## Questions this episode answers ### What are the three waves of AI adoption in business? Naveh describes the first wave as customer support and call centers. The second was assistants like ChatGPT or Copilot, where a person still does the job. The third is agents that carry out tasks across several systems and teams. ### Why do so many AI pilot projects fail once they reach real operations? Naveh says demos are easy to build, but real work has exceptions, edge cases, and policies and regulations that keep changing. He links this to the commonly cited claim that about 90% of proofs of concept fail. ### What does it mean for an AI agent to 'self-heal'? Naveh describes a second loop that collects the agent's conversations with human experts and feedback, and then updates what the agent knows how to do. The agent learns from its own exceptions instead of repeating them. ### Should a bank build its own AI agents or buy them? Naveh says it is tempting to build because it is easy, but nobody wants to maintain what they built. He recommends building only in areas that are strategic or rely on unique data, and buying for generic work like accounts payable. ### Are AI agents riskier than human staff for compliance? Naveh says people also make mistakes, which are hard to find because their steps aren't documented and are often repeated. He says agents can give compliance officers a detailed view and confirm a fix, but their mistakes can compound, so they need business rules and human checks. ## Facts - **Host** [0:00:12]: "Reindeer AI is an enterprise automation platform that deploys custom AI agents to handle labor-intensive back office workflows, including payment approvals, supply chain tracking and invoice processing." - **Host** [0:06:09]: "I listened to your presentation at the Manifest Vegas event and you had a very good way of explaining how there's a difference between just an A I tool and agentic A I" - **Yoav Naveh** [0:02:42]: "initial experimentation started at the call center, the customer support, which is there's one nice thing about it's kind of one big call center that has a lot of repetitive tasks" - **Yoav Naveh** [0:03:04]: "I think the second wave came into play with kind of a very high adoption of assistance." - **Yoav Naveh** [0:03:50]: "Exceptions and edge cases become much more frequent. Um, and I think that is uh such a promising opportunity for enterprise, because operation touches, you know, 70% of what they're doing." - **Yoav Naveh** [0:05:31]: "There's been all this talk about 90% of the P O Cs fail." - **Yoav Naveh** [0:06:38]: "And in most organizations, still, you're asking what they're doing. Like, oh, we gave copilot to all of our employees." - **Yoav Naveh** [0:08:46]: "I think building agents is on its way to become a commodity." - **Yoav Naveh** [0:09:11]: "You want it to actually be able to interact with a subject matter expert, reach out to them, ask them a specific series of questions in order to get themselves out of the hole." - **Yoav Naveh** [0:10:19]: "You wanna have a second loop that is able to aggregate all those conversations, all those pieces of feedback." - **Yoav Naveh** [0:11:33]: "Whenever you have a new version of the agent, you wanna make sure you you a B test it before you roll it out to your entire business." - **Yoav Naveh** [0:12:39]: "it's harder to detect them because there's no documentation of the step by step that it that they're doing." - **Yoav Naveh** [0:13:30]: "It's a slower mistake. It doesn't compound. I think the question is, how do you make sure agents don't make compounding mistakes?" - **Yoav Naveh** [0:15:48]: "seen five different departments in the same company that they all have a different account payable process." - **Yoav Naveh** [0:14:56]: "this woman from Anthropic spoke. So we're getting into a world where everybody's gonna be a product manager and everyone's gonna build." - **Yoav Naveh** [0:19:02]: "the outcome was that they took a process that would take them days to close and turn that into hours." - **Yoav Naveh** [0:19:24]: "Whoever is replying first is much more likely to win the deal." - **Yoav Naveh** [0:19:24]: "The reality is that in two years time, that's going to be the standard." ## Transcript **[0:00:12] Host:** Welcome everyone to the Emerj AI in Business podcast. Today's guest is Yoav Naveh, co-founder and co-CEO at Reindeer AI. Reindeer AI is an enterprise automation platform that deploys custom AI agents to handle labor-intensive back office workflows, including payment approvals, supply chain tracking and invoice processing. Yoav examines how large enterprises in financial services have moved through three waves of AI adoption, from customer support tools to AI co-pilots to full operational agents, and why the third wave demands a fundamentally different approach to governance. **[0:00:47] Host:** Today's episode is sponsored by Reindeer AI. In this episode, we cover how enterprises in banking and financial services are operationalizing agentic AI inside regulated workflows. To go deeper on this topic and learn how banks are using RPA to reduce operational costs and repetitive workflows, and applying deviation detection to identify fraud in real time, download our free PDF report, AI in Banking Executive Cheat Sheet at Emerj dot com slash B C S one. That's E M E R J dot com slash B C S one to download your copy. **[0:01:22] Host:** Now, the conversation with Yoav. Yoav, welcome to the Emerj AI in Business studio. **[0:01:47] Yoav Naveh:** Uh, great to be here. Thanks for having me. **[0:01:48] Host:** Yeah, I'm I'm excited to get into AI in financial services today, but from a different perspective than we've been talking about in the last couple of months and weeks. And we're gonna start where we've been hearing from the operational side of things that there's a real shift, right? So, A I is no longer experimental in financial services, but it also feels like the pressure to operationalize is just getting more and more and more. And in the middle of this, uh, we have agentic A I that also landed, and it landed without a shared definition, almost. **[0:02:21] Host:** And you've been building these systems inside workflows where ambiguity is isn't an option, right? So where compliance and legacy infrastructure and human oversight, they have to coexist in the same operations. How have institutions evolved in their thinking, and where does agentic A I actually fit in? **[0:02:42] Yoav Naveh:** Yeah, I I think it's a really interesting way to frame this. The way I see this, we've we've kind of been experiencing three waves of AI adoption in the enterprise. I think at first, initial experimentation started at the call center, the customer support, which is there's one nice thing about it's kind of one big call center that has a lot of repetitive tasks that are kind of very similar. If someone is asking a question and they're getting an answer. **[0:03:04] Yoav Naveh:** And I think the second wave came into play with kind of a very high adoption of assistance. Uh, you have chat G P T or copilot or cloud in the organization. And it's still kind of the point is let's take someone that has maybe it's an expert employee in my company, have them ask a question, ask A I a question and get some assistance. But at the end of the day, the person is still responsible for for the job. **[0:03:26] Yoav Naveh:** Super important. There's a lot to do there. I think the third wave, which is what we're seeing now, is like, OK, we started to understand how we can use A I, but how can we actually use it on the very broad sense of operation? Which, as you said, that that. Could mean so many different things. It could mean accounting, it could mean treasury, it could mean in financial crimes. And I think one of the interesting things that's happening there is that it's not just a question and answer. **[0:03:50] Yoav Naveh:** We actually have to take a task across multiple systems, multiple teams, um, and it's a lot more difficult to just close a task. It's not like, I've got got asked a question. Here's the response. And then someone would be able to do something with it. So the agent human uh aspect becomes much more important. Exceptions and edge cases become much more frequent. Um, and I think that is uh such a promising opportunity for enterprise, because operation touches, you know, 70% of what they're doing. **[0:04:17] Yoav Naveh:** The ability to provide services faster to their customers, the ability to capture revenue faster, also the ability to introduce cost efficiency. Uh, but the level of complexity in doing that is also dramatically bigger than the two waves. **[0:04:30] Host:** Yeah, and I like the picture that you painted there saying that it's no longer just a question and answer. Context is is very important there. And in this environment, the the compliance constraint is the obvious one, right? But I suspect that that's not the primary bottleneck. What's the thing that trips institutions up that they don't usually anticipate? **[0:04:49] Yoav Naveh:** So, so I think compliance is really important in the sense that. Uh, there's trust that has to be built and governance potentially requires you to kind of track everything that the A I is doing. I mean, I actually think that's that's today kind of almost coming out of the box. It's very easy with the with the um these new A I tools to see what the A I is doing and understand the reasoning. I think that the biggest challenge there in the market is, first of all, it's incredibly noisy. **[0:05:11] Yoav Naveh:** People ask us, who are your competitors? I mean, it's not so much about who's competing with us about the entire noise. How do you actually build a strategy around all these different uh tools? I think when we started reindeer, and that was the question. It was very obvious A I is going to make its way to every function in the enterprise. The question we became obsessed with is how do you actually build a strategy around that? **[0:05:31] Yoav Naveh:** And what we realized is that the biggest issue today is that we we've never lived in a world where it was so easy to build a cool demo and a cool P O C over over the weekend. But when that hits the real world, real world is messy. There are exceptions, there are edge cases, there are policies that changes, there are regulation that keeps uh changing. And that's where all the P O Cs fail. There's been all this talk about 90% of the P O Cs fail. **[0:05:57] Yoav Naveh:** I I think the end of the day that comes into that. I think that's the most interesting question. You mentioned earlier, how do humans and agents work together? It's, hey, how do they work together over time? How do we make sure that humans are able to keep teaching agents over time? **[0:06:09] Host:** Yeah, and I like it, so uh I I. I listened to your presentation at the Manifest Vegas event and you had a very good way of explaining how there's a difference between just an A I tool and agentic A I, and I want us to also draw that into the financial services environment. So, if when does a a financial services leader realize that their existing A I tools aren't getting the job done? What does that look like for them? **[0:06:36] Host:** What are the red flags that they should be noticing? **[0:06:38] Yoav Naveh:** So, so I think realistically, if you ask me a year ago, I would I would expected the entire industry to be in a much more advanced place of where we are. I don't think we're in a point that you're speaking to financial. We just came back from the Gardner, uh, Fine Star Symposium. And in most organizations, still, you're asking what they're doing. Like, oh, we gave copilot to all of our employees. And I think they're not really agents are not ruling the world today. **[0:07:01] Yoav Naveh:** I think when enterprises are playing with that, I think most of them are missing a key aspect. Everybody is measuring the accuracy of uh of AI agents, which obviously that's super important. When an agent gives you a response, it needs to be the right response. But I think often people miss when they're, they're evaluating solution. They're missing the part of, hey, what would happen when the agent doesn't know? And they expect this 100% automation, deterministic world where in reality the agent will know that 100% is not a feasible. **[0:07:31] Yoav Naveh:** It's not even the right uh goal. Um, and I think they're not looking at that. How do you manage the agents when they don't know? **[0:07:39] Host:** Yeah, okay, that's great. I love the way that you framed that because that's gonna lead us right into what I wanted to ask you as, uh, for the second part of our conversation. And that is, we basically started on how institutions arrived at Agentic A I. But we need to get into what do they need to do something real with it. It's great to have it, but how do we do something real with it? And the pattern that I keep coming back to is that the bottleneck isn't capability anymore. **[0:08:06] Host:** And you just basically explain that. It's very capable, um, but it's control. Control is where we need to be more involved, being able to let a system act autonomously inside a regulated environment. Uh, and still be able to explain to an auditor why certain things happened or why certain decisions were made. And I think this is where you've built around that specific tension, right? The oversight layer, the governed logic, if you will, the things that protect an institution's position. **[0:08:35] Host:** Um, and and it's important to get that right. So which capabilities genuinely move the needle, and what is it? What does getting this right actually look like? **[0:08:46] Yoav Naveh:** Yeah, I think you're right. I think building agents is on its way to become a commodity. And I think the the knowledge required to solve a specific problem is is getting narrower and narrower. We see that in coding agents, right? Uh, like, you know, six months ago, like, what was not even where it is today, it's becoming so much easier to do that. And we're gonna start seeing that in all the financial operation as well. I think when you're moving into that, the question becomes the most. **[0:09:11] Yoav Naveh:** I think the most important skill for agents is the ability to ask questions. What you want when an agent fails to do something, you don't want it to stop, you definitely don't want it to hallucinate. You want it to actually be able to interact with a subject matter expert, reach out to them, ask them a specific series of questions in order to get themselves out of the hole. Basically, okay, I've seen a case, whether it's a uh an A M L alert, whether it's um a vendor onboarding, whether it's a customer onboarding. **[0:09:39] Yoav Naveh:** Whenever something goes not according to plan, you want the agent to be able to raise its hand and interact with the subject matter expert. Use the the great benefit of of L L Ms that they're able to have a conversation. Conversation is the best interface. We can talk about, we're talking about technology and we can talk about music or movies. We do that with the same interface, right? It's not a fancy system you have to learn. **[0:10:00] Yoav Naveh:** We all know how to do that. Have agents be able to kind of have that communication. And I think that's the first layer. A lot of people talk about the human in the loop. That's that's the perfect human in the loop, in the sense that it's able not just to stop, not just to hand over to a person, but actually interact to collect data, to learn how to solve that case. The way we think about it, we think there is a second loop. **[0:10:19] Yoav Naveh:** We actually have a two loop system. You wanna have a second loop that is able to aggregate all those conversations, all those pieces of feedback. And have an agent think about, okay, based on everything I learned, uh, about the world, how do I keep evolving my knowledge? How do I self heal? How do I expand what I know, uh, how to do? And I think that's that's the most important part. You ask me, what is the capability? **[0:10:39] Yoav Naveh:** In my mind, that's the capability agents are able to solve their own problems, self heal, and expand what they do. **[0:10:45] Host:** Yeah, that's that's an interesting way to frame it and to look at it. I like the self healing concept and a lot of this conversation is going to lead to decisions and and that's when we need to talk about how to govern the decision logic. So what does that require? Is is that the data architecture? Is it a policy layer or a combination? How do we? How do we work around that? **[0:11:07] Yoav Naveh:** Yeah, I think on the individual case, it's subject matter expert, it's potentially part of the team that is doing that today, because in reality, most of the knowledge like that of how processes happen lives inside people's mind. There's just so much institutional knowledge with with all these uh, I mean, large enterprises. Um, I think the second part of it would be something that would be emerging right now is like, who's managing the agent? And when the agent is changing its context, its its knowledge, who is the person to approve that? **[0:11:33] Yoav Naveh:** And I think that's still emerging in the market. I think on the one hand, um, could be the the person, the group manager, could be the the person that wrote writes the policy today. But it's also the responsibility of of whoever is writing the agent to make sure that it's performing, testing correctly. Think about that as even as how we roll out software. Whenever you have a new version of the agent, you wanna make sure you you a B test it before you roll it out to your entire business. **[0:11:58] Yoav Naveh:** You wanna make sure you run some regression analysis. So we're gonna have these people that have business expertise that they need to kind of understand how do we roll out versions? Um, and and that's a gap that is still being closed, I think, in the industry. **[0:12:09] Host:** Yes, I I think in in the financial services industry, that gap is is still largely with compliance. Because, like you said, there's a point of responsibility and the the point of subject matter experts. And I think it's still difficult for institutions to have their compliance officers or their risk teams completely buy into this. I think they're quite skeptical of any system that will make decisions that they can't fully explain. So how do we build confidence with the audience, not just sign off, but how do we get them to really trust this? **[0:12:39] Yoav Naveh:** Yeah, I think, F first of all, realistically, whether you have a team of uh, um, you know, dozens of people that are sitting in the floor below you, or whether they're sometime offshore and in another place, the reality is people make mistakes as well. I think part of the issues with mistakes that are happening with people is they're they're one is they're. It's very, it's harder to find them, it's harder to detect them because there's no documentation of the step by step that it that they're doing. **[0:13:07] Yoav Naveh:** And the second thing is once you find it and you wanna teach it, people forget, people make the same mistakes. I think with agents, the opportunity is actually to be able to surface a much more detailed view for whoever is in charge of compliance, chief compliance officer, and when once something is fixed, know that it is fixed. I think the other side of it, and that's the risky part here, is that when people make mistakes, it's a. **[0:13:30] Yoav Naveh:** It's a slower mistake. It doesn't compound. I think the question is, how do you make sure agents don't make compounding mistakes? And putting down down business rules to do that, uh, putting down human in the loop in those processes, building an ocean of confidence, kind of as you grow uh into that, are critical pieces. **[0:13:47] Host:** Yeah, I like that it. It takes me back to an episode that we had a few months ago where uh, the guest said, uh, having agentic A I is like, um, switching on a light in a messy room. And they're like, oopsie, the the room is messy. Uh, let's just switch the light back off and then think that the problem is gonna go away. But like you said, it basically amplifies the existing problems, but it's easier to fix them. **[0:14:10] Host:** We can fix them before they become, uh, or before they go into full crisis mode this way. Uh, I want us to also just bring, uh, bring up the decision that I think most leaders are wrestling with at the moment, and that is, do we build it ourselves or do we bring in what we need? So there's a very build versus buy, or build versus build, or buy versus buy. There's the different ways to think about it. **[0:14:32] Host:** And it turns out that this question is hotter than it looks because it's not just about capabilities, like you just mentioned. It's about who owns the outcome, uh, when something goes wrong in a workflow, um, who owns the outcome when something goes wrong in compliance or with customer funds? So how should leaders be approaching that decision? And what do you see institutions getting wrong when having to make that decision? **[0:14:56] Yoav Naveh:** And it's it's so tempting because it's so easy to build today, right? We can all spin up cloud code or or codex and and build something that looks really, really cool at first. That's the other part. It is really cool. Everybody wants to build it. And I think the problem is everybody wants to build it, nobody wants to maintain. And um, I was, I was at an an event, and um, this woman from Anthropic spoke. So we're getting into a world where everybody's gonna be a product manager and everyone's gonna build. **[0:15:21] Yoav Naveh:** Uh, first of all, that's kind of a scary word to world to me, that everyone's a product manager. But I think even if you think about that, what happens when whoever built this, when they, when they leave your company? Like, who owns that? who maintains that? And I think a lot of time when organizations think about this, they're they're getting caught up with how easy it is to build, but they're not building in where they have core competency, where what is what is core in providing a competitive advantage to their business. **[0:15:48] Yoav Naveh:** I think that there's no nothing special about how account payable is be being done. It may, by the way, we've we've seen five different departments in the same company that they all have a different account payable process. But that's not a competitive advantage for almost any company, unless that's your business to to run account payable. But you still see companies saying, OK, we're gonna build it ourselves. And then I'm asking myself, is that the best use of time of your engineers, of your I T department? **[0:16:11] Yoav Naveh:** Shouldn't they be building agents in areas that no one else can, because you're not gonna give them access to your unique data, or that is something you have to own because it's strategic for you. So I think enterprises are often kind of getting swept with, oh, it's so easy to build. Let's build in all these uh, all these places. And when they need to start maintaining it, that that becomes a a very painful vicious cycle. You're starting starting to build tech debt around that. **[0:16:34] Yoav Naveh:** Um, and sometimes it's hard to let go. Once you've kind of made that decision, it's hard to let go and say, okay, you know what? We should actually focus on these agents that are unique for us. **[0:16:42] Host:** Yeah, that makes sense. And I'm wondering that you've obviously now seen this happen in different parts of different industries, different organizations. Uh, is there a a common mistake that you see institutions make at this stage when they need to make this decision? Uh, the one that basically derails, um, a solid strategy. **[0:17:01] Yoav Naveh:** I think the word strategy is the important part here. I think, um, I think the question is, how do you make sure you don't start something without understanding how is that going to work in the the long run? And when people ask me, what is the the most important first step that we can do with A I? I think it's putting a person in charge of building a strategy and deciding what are the areas that I should build. **[0:17:23] Yoav Naveh:** Maybe there are some areas that I need to work with a point solution because they require such a strong specialty. Where do I work with a platform that is going to allow me to work faster with the companies that are offering these type of solutions? Do I go with a deployment company like a forward deployed engineer? Uh, or do I work with a solution that would allow me to build? Like, you know, it's like buy buy so you can build, right? **[0:17:45] Yoav Naveh:** Not not just buy for someone else to build it for you. And how is how is your organization going to look in two years? Are you going to adopt A I, but then become dependent on, like, this so powerful tool that someone is else is running for you? Or are you actually going to transform your organization, make your organization, if not A I native, but at least truly A I powered? **[0:18:05] Host:** That I think those are very good pointers and very good, um, a very good indication of of the reality of what we're seeing at the moment. Uh, if I had to to wrap up our conversation today, I would say very important for our audience is to take away that, you know, we've we've moved past the experimentation stage in financial services and the pressure now is to operationalize, which means that agentic A I has to work inside compliance, inside legacy infrastructure, and with human oversight simultaneously, and and not skip one or work around them. **[0:18:36] Host:** And I think a very important thing that I got from you today is that the capabilities that make this sustainable aren't about what the agent will do autonomously. It's it's about what the governance layer catches and corrects and learns from. And and you keep using the word learn in a very constructive way, in a very good way for for agents to be amplified. Um, and then lastly, I learn. Yeah, sorry, go ahead. **[0:19:02] Yoav Naveh:** I just want to give one example you mentioned kind of in terms of when is the time? And is it like we're still sitting on the fence, seeing what things are happening? I can give just one example. We have, um, one company we work with where they've taken their entire um, uh, loan quoting process and start running that with A I. And the outcome was that they took a process that would take them days to close and turn that into hours. **[0:19:24] Yoav Naveh:** That means directly increase in win rate. In the end of the day, the customer is checking a few quotes. Whoever is replying first is much more likely to win the deal. And today, that gives them a competitive advantage. The reality is that in two years time, that's going to be the standard. In two year, two years time, if you're not doing that, you're not losing a little bit of business, you're out of business. And I think we're at a time that enterprises that don't have a strategy around that really have to ask themselves if they're not putting themselves at big risk. **[0:19:50] Host:** I think that's a great final thought to leave our our leaders with. And I think that is actually a good um conversation to continue with. So, yeah, we're gonna have to get you back in the studio because we need to to elaborate on on this last point. Definitely. It's been great having you here. Thank you for your time and I hope to speak to you again. **[0:20:08] Yoav Naveh:** Sounds good, thanks, it's been a pleasure. **[0:20:24] Host:** Wrapping up today's episode, let's look at the three key takeaways from our conversation with Yoav. First, financial institutions have moved past A I experimentation. Deploying agentic A I at operational scale now requires compliance, legacy infrastructure, and human oversight to coexist simultaneously, not in sequence. Second, the capability that matters most in regulated environments is not what the agent does autonomously. It's the two loop governance model that allows agents to interact with subject matter experts in real time and continuously refine their own decision logic. **[0:20:58] Host:** And finally, the build versus buy decision is not primarily about technology, it's about ownership and leaders who define where accountability sits before they select tools will scale faster and with less regulatory exposure. In this episode, we covered how enterprises in banking and financial services are operationalizing agentic A I inside regulated workflows. To go deeper on this topic and learn how banks are using R P A to reduce operational costs in repetitive workflows, and applying deviation detection to identify fraud in real time, download our free P D F report, A I in Banking Executive Cheat Sheet at Emerj dot com slash B C S one. **[0:21:39] Host:** That's E M E R J dot com slash B C S one to download your copy. On behalf of the team at Emerj, we'll see you on the next episode.