OpenNashPodcasts

AI in Financial Services

Reducing False Positives for Anti-Money Laundering - with Scott Nowson of PwC

1 Jul 2022 · 21m · Daniel Faggella, Scott Nowson · 21 quoted facts

▶ Play episodeOriginal source ↗

In brief

Anti-money laundering teams drown in alerts that are almost all false alarms. Scott Nowson of PwC argues that AI works better as a tool to clear away the harmless 99 percent than as a way to find the rare real case. Cutting false positives saves money quickly and builds trust. The explanations must stay simple enough to defend to a regulator.

The central tension is ambition versus accountability. Banks cannot accept an opaque model that might miss real laundering, so PwC chose explainable methods and let the models run quietly beside human investigators. Scott describes learning what normal looks like first, then proving the system in stages. Stakeholders tolerated some errors in testing, which unlocked bigger gains than anyone expected.

Key insights

  1. Clearing false alarms beats hunting for needles

    Because under 1 percent of alerts are real, a model that calls everything harmless is 99 percent accurate but useless. PwC instead aimed to remove a large share of the harmless alerts, shrinking the pile investigators must search.

  2. Watching investigators reveals the useful signals

    PwC sat with the human agents to see how they dismissed false flags quickly. That showed which patterns, such as changes in money in and out, were worth turning into model inputs.

  3. Regulators force explainable models over deep learning

    Banks must tell auditors why a suspicious case was missed. Scott therefore favors methods like decision trees that unfold into plain rules over neural networks that cannot yet be fully explained.

  4. Start with normal behavior to win early ROI

    The 99 percent of ordinary data gives the most material to learn from and the quickest savings. Early wins give stakeholders the confidence to let the team work on the risky 1 percent later.

  5. Staged rollout and tolerance for errors unlocked gains

    The model first ran alongside humans without interrupting their work, with each alert tagged by its prediction. When the team aimed higher than the no-error pilot, one segment reached a 50 percent cut in false positives with only two errors.

Questions

Why are there so many false positives in anti-money laundering?

Banks run monitoring rules that flag anything that looks unusual, so they generate huge numbers of alerts. Scott Nowson said one bank got about 10,000 a month, and typically less than 1 percent were true suspicious activity.

Can AI just find the real money launderers directly?

Scott argues that finding the rare real cases is very hard, since a model that calls everything harmless would be right 99 percent of the time. His approach was to use AI to eliminate false alarms and make the pile smaller for humans.

Why not use the most advanced deep learning models for this?

Compliance requires a bank to explain to a regulator why it missed a case, and deep learning is hard to explain. Scott prefers approaches such as decision trees, which can be converted into clear rules.

How did PwC prove the system worked without risking a fine?

They first measured error rates on historical data, then ran the model beside human investigators, who still reviewed every alert. Each alert was tagged with what the model would have predicted, so stakeholders could see accuracy before any automation was switched on.

How much work did the AI save, and what was the catch?

One bank targeted a 40 percent reduction in false positives, and in one data set and banking segment PwC reached 50 percent. The catch was two errors, which the stakeholders were willing to tolerate during the pilot.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

Dan Faggella says financial services companies direct most of their AI spending to risk and compliance functions rather than customer-facing features.

“risk-related functions is where the money is going in financial services, and any bank that tells you otherwise is blowing smoke.”

0:00:30 · Daniel Faggella · Bank AI spending

A bank that approached PwC reported receiving about 10,000 transaction-monitoring alerts a month, of which typically less than 1 percent involved real fraud or money laundering.

“we're getting 10 ,000 a month. And when you look into the data, the amount of actual fraud, of true money laundering, of a suspicious activity is typically less than 1%.”

0:03:51 · Scott Nowson · Alert volumes

Scott Nowson says each alert takes a human agent at least 20 to 30 minutes to review, so clearing the false positives requires a large team.

“with a human agent having to look at the 99 % and each one taking a minimum 20 to 30 minutes, that's a huge team that you need to look at this”

0:03:51 · Scott Nowson · Investigation costs

PwC builds transaction monitoring rules with its forensic experts and implements the systems that watch all transactions passing through a bank.

“we will build, using our forensic experts, some of the transaction monitoring rules and implement the transaction monitoring systems that are essentially watching everything that goes through a bank.”

0:03:20 · Scott Nowson · Transaction monitoring

Dan Faggella says Scott Nowson is based in Dubai, where the PwC AI team he leads is growing.

“You're still out there in Dubai. Your team out there with PwC is certainly growing.”

0:02:44 · Daniel Faggella · PwC AI team

Scott Nowson says that labeling every alert as a false alarm would give 99 percent accuracy, which shows why accuracy alone is a poor goal for finding rare real cases.

“You'll get a 99 % accuracy rate when you say everything's wrong and that just doesn't work.”

0:05:10 · Scott Nowson · Class imbalance

Scott Nowson says he learned to start by understanding the problem while working at Xerox alongside ethnographers, and applied this by sitting with real investigation agents.

“something I learned over the years working at Xerox and working with ethnographers was just understand the problem.”

0:05:10 · Scott Nowson · Research method

Scott Nowson says human investigators typically review only about the last six transactions, while machine learning can scan two years of data in seconds.

“the human can only look at like the last six transactions. The whole point about machine learning is big data. It can look over two years in seconds and it can see much bigger patterns.”

0:06:12 · Scott Nowson · Machine learning scale

Scott Nowson says true positives were so varied that roughly 10 to 20 percent of alerts would still need investigation to find the 1 percent that is real.

“it really needs the 10 to 20 % should still be investigated to find the 1%.”

0:08:03 · Scott Nowson · True positives

Scott Nowson says deep learning is unsuitable for anti-money laundering because banks must explain to regulators at any audit why they missed a case.

“But you can't do deep learning in this environment because compliance. You have to be able to go to the regulator and say at any point being audited why you missed that.”

0:08:55 · Scott Nowson · Explainability

Scott Nowson mentions that MIT produced work on image classification that explains what each neuron in a neural network does.

“I think MIT came out with something for image classification that can explain what each neuron is doing in the network.”

0:08:55 · Scott Nowson · Neural network explainability

Scott Nowson recommends decision tree algorithms for this work because they can be unfolded into explicit rules that regulators can audit.

“you take something like a decision tree algorithm, something which can essentially fold out into rules.”

0:08:55 · Scott Nowson · Decision trees

Scott Nowson says many transaction monitoring rules rely on fixed thresholds, which customers eventually cross as their finances grow, producing alerts.

“So a lot of the transaction monitoring rules are about having a threshold. And finances change over time. We make more money, we spend more money. At some point, we pass the threshold.”

0:10:58 · Scott Nowson · Monitoring thresholds

Scott Nowson says PwC has done similar deployments in banks in Europe, where it begins by modeling normal behavior in the data.

“and we've done similar things in banks in Europe, is we start looking at the normal and we start looking at the data”

0:10:58 · Scott Nowson · Bank deployments

Scott Nowson says PwC typically applies its machine learning layer to filter alerts before the bank's transaction monitoring output is reviewed, and uses it to suggest new detection rules.

“What we've typically done then is we've taken this post-transaction monitoring system and applied it before the transaction monitoring is done.”

0:11:27 · Scott Nowson · System design

One bank PwC worked with set a 40 percent reduction in false positives as the target that made up its business case.

“One of the banks we worked with set out, they want 40 % reduction in the false positives that for them was the business case”

0:13:01 · Scott Nowson · Business case

PwC's rollout began with statistical testing of error rates and then moved to a semi-live beta that ran without interrupting the existing workflow.

“So the statistical confidence first, then it's going semi live, almost beta with the system where you don't interrupt the workflow.”

0:16:01 · Scott Nowson · Phased rollout

In the shadow phase of PwC's project, humans still investigated every alert while each was tagged with the algorithm's prediction.

“Every alert is still investigated by a human, but it's now tagged with what the algorithm would have predicted.”

0:16:01 · Scott Nowson · Shadow testing

Scott Nowson says PwC assumed the bank would want zero errors in the pilot, but stakeholders turned out to tolerate some errors.

“But we assumed they would also want no errors in the pilot. No, they were able to tolerate some errors.”

0:18:12 · Scott Nowson · Error tolerance

In the pilot phase PwC's approach identified 20 percent of false positives with no errors.

“we hit like in the pilot phase, 20 % false positive identification with no errors.”

0:18:12 · Scott Nowson · Pilot results

With one data set and one banking segment, PwC achieved a 50 percent reduction in false positives with only two errors, exceeding the 40 percent target.

“We were actually able to get with one data set with one particular banking segment, 50 % reduction in false positives, with it turns out only two errors.”

0:18:12 · Scott Nowson · Pilot results

Chapters

  1. 0:00:30Why banks fund risk and compliance AI

    Dan Faggella frames the episode: financial firms advertise customer-facing AI but spend most on risk and regulation, which is where early AI projects gain traction.

    “risk-related functions is where the money is going in financial services”
  2. 0:03:51The client problem: 10,000 alerts a month

    Scott describes a bank whose analysts spend 20 to 30 minutes on each alert even though almost all are harmless, which creates the business case for AI.

    “of a suspicious activity is typically less than 1%”
  3. 0:06:12Flip the problem: shrink the haystack

    Scott explains why trying to find the needle fails and why removing false positives, informed by watching human agents, is the better aim.

    “you can sure clear off a lot of the hay”
  4. 0:08:55Explainability limits the choice of models

    Auditors can demand reasons for any missed case, so PwC avoids deep learning and builds rule-like models from features humans already use.

    “you can't do deep learning in this environment because compliance”
  5. 0:11:58Learn normal behavior first

    Scott gives a time order for the work: model the 99 percent of normal data first, then use that framework to catch the risky 1 percent.

    “You start with finding the normal and those mechanisms that you build in”
  6. 0:16:01A staged rollout that wins stakeholder trust

    The team moved from statistical testing to a shadow run where humans still reviewed every alert, then turned on automation piece by piece.

    “Every alert is still investigated by a human”
  7. 0:18:12Tolerating errors unlocked bigger gains

    Involving everyone from agents to the chief compliance officer showed that some pilot errors were acceptable, which led to a 50 percent reduction in one segment.

    “50 % reduction in false positives, with it turns out only two errors”

Transcript

0:00:04
Daniel Faggella

This is Daniel Faggella and you're listening to the AI and Business Podcast. In the world of financial services, companies love to talk about everything they're doing that's new for their customers. We're launching this new cool chatbot. We're doing this new cool thing that our users can use in our online banking experience. They like to talk about that, but where they like to actually invest their money is in regulation and compliance related applications. It's not where they're putting all the dollars.

0:00:30

Certainly, customer experience is getting some of the funding. But risk-related functions is where the money is going in financial services, and any bank that tells you otherwise is blowing smoke. It behooves them to make it seem like they're investing in all the hip stuff. But risk is where the money's at, and it's where a lot of the early AI projects are gaining traction. We're talking today about anti-money laundering, a gigantic regulatory concern for financial services organizations.

0:00:56

And our guest is Dr. Scott Nowson. Scott runs artificial intelligence for PwC, one of the largest services companies in the world. Scott has been on our program in the past. He's also been on the AI and Business podcast. If you're not a subscriber to AI and Business, I recommend you do so. He's also been quoted in plenty of our Emerj Plus articles. Again, for those of you who don't know, Emerj Plus is our complete library of AI use cases and best practice frameworks for consultants and AI practitioners who want to be able to build trust with stakeholders and bring AI projects to life.

0:01:27

I'll talk a little bit more about Emerj Plus in the outro. You can learn more at emerj.com slash p1 if you want to put much of what you're learning here in action. Emerj Plus is a great place to learn, emerj.com slash p1. But again, I'll talk more in the outro there. In this episode, Scott dives in on the topic of reducing false positives for anti-money laundering. Whether you're focused on cybersecurity, payment fraud, financial crime, the work of analysts is mostly looking at things that have no risk inherently at all.

0:01:56

They're looking at things that really don't need to be investigated, but we have to kind of dig through all of our transactions, dig through all of our online activity and figure out where those needles are in the haystack that represent risk for us as a company, not just risk financially, but also risk in a regulatory sense. And in this episode, Scott does a great job of walking through how artificial intelligence impacts the analyst's experience before and after.

0:02:20

So if you're involved in anything involving detecting anomalies or reducing risk in FinServe, hopefully this episode will be more than useful. I appreciate Scott being with us. So without further ado, let's fly right in. This is Dr. Scott back on the program on the AI and Financial Services podcast.

0:02:44
Daniel Faggella

So Scott, welcome back to the program. It's been a while. Thank you, Dan. It has indeed. Thank you for inviting me back on again. Yeah, it's a pleasure to chat. You're still out there in Dubai. Your team out there with PwC is certainly growing. You guys are doing some exciting projects. Obviously, FinServe is always a sector you're close to. And I know today we're talking about compliance. We see so much activity in the compliance space when it comes to AI adoption.

0:03:05

And I know we're going to be diving into the reduction of false positives, which probably everybody in FinServe can relate to in some way, shape or form if you are in the fraud, financial crime, AML space. Talk to us a little bit about the client situation that you walked into, and then we'll talk about AI from there.

0:03:20
Scott Nowson

So we're very familiar with a lot of these situations in the bank because obviously we have large compliance sectors of our firm as PwC. This is what we look at. And we will build, using our forensic experts, some of the transaction monitoring rules and implement the transaction monitoring systems that are essentially watching everything that goes through a bank. So we're very familiar with what they do that generates this. And what one bank came to us with was the number of alerts that we are getting is just simply too many.

0:03:51

You know, we're getting 10 ,000 a month. And when you look into the data, the amount of actual fraud, of true money laundering, of a suspicious activity is typically less than 1%. And with a human agent having to look at the 99 % and each one taking a minimum 20 to 30 minutes, that's a huge team that you need to look at this. It's usually costly and it's easy to dismiss the false positives, but it's just that you have to go through that time.

0:04:23

And the question was, can AI help with this?

0:04:27
Daniel Faggella

Yeah. Okay. So there's a lot of reasons why we've seen AI creep into compliance. I'm going to get your thoughts on that before we wrap up, but the business case makes sense. We've got a big problem. We have to kind of look at everything currently, and it's less than 1 % that actually is AI, which means there's a lot of human hours looking at stuff that everybody sort of knows it's a bit of a needle in a haystack type of a situation.

0:04:50

It's expensive to do that. It's time consuming to do that. So I understand the business case. And there's a lot of, obviously, as you're aware, point solution vendors that are really focused on this space specifically. Clearly you guys are more of a services firm, but you also have a lot of subject matter expertise in this domain. Where did you start when it came to figuring out where AI could add value on this problem?

0:05:10
Scott Nowson

Well, the biggest thing was you're exactly right about the needle in the haystack. And that was in the client's mind, the problem that they wanted us to solve. Can you find the needle in the haystack? But in any data problem, finding the 1 % is just insane. You'll get a 99 % accuracy rate when you say everything's wrong and that just doesn't work. So the best way we did it was something I learned over the years working at Xerox and working with ethnographers was just understand the problem.

0:05:43

And we went and sat with the real agents who did the investigation to see what they did. And along with there was some obvious process improvement opportunities, we were able to see how easy it was for them to dismiss some of these false flags and how they did it. And so that gave us the idea to just make more sense to flip the problem and make it more. Why doesn't AI just eliminating the false positives, eliminating the effort?

0:06:12

You're never going to find the needle in the haystack, but you can sure clear off a lot of the hay and make it a smaller haystack. And the biggest thing with AI was how are they doing this? Well, they're trying to look at patterns in transactions. But the human can only look at like the last six transactions. The whole point about machine learning is big data. It can look over two years in seconds and it can see much bigger patterns.

0:06:40

So that was really where the advantage came in of that scaling the eyes on the data.

0:06:45
Daniel Faggella

Yeah. And so it sounds like you started with how are the human experts sort of getting rid of some of this extra hay as we're referring to it here. So these examples of flags that we all, you know, they kind of understand pretty quickly are really not worth spending a lot of time on. You're kind of looking at how are humans doing some of that parsing. it seems like there's a number of approaches you could have begun with if the goal was to reduce the the false positives there is well there's a tuning in ai model better and better and better to what do the real positives that we can detect actually have in common and then being able to

0:07:25

kind of better pattern match that so having a having a better calibrated kind of confidence threshold of every time something's over 80 maybe it really is actually worth investigating versus right now, it's just one of the 200 things that's a no before we get to something that's a yes. So better tuning in the AI onto itself. There's looking over the shoulders at your best human beings and being able to see what are the mental rules they're using and is there a way to bake in those mental rules to kind of streamline our process.

0:07:52

There's so many places to start to train an algorithm here. Talk a little bit about, I guess, what were the most important ingredients to getting rid of that extra hay, those false positives?

0:08:03
Scott Nowson

Well, so the idea of really working with them was also understanding and talking to them about what makes the true positives, what makes that 1%. And what became very clear is each of them was so different and so unique and a combination of so many of the transaction monitoring rules that it really needs the 10 to 20 % should still be investigated to find the 1%. So trying to narrow down that space was just so challenging and so complex.

0:08:37

But at the other end, what looking over the shoulders really does for us is it's feature engineering. And, you know, we don't like to talk about this anymore because it's so time-consuming in the data scientist's life. Yes, yes.

0:08:52
Daniel Faggella

It's where they're bogged down all day long and yada, yada, yada.

0:08:55
Scott Nowson

But you can't do deep learning in this environment because compliance. You have to be able to go to the regulator and say at any point being audited why you missed that. And, yes, there's a lot of progress on understanding the neural algorithms. I think MIT came out with something for image classification that can explain what each neuron is doing in the network. But at this stage, it's like, OK, well, just you take something like a decision tree algorithm, something which can essentially fold out into rules.

0:09:25

And so if we're going to be having to do feature engineering, let's at least learn from the humans what features we're doing and scale them, you know, change in transactions, change in the amount coming in, the amount going out and things like that.

0:09:41
Daniel Faggella

So, yeah, being able to coax out some of those patterns from humans. I guess a bit of the challenge here is that humans are just a collection of their own anecdotes. So knowing if that's exactly the insight we want to scale or not is kind of questionable. So of course, we probably have to then run these algorithms, see, do they do better or worse in terms of false positives, and then see if actually those rules are the ones we want to stick with.

0:10:02

The way I've sometimes described a little bit of how this goes down in fraud is the following. And I'd love you to actually amend this because my job is always to nutshell things for a business audience to make things quickly understandable. And obviously, you're very close to this problem. So I actually prefer your nutshelling to mine. But I've often said in the domains of fraud, we're sort of looking at two things. we're looking at what are the patterns of actual fraud and how can we do better at picking up on when something is fraud again.

0:10:30

And then the second thing we need to do is look at the patterns of normal and we need to better pick up on with a confidence ratio, the divergence from normal, because as you and I both know, there's new ways to do fraud all the time. We can't just train on the existing fraud. There's going to be new kinds. So it's better pattern matching existing fraud and it's better pattern matching variants from patterns of normal. How would you amend that?

0:10:54

Does that make sense as a nutshell, or would there be parts of that you'd like to glom on

0:10:58
Scott Nowson

as you were involved in this project? No, that 100 % makes sense, but I would put it in a temporal order. So what we've typically seen in the deployments we've done, and we've done similar things in banks in Europe, is we start looking at the normal and we start looking at the data and part of it is understanding what's the new normal. So a lot of the transaction monitoring rules are about having a threshold. And finances change over time.

0:11:27

We make more money, we spend more money. At some point, we pass the threshold. It's about looking for the normal and adjusting that. What we've typically done then is we've taken this post-transaction monitoring system and applied it before the transaction monitoring is done. And you're then able to come up with new rules and new ways of detecting what looks like fraud. So exactly to your point that it's the normal and the abnormal, but we've just found there's an order to it.

0:11:58

You start with finding the normal and those mechanisms that you build in, looking at the 99 % of the data will help you build a framework that will then let you be better at finding the 1%.

0:12:11
Daniel Faggella

What is the reasoning behind that temporal order? So I really like the, you know, the fact that you're clarifying that actually there's a little bit more of a structured approach here. What's the reason for beginning there? How does that, how do the lessons there more easily transfer to picking up on fraud versus vice versa?

0:12:31
Scott Nowson

The fundamental reason for me is the 99 % versus the 1%. I've got more to play with while I'm learning about the data, while I'm understanding the domain, the idiosyncrasies of this bank, this customer profile. Because again, you're also going to look at corporate customers, retail banking. You're going to dig in. Do that with the 99 % with the big data set that you've got, because that's also going to be your quickest way to get ROI.

0:13:01

Because I'm going to save you 10, 20, 30, 40%. One of the banks we worked with set out, they want 40 % reduction in the false positives that for them was the business case that's the big saving the other end also that's the risk that's the risk that 1% so they want to have ultimate confidence the stakeholders want to know that you're just messing around at the bottom end you're messing around at the easy end you're proving the value then they have the confidence when you go and mess at the high value huh okay so yeah we obviously have to think about

0:13:36
Daniel Faggella

this in the enterprise. And in fact, one thing I like about our show is that we actually do talk about not just the use case in abstract, certainly not the technology in abstract, but what does it take to get this deployed? And there is the, well, there's the politics and the culture. There's also the winning of stakeholder buy-in. These are all very touchy topics that none of which are easy and frankly, kind of get left out of the discourse, despite the fact that they're often the linchpin between a win or a loss.

0:14:00

If I'm hearing you correctly, what you're saying is, hey, to be able to train on data, come up with some learnings, see some short-term improvements that we can take to leadership and then build. The way I think about this is build more approval to kind of have some more budgets and more team members and ability to continue the project and ability to deploy the project. We need that initial momentum. If I'm hearing you correctly, training on normal where we have the most data and possibly the most ability to come up with some early improvements.

0:14:31

That's the best way to start that early momentum. Am I hearing you right?

0:14:35
Scott Nowson

To me, yeah, perfectly summed it up.

0:14:37
Daniel Faggella

Okay, great. So yeah, in any enterprise project, obviously, and this is one of the tough parts is it's really easy to, well, it's actually not super simple, but if you've got smart enough subject matter people and technical folks, we can draw up what would this project look like? How are we going to roll it out? What are the factors involved? But now we need to look at that whole map and we need to ask, what are the parts of this we can break out that can be done in a relatively short period of time and that can show enough value so that we get the budget to complete this whole roadmap because it's not all going to happen at once.

0:15:09

And so what you're saying is the breakout of that often is let's start with the patterns of normal. Let's show some real lessons learned. Do you need to, in order to get the stakeholder snowball rolling, do you need to bake in some new rules from that machine learning and then have actual users be using it? Do you need to more just come up with a pilot where you can show the stakeholders in leadership? Hey, here's the patterns we've detecting.

0:15:32

We believe these are very promising can we move forward what kind of evidence do we need from that phase one of looking at normal that's going to help unlock the rest of the project

0:15:41
Scott Nowson

so yeah phase one was statistical just looking to see what were the error rates what's the percentage of classification what can we get understanding the tolerance of the stakeholders because you know one mistake one true positive mist that's a potential million dollar fine from

0:15:59
Daniel Faggella

Oh, big time. Yeah.

0:16:01
Scott Nowson

So the statistical confidence first, then it's going semi live, almost beta with the system where you don't interrupt the workflow. Every alert is still investigated by a human, but it's now tagged with what the algorithm would have predicted. So now it's not, oh, well, our historical data says 40%. The people that you trust to do this job now say we can hit 45 % with confidence. If you want to dig in, here's the data. And start scaling up that way, turning on the automation piece by piece, feeding out the insights as you go.

0:16:38

Like, oh, we've done this for two months. Here are some of the new rules that we're seeing. Let us try this. Again, completely into the manual process. We think these are things you should be looking at. get the approval, get the buy-in, talk their language.

0:16:54
Daniel Faggella

Cool. Okay. So yeah, beginning statistically scaling in that regard. In terms of, I'm thinking about transferable lessons from this particular project that might be useful for other folks in financial services. There's a couple of things that have really come through for me. One is the fact that there, at least from your perspective, there is an order in terms of detecting those patterns of fraud and normal. There is a thought process around what is the phase one to show the plausibility, to build the trust, to show the potential of the technology.

0:17:26

And then we can move forward with a more thorough deployment. What other lessons for you do you feel like maybe are worth writing down for other folks in FinServe who want to be able to reduce false positives or want to be able to attack money laundering in this way? I mean, one of the key

0:17:42
Scott Nowson

ones for me was, and it's a little painful at times, and it can slow things down, but it really is getting everyone involved and understanding all sides from the agents to the chief compliance officer. And so long as you can communicate openly between all, then, and you explain what you're doing, why you're doing it, show the value quickly, you know, talk their language. One example of that was we assume, you know, I talk about no errors.

0:18:12

Ultimately, the end in the system that goes live, they want no errors. But we assumed they would also want no errors in the pilot. No, they were able to tolerate some errors. So we could, you know, we hit like in the pilot phase, 20 % false positive identification with no errors. But we said, let's see what happens if we actually try and hit our 40 % target. We were actually able to get with one data set with one particular banking segment, 50 % reduction in false positives, with it turns out only two errors.

0:18:45

So, you know, that's where we were able to say, okay, we can confidently do this. We promise we will reduce those errors. And now we can show a benchmark of a better understanding of the data. And it was presenting back to those all stakeholders all the way through the process.

0:19:02
Daniel Faggella

Cool. Okay. So another worthwhile lesson from the adoption side of things. And yeah, I mean, AI as a team sport, being able to get people on the same page is not easy, but I think the experienced folks with hands-on work in this domain know that getting on the same page expectation-wise, getting on the same page about building towards value and making sure people are going to use it once we build it, absolutely mission critical.

0:19:25

Same in FinService and every other sector. So certainly a worthwhile point for the folks who are tuned in. Scott, I know that's all we have for time on this particular episode, but I'm really glad we're able to have you back on. Thanks for joining us. Thank you again, Dan.

0:19:41
Daniel Faggella

So that's all for this episode of the AI and Financial Services Podcast. A big thank you to Scott for joining us again all the way over from the UAE. And thank you to you as a listener for being able to tune all the way in. I did mention at the top of this episode, I would talk a little bit more about Emerj Plus. Again, as I mentioned, Emerj Plus is really our complete library for folks who need to not only identify high quality AI projects, but be able to communicate the value of those projects to leadership so they can acquire a budget, so that they can move forward with those projects.

0:20:10

And this is where we have our complete library of frameworks and infographics. So if you're interested in a simple infographic for selecting AI vendors that we've taken directly from the head of AI at a Fortune 20 defense contracting firm, well, you can get that in Emerj Plus. If you're interested in our complete framework for AI maturity, drawn from interviews with dozens of enterprise leaders across sectors that you can apply to your clients or to your business.

0:20:35

You'll find that in there as well. And if you want dozens and dozens of use cases across sectors, from telco to manufacturing to banking to insurance and more, again, Emerj Plus is your place to check that out. You can learn more at emerj.com slash P1. That's P as in plus and then the number one. We have members from companies as big as Microsoft and Verizon all the way down to smaller consultants who want to be able to bring proven strategies to their clients and be able to say, hey, here's a framework that's worked for other enterprises.

0:21:03

You guys might want to use this too. So you can learn more at emerj.com slash p1. Otherwise, that is all for this episode. I appreciate you being here as a listener. I look forward to catching you in the next one. You're on the AI and Financial Services Podcast.