OpenNashPodcasts

Fintech One-On-One

Why AI Compliance Agents Need a Harness, Not Just a Model With Peter Piatetsky, CEO of Castellum

5 Nov 2026 · 32m · Peter Piatetsky, Peter Renton · 23 quoted facts

▶ Play episodeOriginal source ↗

In brief

Peter Piatetsky argues that AI compliance agents need a 'harness' (data, testing, quality checks and each bank's own policies built around a general model) to be trustworthy. Castellum's benchmark found top models were right about 70% of the time alone and 99% with its harness. His case is that banks should weigh building and maintaining that system against buying it.

The tension is between alert overload and accuracy. Banks over-generate alerts to avoid regulator fines, so analysts drown in false positives, while criminals use AI to scale scams. Piatetsky says agents can clear most alerts reliably and expects Level 1 and much Level 2 review to be automated. The open question is who carries the risk and upkeep of that automation.

Key insights

  1. A harness lifted accuracy from 70% to 99%

    Castellum tested nine models with and without its harness on reading policies, screening and adjudicating alerts. Without it, about three in ten alerts were wrong, which Piatetsky says would get a compliance officer fired.

  2. Consistency matters as much as accuracy

    Given the same alert three times, a bare model did not always reach the same decision. Piatetsky says the harness brought consistency to 100%.

  3. The real choice is build-and-maintain versus buy

    Big firms can build agents, but must staff compliance engineers and experts, validate models and retest hundreds or thousands of cases whenever a new model arrives. General counsel's offices, he says, often prefer not to own that risk.

  4. Banks over-generate alerts because regulators punish too few

    Program managers tune systems to fire many alerts so they are not fined for missing something. The result is mostly useless alerts that bury real financial crime.

  5. Agents can slot into existing screening systems

    Castellum's agents run inside tools like LexisNexis Bridger and Perkosoft, so clients keep their screening setup without new model validation. Piatetsky compares it to putting a filter on a tap instead of replacing the pipes.

Questions

Why can't a bank just use Claude or ChatGPT to review compliance alerts?

Piatetsky says Castellum's tests found top models alone were accurate only about 70% of the time and not consistent, versus 99% with a purpose-built harness. He argues that gap is unacceptable in compliance.

What is an AI 'harness' in plain English?

Piatetsky defines it as the infrastructure, data, regression testing, quality checks and specialized instructions put around a general language model to make it purpose-built. He compares the model to an engine and the harness to the rest of the race car.

What is the difference between L1 and L2 alert review?

Piatetsky says L1 means an analyst judges an alert using only the information in it, such as a name mismatch. L2 means doing outside research to resolve unclear cases.

How are criminals using AI right now?

Piatetsky says existing scams now run at 10 to 20 times the volume, and fake IDs, deepfakes and synthetic personas make account opening and wire fraud easier. He says this is why regulators have warmed to AI in compliance.

Will AI replace compliance analysts?

Piatetsky predicts that in two to three years L1 review as a job will largely disappear and much L2 work will be automated. He also expects financial crime fighting to become more effective.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

Castellum tested nine AI models both with and without its harness on reading policies, screening and adjudicating alerts.

“we tested nine different models, both with a Castellum harness and without, and we found that the top AI models, when they were asked to read policies and procedures”

0:00:10 · Peter Piatetsky · Benchmark

Castellum reported that top AI models were accurate about 70% of the time without its harness and 99% with it.

“they were accurate only 70 % of the time without a harness versus 99 % of the time with a harness”

0:00:10 · Peter Piatetsky · Accuracy

Piatetsky says the harness raised consistency to 100%, meaning the same alert is adjudicated the same way each time, which bare models did not do.

“the harness greatly improved accuracy, but it also really made consistency goes to 100%. So if you give an agent the same alert three times, is it going to adjudicate it the same way?”

0:18:56 · Peter Piatetsky · Consistency

Peter Piatetsky ran US compliance for a major Asian bank before founding Castellum in 2019.

“After that, he ran US compliance for a major Asian bank before founding Castellum in 2019.”

0:01:23 · Peter Renton · Founding · 2019

Peter Piatetsky's first job was as an intelligence analyst in Afghanistan.

“My first real job was as an intelligence analyst in Afghanistan.”

0:02:40 · Peter Piatetsky · Career

After Russia's first invasion of Ukraine, Piatetsky became the lead Russia analyst at the US Treasury's OFAC, having previously worked on Iran.

“Then Putin invaded Ukraine the first time. And And they were like, hey, who here speaks Russian? It's like me. So they were like, okay, you are now lead Russia analyst.”

0:04:18 · Peter Piatetsky · OFAC career

Castellum collects its own data and gets media directly from publishers, turning it into adverse media data that separates harmless from risk-relevant stories.

“we get media directly from publishers, and then we turn it into adverse media. So we separate, you know, let's say, Russian oligarch donates money to charity from Russian oligarch involved in corruption with construction, for example”

0:07:01 · Peter Piatetsky · Data sourcing

Castellum split its offering into separate modules for data, screening, case management and agents so clients could buy only what they needed.

“we decided to take the company in a modular direction. So instead of saying that, hey, you have to buy data, screening, case management, and agents, we separated out the modules.”

0:07:49 · Peter Piatetsky · Product strategy

Piatetsky says organizations are fined for generating too few alerts rather than too many, so compliance managers configure systems to over-generate alerts.

“organizations get fined for not enough alerts, not for too many alerts, when systems are configured, program managers generally configure it to overgenerate alerts”

0:09:08 · Peter Piatetsky · Alert overload

Piatetsky says Castellum's agents let some clients remove more than 95% of alerts accurately.

“are already allowing them to remove, in some cases, 95 plus percent of alerts accurately, which is really important because inaccurately reducing workload is useless.”

0:09:42 · Peter Piatetsky · Alert reduction

Castellum's family of AI agents is called Arbiter and receives alerts through an API endpoint.

“that alert then goes to the API to our Arbiter endpoint. Point Arbiter is our family of AI agents.”

0:10:36 · Peter Piatetsky · Arbiter agents

Castellum routes each alert to a sanctions, politically exposed person (PEP) or adverse media clearing agent.

“the alert gets routed to either a sanctions alert clearing agent, a PEP alert clearing agent, or an adverse media alert clearing agent.”

0:11:07 · Peter Piatetsky · Alert routing

Castellum trains its agent on a client's policies in a proof of concept lasting 10 business days before live alerts are routed to it.

“So we go through this training period, which we do in 10 business days, and then the alerts go to an already trained agent.”

0:12:22 · Peter Piatetsky · Onboarding

Client policies and procedures that Castellum's agents ingest range from about 13 to 145 pages, according to Piatetsky.

“sometimes these are 13 pages, other times they're 145 pages”

0:11:40 · Peter Piatetsky · Policies

Piatetsky says Castellum's agent during trials almost always finds cases where human analysts' decisions did not match the client's own policies.

“Interestingly enough, our agent almost always identifies cases where the human analysts whose work we are kind of being graded against have made mistakes”

0:11:40 · Peter Piatetsky · Human error

Castellum focuses on US-based banks and fintechs, mainly in the mid-market, though larger tier-one banks have shown interest.

“We focus on U.S.-based banks and fintechs. We are primarily trying to work with mid-market banks and fintechs.”

0:14:23 · Peter Piatetsky · Target market

Piatetsky uses Formula One engine suppliers such as Mercedes as an analogy: teams using the same engine still get different results because of the rest of the car, like a harness around an AI model.

“like Mercedes manufactures engines for itself, but also for other cars, for other teams.”

0:16:59 · Peter Piatetsky · Harness analogy

Castellum's agents are integrated with LexisNexis Bridger and Perkosoft, and with TRM and Hummingbird.

“our agents are integrated into a number of different systems, including LexisNexis Bridger, LexisNexis Perkosoft. We just announced an integration of TRM and we have an integration of Hummingbird.”

0:21:18 · Peter Piatetsky · Integrations

Piatetsky describes scam farms in Southeast Asia where kidnapped people are forced to commit elder fraud, and says AI lets such centers run the same scams with fewer people.

“scam farms, especially in Southeast Asia where people are kidnapped, passports taken away, they're told they're going to work somewhere and then they're put into this call center where they have to commit elder fraud.”

0:26:43 · Peter Piatetsky · Scam farms

Piatetsky says criminals create fully synthetic personas to open accounts and commit money laundering, sanctions evasion or fraud.

“people are creating synthetic personas, complete synthetic personas using them to open accounts and then commit money laundering, sanctions, evasion, or just fraud.”

0:27:51 · Peter Piatetsky · Synthetic identities

Piatetsky defines Level 1 (L1) compliance review as an analyst judging an alert only from the information in it, without outside research.

“L1 is an alert is generated and then an analyst looks at the alert, but they're only looking at what is in the alert. They are not doing external research.”

0:28:36 · Peter Piatetsky · L1 review

Piatetsky defines Level 2 (L2) review as taking an unclear L1 alert and researching external sources to reach a decision.

“that is basically taking an L1 alert where there is lack of clarity and then going out to certain external sources to do research, come back and help make a decision.”

0:29:37 · Peter Piatetsky · L2 review

Piatetsky predicts near full automation of L1 and L2 compliance roles within two to three years, along with more effective fighting of financial crime.

“we are going to see near full automation of L1 and L2 roles in the space, but also a corresponding rise in effectiveness in fighting financial crime.”

0:30:39 · Peter Piatetsky · Future of compliance

Chapters

  1. 0:02:40From Afghanistan to OFAC sanctions work

    Piatetsky's path through intelligence, the Treasury's sanctions office and bank compliance explains why he founded Castellum.

    “My first real job was as an intelligence analyst in Afghanistan.”
  2. 0:06:23Why Castellum stayed a primary data source

    Collecting its own sanctions, PEP and media data set up the later move into AI agents and a modular product.

    “we did not outsource our data collection at any point.”
  3. 0:13:24How an alert moves through the agents

    Alerts are routed to specialized agents trained on a client's policies and split into many sub-checks.

    “we really break down the alert into as many different components and evaluate each one of those separately”
  4. 0:18:18The harness benchmark and build-versus-buy

    The core evidence for the episode: models alone are far less accurate than models inside a harness.

    “they were accurate only 70 % of the time without a harness”
  5. 0:21:56Adding agents without ripping out screening

    Agents can run inside existing screening tools, which lowers change-management cost.

    “just buy this water filter and put this water filter onto your tap”
  6. 0:25:24Telling AI compliance vendors apart

    Piatetsky admits standing out is hard and points to a market map and benchmark.

    “The hardest thing right now to do in this space is to stand out.”
  7. 0:26:43Criminals scale up with AI

    Volume and synthetic identities are the main changes, and they push regulators toward AI.

    “now occurring at 10 to 20x in terms of volume”
  8. 0:30:39The future of L1 and L2 work

    Piatetsky predicts near full automation of L1 and L2 and a possible drop in financial crime.

    “near full automation of L1 and L2 roles in the space”

Transcript

0:00:10
Peter Piatetsky

We tested nine different models, both with a Castellum harness and without, and we found that the top AI models, when they were asked to read policies and procedures, conduct screening, and create an alert and adjudicate an alert, they were accurate only 70 % of the time without a harness versus 99 % of the time with a harness. So in other words, kind of for a compliance firm, if you're using just Claude, you are at high risk of getting three out of 10 of your alerts wrong, which is kind of what gets any compliance officer fired immediately.

0:00:48
Peter Renton

This is the FinTech One-on-One Podcast, the show for FinTech enthusiasts looking to better understand the leaders shaping fintech and banking today. My name is Peter Renton, and since 2013, I've been conducting in-depth interviews with fintech founders and banking executives. Today on the show, I'm delighted to welcome Peter Piatetsky, the CEO and co-founder of Castellum.ai. Peter began his career as an intelligence analyst in Afghanistan. He then moved to the US Treasury Department, where he worked at OFAC on Iran, and later became the lead Russia analyst.

0:01:23

After that, he ran US compliance for a major Asian bank before founding Castellum in 2019. In our conversation, we talk about why Castellum chose to be a primary source for sanctions, PEP, and adverse media data, and how it grew into building AI agents that resolve compliance alerts. Peter walks through what actually happens when an alert fires, explains why the real debate today is build and maintain versus buy and shares benchmark results showing how much an LLM harness improves accuracy and consistency.

0:01:57

We also discuss how to tell vendors apart, how criminals are using AI at scale and its prediction for L2 and L1 compliance work. Now, let's get on with the show. Welcome to the podcast, Peter.

0:02:20
Peter Piatetsky

Thank you, Peter. I'm going to enjoy saying that.

0:02:23
Peter Renton

Yes, indeed. It's nice to have another Peter on the show. So let's kick it off by giving listeners a little bit of background. I know you've had an interesting career, spent quite a bit of time in government before starting your own company here. So give us some of the highlights of your career to date.

0:02:40
Peter Piatetsky

My first real job was as an intelligence analyst in Afghanistan. That's a heck of a first job. it really is it i think confused me as to what most jobs actually are but that was my first like w2 and i really wanted to do national security and like fight the bad guys and went to afghanistan and it was supporting a special forces unit there and it was unclear to me kind of what our why we were really there nobody could really tell me and i think you'll hear a similar story from anyone that was in Afghanistan, at least from the U.S.

0:03:16

side. And I read about sanctions. And I was like, okay, you know what? This sounds great. You get to fight bad guys. And it's very focused. I mean, that was one of my challenges with our mission in Afghanistan was that I wasn't sure, are we fighting terrorism? Are we building democracy? Are we training national security forces? Are we doing anti-drug trafficking? There was not one clear remission that we could win at. And so with sanctions, I thought, okay, here is something very clear.

0:03:43

There are, let's say, narco traffickers or proliferators or governments or terrorist financiers, they all either need money or use money. So they either need money or they want money. Narco traffickers, they do it for money. Terrorists generally don't do terrorism for money, but they need it. And so having that really narrow scope of the work, I think was really energizing and motivating because I felt like I could measure success. Money frozen, aircraft impounded. These are, in many cases, directly translates to lives saved.

0:04:18

So I worked in Afghanistan, then got a job at the Treasury Department first at OFAC, the sanctions office. And then Putin innovated. I was working on Iran. Then Putin invaded Ukraine the first time. And And they were like, hey, who here speaks Russian? It's like me. So they were like, okay, you are now lead Russia analyst. And worked on Russia, Ukraine, Eastern Europe, and then during the JCPOA back to Iran. But in all of my government work, I found it very difficult to run investigations.

0:04:54

And I found it very backwards that the US and other governments are spending hundreds of millions of dollars to identify bad actors and sanction them and say, hey, here, general public, here are these people that are very risky. But then actually finding that information was incredibly difficult, even for me while I was in the government. And so the origin of Castellum is very much so from my work in government and my frustrations there. After I went to banking, ran U.S.

0:05:27

compliance for a major Asian bank. And I think kind of seeing how compliance technology was used across the private sector really convinced me that there was a lot of room to improve in terms of speed, in terms of accuracy, in terms of ease of usability. and you know now uh here we are because tell them is a seven years old and we have top four u.s bank top three global payments provider top five global crypto exchange and we're really excited to be

0:05:58
Peter Renton

solving financial crime problems for clients right right so when you started and you started started 2019 i believe so that was before the ai revolution was upon us and you know you've certainly um embraced that. So tell us a little bit about what you were doing when you started and how that's evolved as we now live in a world of AI agents.

0:06:23
Peter Piatetsky

So we started as a data company and we initially started with sanctions data. And then we would go to clients and they would say, okay, this is cool, but we need politically exposed persons data. And we added politically exposed persons data. And then, you know, we went back to clients and they said, Okay, this is great, but we also need adverse media. So we want to add adverse media. And I think a this initially felt like it was a mistake, but it's now turning out to have been, you know, very fortuitous move is we did not outsource our data collection at any point.

0:07:01

So we are a primary source. Think global sanctions that ourselves will build our own pep database, we get media directly from publishers, and then we turn it into adverse media. So we separate, you know, let's say, Russian oligarch donates money to charity from Russian oligarch involved in corruption with construction, for example, and then identify who all the bad actors are the company. So we are a primary data source. And I think what we learned is that the compliance data market is actually really extremely difficult to work in because we might go to folks and say, hey, our software and our data is going to lead to 80%, 90 % efficiency improvements if no loss of accuracy.

0:07:49

And people would say, that's amazing. But then there's change management costs. And so I'd be like, okay, well, you just need to integrate and it's this easy. And then that's when people just start ghosting you. And so we decided to take the company in a modular direction. So instead of saying that, hey, you have to buy data, screening, case management, and agents, we separated out the modules. And that's really been huge for the company in terms of meeting clients where they are making it a lot easier to get the product.

0:08:18

and the product that is really taking off right now is AI agents for alert resolution. And I think it's partially because we have narrowed down our focus, like some specifically to AI agents for sanctions, perhaps adverse media alerts, you know, level one, which is resolving the alert with the information available and then level two, which is kind of going outside of the alert and conducting additional due diligence. and it's been frankly like really really exciting when we founded the company we thought okay this is something that we could do and it's possible but now actually seeing our products do this is incredible and i think it's there's an actual sea change coming in the world of financial crime because no

0:09:08

large organization actually knows what is going on inside of it simply due to the volume and you have all of these systems that are designed to tee up alerts, but because organizations get fined for not enough alerts, not for too many alerts, when systems are configured, program managers generally configure it to overgenerate alerts because they're trying to not get fined. Oftentimes more so than like, of course, they want to catch up on national crime, but really what the biggest thing they're trying to avoid is being yelled at by the regulator and the regulator telling them you're not trying hard enough.

0:09:42

And so, as we all know, investigators, analysts are drowning in mostly useless alerts. And so a lot of actual financial crime is not getting caught because of quite simple volume issues. And I think AI agents are going to, at least for our clients, are already allowing them to remove, in some cases, 95 plus percent of alerts accurately, which is really important because inaccurately reducing workload is useless. But yeah, it's really allowing them to spend a ton of time on alerts that really matter.

0:10:15

And that is phenomenal.

0:10:17
Peter Renton

So let's dig into that just for a little bit. Can you take us through the process when an alert fires at one of your clients? What actually happens then and how is sort of the AI agent involved and where is the human involved and when does that happen?

0:10:36
Peter Piatetsky

Some of our clients use Castellum for data and screening, and then they use our agents. And other clients use different screening systems, like let's say TRM or BrateLexisNexisBridger or Hummingbird with integrations with those. I'm happy to talk about that more later. But let's say an alert is generated either in our system or in another system. And so what happens is that alert then goes to the API to our Arbiter endpoint. Point Arbiter is our family of AI agents.

0:11:07

And the first thing is the alert gets routed to either a sanctions alert clearing agent, a PEP alert clearing agent, or an adverse media alert clearing agent. So because we started as a data and screening service company, we have a lot of IP and knowledge around how alerts are resolved. And there are very different risk tolerances between the same alert that matches on a sanctions name is going to have a much higher threshold for clearing it than the same alert, but the matches on adverse media.

0:11:40

So before our agent reviews any alerts, our agent ingests the client's policies and procedures. And sometimes these are 13 pages, other times they're 145 pages. And usually before we even sign a contract to the client, we go through a full proof concept where our agent learns the client's policy and procedures, then does an open book test that alerts with adjudications. And then we can see, hey, is our agent hitting 100? Interestingly enough, our agent almost always identifies cases where the human analysts whose work we are kind of being graded against have made mistakes that don't correspond with the policy and procedures.

0:12:22

So even during kind of the POC phase, we're identifying improvements, But our agent ingests the policy and procedures, our agent takes the open book test, and then our agent takes a closed book test, which is alerts without adjudications. So we go through this training period, which we do in 10 business days, and then the alerts go to an already trained agent. And so if it's a sanctions, perhaps an adverse media alert gets routed that way. And then each one of those agents has dozens of sub-agents.

0:12:52

So there's, for example, a name normalization There's a text normalization agent that will clean up the alert if it's dirty and unreadable or might have a typo in it. Then there's a name normalization agent. Sounds really silly, but for example, is Li Feng Wei written in Mandarin characters the same as Li Feng Wei written in English? Is Muhammad, let's say, Bustrukayev in Russian the same as Muhammad Bustrukayev spelled differently in English? So kind of difficult transliteration phonetic analysis.

0:13:24

location analysis, contextual analysis, severity analysis. So we do not take an alert and just send an alert to an LLM and then say, hey, what do you think? That leads to very low accuracy and very low consistency. And in our space, that's unacceptable. So that's a short answer. I know that was like five minutes, but the agent learns the policy and procedures, the agent gets tested, and then when alerts get sent to the agent, we really break down the alert into as many different components and evaluate each one of those separately and then come back with an answer.

0:14:03
Peter Renton

So as you're talking, it seems to me that every single financial institution needs this technology. But I'm curious about who is your market? I mean, who are you focused on? Are you focused just on the sponsor banks, on the fintechs, on the community banks, on everybody? What's the market?

0:14:23
Peter Piatetsky

We focus on U.S.-based banks and fintechs. We are primarily trying to work with mid-market banks and fintechs. But in practice, we have seen a fair amount of interest from tier ones. And so that is, we're not going to say no to top 10 global bank or top four US bank knocking on our door. So essentially, in practice, we work mid-market and up. But what we've seen is, in one year, the space has gone from AI agents are not reliable enough to do this to, I need AI agents to do this now.

0:15:03

And the only question is, do I have budget for it this year or do I need budget for it next year? and we have seen this from folks that were ai skeptics we have seen this from folks that have laid off people and rehired them and we have seen this from people that have been kind of very bullish on on ai use cases all along and i think what's part of what is ai tools have gotten exponentially better but also we've been able to really show and some other companies in the market as well, being able to show that, hey, agents can clear alerts reliably.

0:15:37
Peter Renton

Right. That is the key, because I think that was the knock early on on some of the AI technology to set and hallucinations with LLMs has made it. A lot of people think that there is a reliability issue. So when you're talking with potential clients, is that reliability piece central or is that debate kind of falling by the wayside now?

0:16:02
Peter Piatetsky

The reliability debate has actually fallen. It's gone. We have very few conversations where people say, I wouldn't trust AI to do this. And if someone says that they don't trust AI to do this, we say, that's totally fine. We're going to go talk to the next person because we have so much demand right now, which is a great place to be. And the question that we have much more often is, why should I not use Claude?

0:16:30

And I think that's a great question. And it's a very reasonable and smart question. And I think we also, whether it's Claude or Aster or Copilot, a lot of institutions have enterprise AI accounts with really large budgets. And they're saying, look, Why don't we just build it? And so the question that we mostly have is what we used to call build versus buy, but what is much more accurate to be called build and maintain versus buy.

0:16:59

And so we have done a fair amount of really detailed data-driven AI research showing what it is like to use an LLM and what it is like to use an LLM with a harness. A harness is a combination of infrastructure, data, regression, testing, QA, QC, specialized instructions, and more that is put around an LLM to make it purpose-built. And for those that follow Formula One, there's a couple of companies that are engine manufacturers, like Mercedes manufactures engines for itself, but also for other cars, for other teams.

0:17:39

But then the teams that buy the engines from Mercedes, they still have varying results than Mercedes. And they will put on different tires, they have a different driver, different wings, brakes, etc. So using Anthropic or OpenAI as the engine, everyone should be using an engine. But then the harness around it is actually what makes the difference. And so for financial crime compliance, that harness includes specialized data, specialized training, specialized regression testing, built-in QA, QC, and the results are enormously different.

0:18:18

So we tested nine different models, both with a Castellum harness and without. And we found that the top AI models, when they were asked to read policies and procedures, conduct screening, create an alert and adjudicate an alert, they were accurate only 70 % of the time without a harness versus 99 % of the time with a harness. So in other words, kind of for a compliance program, if you're using just Claude, you're at high risk of getting three out of 10 of your alerts wrong, which is kind of what gets any compliance officers fired immediately.

0:18:56

That's an unacceptable number. Like three out of a thousand would get someone in trouble, but three out of 10. And so we found that the harness greatly improved accuracy, but it also really made consistency goes to 100%. So if you give an agent the same alert three times, is it going to adjudicate it the same way? And if we're just in a LLM by itself, the answer was no, it would not. So LLMs are incredibly smart and really powerful, but they absolutely cannot be trusted to be given a financial crime compliance problem and to solve it.

0:19:35

They need to understand the policies and procedures of each institution that need to understand the risk metrics. And the harness is really what does that. And so we never say, we work with some very large fintechs that have engineering teams bigger than our whole company. So I would never say to those people, you couldn't build it. But of course they could. It's a question of, do they want to build it, maintain it? Are they going to staff their agent with dedicated compliance engineers and dedicated compliance subject matter experts, are they going to do the model validation?

0:20:10

Are they going to do the regression testing? Every time a new model comes out, whatever is going to power the agent on the back end, it has to be retested on hundreds or thousands of cases and alerts. And so we've actually found the really interesting kind of allies amongst the general counsel's office, where they'll say, like, look, we know my team at Named DigSintech can build this, but do we want to own the risk? Also, it's way more expensive for us to build it and maintain.

0:20:38

So really the question is build and maintain or buy. And that's kind of the debate we're having more

0:20:45
Peter Renton

often now. Everyone has existing sanctioned screening and a whole range of compliance tools at their disposal today that had them for decades in many cases. When someone's deploying your technology, what kind of improvements can they expect? And I know you can, you know, it's going to be different for every system out there, but give us some sense of when Castellan comes in, what happens as far as improvements to their systems typically? I'll talk more so about

0:21:18
Peter Piatetsky

our partnership motion. So we have, our agents are integrated into a number of different systems, including LexisNexis Bridger, LexisNexis Perkosoft. We just announced an integration of TRM and we have an integration of Hummingbird. And there are some others. So when someone adds our agents over the top of an existing screening service, the first thing that they can expect is not to change their existing screening service. That's great from a change management perspective. And the way I like to think about this is if someone comes to you and says, you have lead in your pipes in your house, you can't drink that water anymore.

0:21:56

And there's a number of solutions. And one of the solutions is you rip out all the pipes and you replace them. That's very expensive. You're gonna have to live in a hotel for a little bit. And the Castellum approach is, hey, just buy this water filter and put this water filter onto your tap. It'll filter everything out. You can drink clean water. So that is kind of our agent approach with existing screening systems. Our agent runs inside of Bridger and inside of FERCOSOF.

0:22:21

So what that means is clients can continue to use Bridger, FERCOSOF, or other systems that they have in exactly the same way they were. No new model validations, no new tuning reports, and then just alerts are handed over to the agent. And this is kind of an approach of just saying, let's be realistic about what can be done, about budgets, about change management appetite. This is a way to focus on results first. We're not going to say that, hey, we're going to do this massive reimagining of a whole compliance program.

0:22:56

Usually whoever starts that tends to leave before it's finished. We're saying those are like three to five-year projects at a minimum. But if someone, let's say, has Perkisoft or Bridger and they want to get immediate results, cool our agents are already inside it's like turning it on and that's really our our focus right now is on supporting clients through our channels as we were saying before we hit record

0:23:20
Peter Renton

i was at money 2020 this week walking the the halls of the show there there's there was some you know ai companies talking about it um you know sanction screening and different alerts that their agents can do so you're not the only game in town and it's difficult i think for people to know who's better than who's and who's really not. So if someone's listening to this and sort of say, well, how do I evaluate these companies that all say they're doing something similar?

0:23:52
Peter Piatetsky

How do we tell you guys apart? You know, last year, FinTech meet up and a bank CEO telling me like literally that I don't know how to tell you guys apart with some words in there. And And we have a couple of resources, which I would love if you're able to direct your listeners to. We have an agentic screening market map. So we have done a bunch of analysis and put together a number of vendors, I think over a dozen, that really focus on this.

0:24:22

So we're not including a legacy company that bolts on agents or somebody that is just white labeling like Chad GPT and sending alerts directly to a public unencrypted LLM service. It's who is really focusing on agents. And so we encourage everyone to take a look at the Castellum agent screening market map. That's one that helps people understand who's in the space. obviously someone could say that we're biased, but we do put a bunch of our competitors up there and say good things about them.

0:24:55

And I would also say, I don't view other agentic companies as really competing with us. I see it as we're all going after the same problems and we're really competing against BPOs and consulting firms that are just used to outsource this work somewhere. So I think the agentic screening market map is one, And then we have a thin crime agent benchmark, which I mentioned earlier, where we actually show research like, look, this is what our agents have done.

0:25:24

This is how a LLM without a harness will perform. And I, Peter, I think you're right. The hardest thing right now to do in this space is to stand out.

0:25:33
Peter Renton

well i'll make sure we do link to that in the show notes here okay so i want to talk about the other side the financial criminal side of things because they're they're using ai at scale like like never before they don't have to deal with any compliance rules or anything like that what are you seeing on the offensive side that was not there say two or three years ago that that where the criminals are taking advantage of the of the latest tools so it's uh horrifying and i mean

0:26:02
Peter Piatetsky

that's actually one of the reasons that regulators have really come around to ai in sin crime is because they are seeing criminals use it extremely well and so regulators are basically initially had a lot of discomfort around the hallucinations and is ai reliable and now it's just more so like there is a deluge of AI-powered financial crime. And so the only way to really fight back is to use AI because AI allows you to, done well, allows you to scale up your fin crime operations or anti-fin crime operations, but also allows criminals to scale up their fin crime operations.

0:26:43

So in terms of what we're seeing, the first is scale. So any scam that was occurring beforehand or crime that was occurring beforehand is now occurring at 10 to 20x in terms of volume. I think we've all heard about scam farms, especially in Southeast Asia where people are kidnapped, passports taken away, they're told they're going to work somewhere and then they're put into this call center where they have to commit elder fraud. And those centers don't need as many people anymore.

0:27:14

They're able to run the same scams of fewer people. So I think one is just volume. Two is fake IDs and fake personas. So it's just become that much easier to create fake documentation. People can create a deep fake online. Unless you're touching someone, you may not know that the person that is there is the person who is there. And that creates a lot of issues for account opening, wire transfers, theft of funds. And so, I mean, those are the two primary attack vectors that we're seeing.

0:27:51

One is just, and it's not so much an attack vector, but all the existing scams are now superpowered in terms of volume. And then there's a lot in terms of account opening that is being done that is in the financial crime world because people are creating synthetic personas, complete synthetic personas using them to open accounts and then commit money laundering, sanctions, evasion, or just fraud.

0:28:16
Peter Renton

Okay, so last question then. As you think about the future and how AI agents are going to continue to reshape compliance programs, take us through where you think two to three years out, what the landscape will look like, and where does Castellum fit into that landscape?

0:28:36
Peter Piatetsky

Two to three years out, Castellum continues to be the leading provider of AI agents to banks and fintechs in the U.S. and knock on wood. But I think where we see the space going is that all work that is known right now as L1 has been fully automated. So L1 is an alert is generated and then an analyst looks at the alert, but they're only looking at what is in the alert. They are not doing external research.

0:29:06

They're just like Peter Piotenski matched to Peter Pizatski, false positive, name mismatch. And the reason, again, those alerts are generated in the first place is because compliance program owners tend to bias towards, let's generate more alerts so we don't get fined. Because if we miss something, that's when we really get in trouble. And so with agents, you can continue to generate a fair number of alerts, but reviewing them becomes much less expensive. So I think L1 as a job does not really exist anymore.

0:29:37

And most of those analysts are not going to be working on L2 alerts. So things where you need to do some external research and analysis. Now, I think a lot of L2 work is also going to be automated because that is basically taking an L1 alert where there is lack of clarity and then going out to certain external sources to do research, come back and help make a decision. So that is something that agents are very good at doing, like gathering data, creating a decision point.

0:30:06

But I think what this means also is that I do believe we will see, depending on how this technology is implemented, we could see a real true global drop in financial crime because you would actually see some of the compliance drops at large banks and fintechs become very effective. and for someone to be able to say, hey, I actually know what's going on in my bank fully. That's something that's never been possible for a large bank.

0:30:39

That's only been possible for a private bank where maybe everyone, the bankers all know all the clients. Similarly, it's much more what you would see at a credit union, but even credit unions are trying to grow now. And many have, almost all have online account opening and don't know all the members. So, long story short, Peter, we are going to see near full automation of L1 and L2 roles in the space, but also a corresponding rise in effectiveness in fighting financial crime.

0:31:09
Peter Renton

Well, that is an optimistic and a positive future. Great place to end. Peter, really appreciate you coming on the show. I hope that your prediction comes true and everyone is much more effective at fighting financial crimes. So best of luck and thanks again.

0:31:26
Peter Piatetsky

Thank you, Peter. Pleasure to be here.

0:31:34
Peter Renton

I loved Peter's framing of the impact of compliance AI when it comes to large financial institutions. He said today that none of them truly know what is going on inside their company. Only a small private bank where the bankers know every client has ever had that kind of visibility. Peter believes AI agents could finally give a top-tier bank the same understanding and that this could lead to a real drop in financial crime. That is a bold prediction, but it changes how we should think about AI and compliance.

0:32:05

This is not just about cutting the cost of L1 reviews. If he is right, compliance teams will finally get to do the job they were always meant to do. Anyway, that's it for today's show. If you enjoy these episodes, please go ahead and subscribe, tell a friend or leave a review. And thanks so much for listening. Thank you.