OpenNashPodcasts

Person

Peter Piatetsky

Peter Piatetsky is the CEO of Castellum, a company that builds AI agents for financial compliance. On the Fintech One-On-One podcast, he discussed why these agents need a harness around the AI model, how Castellum handles compliance alerts, and how criminals are using AI.

Piatetsky said his first job was as an intelligence analyst in Afghanistan, and that he later became the lead Russia analyst at the US Treasury's OFAC. Castellum reported that top AI models were accurate about 70% of the time without its harness and 99% with it. Piatetsky also predicted near full automation of Level 1 and Level 2 compliance roles within two to three years.

1 episodes · 22 quoted claims

Castellum tested nine AI models both with and without its harness on reading policies, screening and adjudicating alerts.

“we tested nine different models, both with a Castellum harness and without, and we found that the top AI models, when they were asked to read policies and procedures”

Fintech One-On-One · 0:00:10

Castellum reported that top AI models were accurate about 70% of the time without its harness and 99% with it.

“they were accurate only 70 % of the time without a harness versus 99 % of the time with a harness”

Fintech One-On-One · 0:00:10

Piatetsky says the harness raised consistency to 100%, meaning the same alert is adjudicated the same way each time, which bare models did not do.

“the harness greatly improved accuracy, but it also really made consistency goes to 100%. So if you give an agent the same alert three times, is it going to adjudicate it the same way?”

Fintech One-On-One · 0:18:56

Peter Piatetsky's first job was as an intelligence analyst in Afghanistan.

“My first real job was as an intelligence analyst in Afghanistan.”

Fintech One-On-One · 0:02:40

After Russia's first invasion of Ukraine, Piatetsky became the lead Russia analyst at the US Treasury's OFAC, having previously worked on Iran.

“Then Putin invaded Ukraine the first time. And And they were like, hey, who here speaks Russian? It's like me. So they were like, okay, you are now lead Russia analyst.”

Fintech One-On-One · 0:04:18

Castellum collects its own data and gets media directly from publishers, turning it into adverse media data that separates harmless from risk-relevant stories.

“we get media directly from publishers, and then we turn it into adverse media. So we separate, you know, let's say, Russian oligarch donates money to charity from Russian oligarch involved in corruption with construction, for example”

Fintech One-On-One · 0:07:01

Castellum split its offering into separate modules for data, screening, case management and agents so clients could buy only what they needed.

“we decided to take the company in a modular direction. So instead of saying that, hey, you have to buy data, screening, case management, and agents, we separated out the modules.”

Fintech One-On-One · 0:07:49

Piatetsky says organizations are fined for generating too few alerts rather than too many, so compliance managers configure systems to over-generate alerts.

“organizations get fined for not enough alerts, not for too many alerts, when systems are configured, program managers generally configure it to overgenerate alerts”

Fintech One-On-One · 0:09:08

Piatetsky says Castellum's agents let some clients remove more than 95% of alerts accurately.

“are already allowing them to remove, in some cases, 95 plus percent of alerts accurately, which is really important because inaccurately reducing workload is useless.”

Fintech One-On-One · 0:09:42

Castellum's family of AI agents is called Arbiter and receives alerts through an API endpoint.

“that alert then goes to the API to our Arbiter endpoint. Point Arbiter is our family of AI agents.”

Fintech One-On-One · 0:10:36

Castellum routes each alert to a sanctions, politically exposed person (PEP) or adverse media clearing agent.

“the alert gets routed to either a sanctions alert clearing agent, a PEP alert clearing agent, or an adverse media alert clearing agent.”

Fintech One-On-One · 0:11:07

Castellum trains its agent on a client's policies in a proof of concept lasting 10 business days before live alerts are routed to it.

“So we go through this training period, which we do in 10 business days, and then the alerts go to an already trained agent.”

Fintech One-On-One · 0:12:22

Client policies and procedures that Castellum's agents ingest range from about 13 to 145 pages, according to Piatetsky.

“sometimes these are 13 pages, other times they're 145 pages”

Fintech One-On-One · 0:11:40

Piatetsky says Castellum's agent during trials almost always finds cases where human analysts' decisions did not match the client's own policies.

“Interestingly enough, our agent almost always identifies cases where the human analysts whose work we are kind of being graded against have made mistakes”

Fintech One-On-One · 0:11:40

Castellum focuses on US-based banks and fintechs, mainly in the mid-market, though larger tier-one banks have shown interest.

“We focus on U.S.-based banks and fintechs. We are primarily trying to work with mid-market banks and fintechs.”

Fintech One-On-One · 0:14:23

Piatetsky uses Formula One engine suppliers such as Mercedes as an analogy: teams using the same engine still get different results because of the rest of the car, like a harness around an AI model.

“like Mercedes manufactures engines for itself, but also for other cars, for other teams.”

Fintech One-On-One · 0:16:59

Castellum's agents are integrated with LexisNexis Bridger and Perkosoft, and with TRM and Hummingbird.

“our agents are integrated into a number of different systems, including LexisNexis Bridger, LexisNexis Perkosoft. We just announced an integration of TRM and we have an integration of Hummingbird.”

Fintech One-On-One · 0:21:18

Piatetsky describes scam farms in Southeast Asia where kidnapped people are forced to commit elder fraud, and says AI lets such centers run the same scams with fewer people.

“scam farms, especially in Southeast Asia where people are kidnapped, passports taken away, they're told they're going to work somewhere and then they're put into this call center where they have to commit elder fraud.”

Fintech One-On-One · 0:26:43

Piatetsky says criminals create fully synthetic personas to open accounts and commit money laundering, sanctions evasion or fraud.

“people are creating synthetic personas, complete synthetic personas using them to open accounts and then commit money laundering, sanctions, evasion, or just fraud.”

Fintech One-On-One · 0:27:51

Piatetsky defines Level 1 (L1) compliance review as an analyst judging an alert only from the information in it, without outside research.

“L1 is an alert is generated and then an analyst looks at the alert, but they're only looking at what is in the alert. They are not doing external research.”

Fintech One-On-One · 0:28:36

Piatetsky defines Level 2 (L2) review as taking an unclear L1 alert and researching external sources to reach a decision.

“that is basically taking an L1 alert where there is lack of clarity and then going out to certain external sources to do research, come back and help make a decision.”

Fintech One-On-One · 0:29:37

Piatetsky predicts near full automation of L1 and L2 compliance roles within two to three years, along with more effective fighting of financial crime.

“we are going to see near full automation of L1 and L2 roles in the space, but also a corresponding rise in effectiveness in fighting financial crime.”

Fintech One-On-One · 0:30:39