Questions
Why are there so many false positives in anti-money laundering?
Banks run monitoring rules that flag anything that looks unusual, so they generate huge numbers of alerts. Scott Nowson said one bank got about 10,000 a month, and typically less than 1 percent were true suspicious activity.
Can AI just find the real money launderers directly?
Scott argues that finding the rare real cases is very hard, since a model that calls everything harmless would be right 99 percent of the time. His approach was to use AI to eliminate false alarms and make the pile smaller for humans.
Why not use the most advanced deep learning models for this?
Compliance requires a bank to explain to a regulator why it missed a case, and deep learning is hard to explain. Scott prefers approaches such as decision trees, which can be converted into clear rules.
How did PwC prove the system worked without risking a fine?
They first measured error rates on historical data, then ran the model beside human investigators, who still reviewed every alert. Each alert was tagged with what the model would have predicted, so stakeholders could see accuracy before any automation was switched on.
How much work did the AI save, and what was the catch?
One bank targeted a 40 percent reduction in false positives, and in one data set and banking segment PwC reached 50 percent. The catch was two errors, which the stakeholders were willing to tolerate during the pilot.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Dan Faggella says financial services companies direct most of their AI spending to risk and compliance functions rather than customer-facing features.
“risk-related functions is where the money is going in financial services, and any bank that tells you otherwise is blowing smoke.”
0:00:30 · Daniel Faggella · Bank AI spending
A bank that approached PwC reported receiving about 10,000 transaction-monitoring alerts a month, of which typically less than 1 percent involved real fraud or money laundering.
“we're getting 10 ,000 a month. And when you look into the data, the amount of actual fraud, of true money laundering, of a suspicious activity is typically less than 1%.”
0:03:51 · Scott Nowson · Alert volumes
Scott Nowson says each alert takes a human agent at least 20 to 30 minutes to review, so clearing the false positives requires a large team.
“with a human agent having to look at the 99 % and each one taking a minimum 20 to 30 minutes, that's a huge team that you need to look at this”
0:03:51 · Scott Nowson · Investigation costs
PwC builds transaction monitoring rules with its forensic experts and implements the systems that watch all transactions passing through a bank.
“we will build, using our forensic experts, some of the transaction monitoring rules and implement the transaction monitoring systems that are essentially watching everything that goes through a bank.”
0:03:20 · Scott Nowson · Transaction monitoring
Dan Faggella says Scott Nowson is based in Dubai, where the PwC AI team he leads is growing.
“You're still out there in Dubai. Your team out there with PwC is certainly growing.”
0:02:44 · Daniel Faggella · PwC AI team
Scott Nowson says that labeling every alert as a false alarm would give 99 percent accuracy, which shows why accuracy alone is a poor goal for finding rare real cases.
“You'll get a 99 % accuracy rate when you say everything's wrong and that just doesn't work.”
0:05:10 · Scott Nowson · Class imbalance
Scott Nowson says he learned to start by understanding the problem while working at Xerox alongside ethnographers, and applied this by sitting with real investigation agents.
“something I learned over the years working at Xerox and working with ethnographers was just understand the problem.”
0:05:10 · Scott Nowson · Research method
Scott Nowson says human investigators typically review only about the last six transactions, while machine learning can scan two years of data in seconds.
“the human can only look at like the last six transactions. The whole point about machine learning is big data. It can look over two years in seconds and it can see much bigger patterns.”
0:06:12 · Scott Nowson · Machine learning scale
Scott Nowson says true positives were so varied that roughly 10 to 20 percent of alerts would still need investigation to find the 1 percent that is real.
“it really needs the 10 to 20 % should still be investigated to find the 1%.”
0:08:03 · Scott Nowson · True positives
Scott Nowson says deep learning is unsuitable for anti-money laundering because banks must explain to regulators at any audit why they missed a case.
“But you can't do deep learning in this environment because compliance. You have to be able to go to the regulator and say at any point being audited why you missed that.”
0:08:55 · Scott Nowson · Explainability
Scott Nowson mentions that MIT produced work on image classification that explains what each neuron in a neural network does.
“I think MIT came out with something for image classification that can explain what each neuron is doing in the network.”
0:08:55 · Scott Nowson · Neural network explainability
Scott Nowson recommends decision tree algorithms for this work because they can be unfolded into explicit rules that regulators can audit.
“you take something like a decision tree algorithm, something which can essentially fold out into rules.”
0:08:55 · Scott Nowson · Decision trees
Scott Nowson says many transaction monitoring rules rely on fixed thresholds, which customers eventually cross as their finances grow, producing alerts.
“So a lot of the transaction monitoring rules are about having a threshold. And finances change over time. We make more money, we spend more money. At some point, we pass the threshold.”
0:10:58 · Scott Nowson · Monitoring thresholds
Scott Nowson says PwC has done similar deployments in banks in Europe, where it begins by modeling normal behavior in the data.
“and we've done similar things in banks in Europe, is we start looking at the normal and we start looking at the data”
0:10:58 · Scott Nowson · Bank deployments
Scott Nowson says PwC typically applies its machine learning layer to filter alerts before the bank's transaction monitoring output is reviewed, and uses it to suggest new detection rules.
“What we've typically done then is we've taken this post-transaction monitoring system and applied it before the transaction monitoring is done.”
0:11:27 · Scott Nowson · System design
One bank PwC worked with set a 40 percent reduction in false positives as the target that made up its business case.
“One of the banks we worked with set out, they want 40 % reduction in the false positives that for them was the business case”
0:13:01 · Scott Nowson · Business case
PwC's rollout began with statistical testing of error rates and then moved to a semi-live beta that ran without interrupting the existing workflow.
“So the statistical confidence first, then it's going semi live, almost beta with the system where you don't interrupt the workflow.”
0:16:01 · Scott Nowson · Phased rollout
In the shadow phase of PwC's project, humans still investigated every alert while each was tagged with the algorithm's prediction.
“Every alert is still investigated by a human, but it's now tagged with what the algorithm would have predicted.”
0:16:01 · Scott Nowson · Shadow testing
Scott Nowson says PwC assumed the bank would want zero errors in the pilot, but stakeholders turned out to tolerate some errors.
“But we assumed they would also want no errors in the pilot. No, they were able to tolerate some errors.”
0:18:12 · Scott Nowson · Error tolerance
In the pilot phase PwC's approach identified 20 percent of false positives with no errors.
“we hit like in the pilot phase, 20 % false positive identification with no errors.”
0:18:12 · Scott Nowson · Pilot results
With one data set and one banking segment, PwC achieved a 50 percent reduction in false positives with only two errors, exceeding the 40 percent target.
“We were actually able to get with one data set with one particular banking segment, 50 % reduction in false positives, with it turns out only two errors.”
0:18:12 · Scott Nowson · Pilot results