In brief
Machine learning can take over much of the slow, manual work banks do to catch fraud, money laundering and bad customers, but only if the models are watched. Krishna Gade argues that AI is a black box that decays over time, so businesses need continuous monitoring and human oversight from the start.
The tension is between efficiency and trust. Automation cuts labor and errors, yet models cannot be read like normal code and can quietly get worse. Gade points to Zillow's losses as the cost of skipping monitoring, and to his Facebook experience as the origin of Fiddler's transparency tools.
Questions
Why is machine learning good at catching fraud and money laundering?
Gade says the volume of transactions can reach millions, which is too many to review by hand. Models can score each user on money laundering risk using data like location and transfer amount, and flag only those over a threshold.
What does it mean that AI is a 'black box'?
Gade says machine learning models store patterns in structures like neural networks and decision trees that humans cannot read the way they read code. That is why separate tools are needed to explain and validate them.
Why do AI models need monitoring after launch?
Gade says models are not static and can drift and deteriorate as the data changes. A fraud model might start marking bad transactions as good, hurting customers or letting fraud through.
Do humans still have a role if banks use AI?
Yes. Gade says he is a believer in 'human in the loop' AI, where people inspect outputs, check why something was flagged, and make sure legitimate customers are not wrongly classed as fraud.
What happened to Zillow according to this episode?
Gade says Zillow used machine learning to predict house prices without proper monitoring and ended up overpricing homes. He says it caused heavy losses and thousands of layoffs.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Krishna Gade says banks employ thousands of people to monitor fraud and that machine learning could make this work more efficient.
“banks employ thousands of people today to monitor fraud, But AI and machine learning can really enable them to do that efficiently”
0:01:20 · Krishna Gade · Fraud monitoring
Krishna Gade says anti-money laundering models can rank users by a money laundering score and raise an alert when it crosses a threshold.
“The models can rank transacting users according to a money laundering score and erase an alert when the score crosses the threshold.”
0:03:03 · Krishna Gade · Anti-money laundering
Krishna Gade says inputs to money laundering models include user location, the speed of transfers and the amount transferred.
“things like user location, how fast, how much amount that they're trying to transfer. All kinds of these things can go into models”
0:03:42 · Krishna Gade · Model inputs
Krishna Gade says Airbnb hosts and Uber drivers must go through a customer due diligence process.
“If you want to become a host at Airbnb or if you want to be a driver in Uber, you need services, you need to go through a customer due diligence process.”
0:06:57 · Krishna Gade · Customer due diligence
Krishna Gade says modern tech companies use image processing to check whether an uploaded driver license is legitimate.
“they try to use AI and machine learning to, you know, identify, basically use image processing techniques to see if you have uploaded a legitimate, you know, driver license”
0:06:57 · Krishna Gade · Identity verification
Krishna Gade says banks use a process called model risk management to validate models thoroughly before launching them into production.
“So they have this process of managing the risk around models called model risk management, where they validate the models even before they launch to production thoroughly.”
0:11:22 · Krishna Gade · Model risk management
Krishna Gade says machine learning models, unlike traditional software, cannot be read line by line to understand how they work.
“unlike traditional software where you can read the code line by line and understand how they work, machine learning models are these arcane artifacts”
0:11:57 · Krishna Gade · Black box models
Krishna Gade says a machine learning model trained a year, six months or even a month ago may no longer be relevant, depending on the use case.
“the model that was trained a year ago or six months ago or a month ago may not be relevant anymore, depending upon the use case.”
0:12:40 · Krishna Gade · Model drift
Krishna Gade says an unmonitored fraud model that starts flagging bad transactions as good would hurt customers.
“And all of a sudden it starts flagging bad transactions as good. It's going to hurt your customers.”
0:13:13 · Krishna Gade · Fraud models
Krishna Gade says Zillow used machine learning models to predict house prices without proper monitoring in place.
“Zillow used machine learning models to predict house prices and they did not have proper monitoring in place.”
0:14:49 · Krishna Gade · Zillow
Krishna Gade says Zillow overpriced houses, suffered losses and laid off thousands of people.
“they were overpricing the house prices and it just tanked their business. They suffered losses. They had to lay off thousands of people.”
0:14:49 · Krishna Gade · Zillow losses
Krishna Gade says he spent more than a dozen years at consumer internet companies including Bing at Microsoft, Twitter, Pinterest and Facebook.
“I spent about a dozen years, more than a dozen years, in fact, working on consumer internet technology companies, all the way from Bing, Microsoft, to Twitter, Pinterest, and Facebook.”
0:16:27 · Krishna Gade · Career
Krishna Gade says that by 2016, when he joined, Facebook used deep neural networks to rank its news feed.
“by 2016, by the time when I joined the company, we were using very sophisticated, deep neural networks to rank the newsfeed”
0:16:56 · Krishna Gade · Facebook news feed · 2016
Krishna Gade says that after the 2016 elections, Facebook felt it needed to understand how its ranking algorithms worked internally.
“So in the aftermath of 2016 elections, it became quite important for us that, hey, we really need to understand how these algorithms are working ourselves first”
0:17:27 · Krishna Gade · Algorithm transparency · 2016
Facebook built an internal tool called Why Am I Seeing This? to help developers understand how its news feed algorithms worked, according to Krishna Gade.
“we built tools internally at Facebook called Why Am I Seeing This? that helped us internally as developers to understand how newsfeed algorithms worked.”
0:17:27 · Krishna Gade · Why Am I Seeing This · 2016
Krishna Gade says Facebook employees could file a ticket with the news feed quality team if a recommendation seemed irrelevant.
“if they didn't find the recommendation relevant, they could file a ticket or complaint to the newsfeed quality team, which would then get triaged.”
0:18:04 · Krishna Gade · Feedback loop
Krishna Gade says he started Fiddler because large firms like Facebook, Google and Microsoft could build model transparency tools but most organizations could not.
“I felt like, hey, Facebook could build it, Google could build it, Microsoft could build it, but not every organization could build it or may want to build it.”
0:18:31 · Krishna Gade · Fiddler founding
Krishna Gade describes Fiddler as a watchdog on a company's AI that helps confirm it is working as intended.
“it's essentially Fiddler is like a watchdog on your AI and it helps you make sure that your AI is working for you.”
0:10:19 · Krishna Gade · Fiddler AI