In brief
Machine learning used in regulated finance and insurance needs independent auditing, which Andrew Clark calls machine learning assurance. Models that decide on loans, credit cards or life insurance affect people's lives, so companies must prove where the data came from, how decisions were made, and that the model stays fair and accurate over time.
The central tension is a stalemate. Data scientists build strong models, but regulators who do not yet know what evidence they need push back, and the models stall. Clark argues regulators must update their understanding and set clear standards. Without that certainty, regulated industries cannot deploy AI the way ad tech and social media have.
Questions
What does machine learning auditing mean?
Andrew Clark prefers the term machine learning assurance: being able to trust that your models do what you think they do and meet business goals. He compares it to a financial audit confirming a balance sheet is accurate.
Why is model drift a problem for banks?
Drift means the real-world data changes from what the model was trained on, for example average credit scores moving over time. Outputs can then differ from what was expected and bias can appear, as Clark says happened with Apple and Goldman Sachs.
How would an insurer get an automated life insurance underwriting model approved?
Clark says it needs documented data sources, sign-offs and controls, plus a history of running in parallel with the existing process to show accuracy and fairness. Regulators must also be able to pick a specific policy and trace how and why the decision was made.
Why do regulators slow down AI projects in finance?
According to Clark, regulators often say they don't understand a model and don't specify what they need, so the project stalls with the data scientists. He says they need things like decision replay, logging, drift monitoring and bias controls.
Is fraud detection audited the same way as credit decisions?
Not as strictly, according to Clark. Fraud flags are correlations that protect consumers, while decisions such as granting a card or setting a limit are explicit and fall under fair lending rules, so auditing matters most there.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Andrew Clark describes the financial industry's three lines of defense as operations doing the work, a second line setting risk frameworks and controls, and a third line of auditors verifying them.
“Third line is the auditors that are actually verifying that these things exist.”
0:08:23 · Andrew Clark · Three lines of defense
Andrew Clark cites a recent problem involving Apple and Goldman Sachs as an example of unexpected outcomes and bias appearing in a model.
“And the problems can arise, and Apple had a problem with this recently with Goldman Sachs, is you can sometimes get unexpected outcomes and bias starts getting introduced into the model.”
0:04:26 · Andrew Clark · Model bias
The UK Information Commissioner's Office released a draft framework for explaining AI, according to Andrew Clark.
“There's the ICO, Information Commissioner's Office in the UK, that has right now released a very good draft framework for explaining AI.”
0:16:25 · Andrew Clark · AI regulation · 2020
Andrew Clark says Senator Cory Booker introduced an Algorithmic Accountability Act in the US Senate.
“There's an Algorithmic Accountability Act in the Senate by Cory Booker a while ago and things like that”
0:17:18 · Andrew Clark · AI regulation
Andrew Clark says the EU was preparing a white paper on AI and that the US was lagging behind Europe and the UK in regulatory understanding.
“EU is starting to do a white paper on that. The White House has done some loose regulations on AI regulation, but US right now is actually lagging behind Europe”
0:16:50 · Andrew Clark · AI regulation · 2020
Andrew Clark was previously the principal machine learning auditor at Capital One.
“He was previously the principal machine learning auditor at Capital One, a company that I happen to respect.”
0:00:31 · Daniel Faggella · Guest background
The episode was recorded during the coronavirus pandemic, which Dan Faggella linked to possible effects on credit data such as high unemployment.
“you and I are going through that right now with this coronavirus.”
0:05:09 · Daniel Faggella · COVID-19 · 2020
Andrew Clark uses a credit card model trained on a range of features such as age and FICO score as an example of a supervised model whose expected input ranges can drift.
“You've trained a supervised model. You know what you're expecting.”
0:03:37 · Andrew Clark · Model drift
Andrew Clark says many insurers want algorithms to speed up life insurance underwriting using actuarial data, which could lower prices for consumers and reduce staffing.
“There's a lot of companies in insurance now that are wanting to speed this process up by the great fact that we have a lot of data now.”
0:12:44 · Andrew Clark · Insurance underwriting
Andrew Clark says insurers must show a history of a new algorithm running in parallel with the existing process before regulators allow it to be sold.
“Then we have to show a history of this running in parallel with your existing process, showing it's accurate, showing it's fair”
0:13:24 · Andrew Clark · Insurance underwriting
Andrew Clark says financial companies often hire employees from Google, so their slower AI adoption reflects regulation, not lack of talent.
“A lot of times these financial companies are poaching Google employees.”
0:18:16 · Andrew Clark · Talent
Andrew Clark says many major financial corporations had already started deploying machine learning for fraud detection.
“there's a lot of major financial corporations that have already started deploying machine learning for fraud detection”
0:19:55 · Andrew Clark · Fraud detection
Andrew Clark argues that banking, insurance and healthcare are regulated while social media and ad tech largely are not, and that this explains the difference in AI innovation.
“the regulators are regulating these areas and they don't have the understanding to let the companies innovate versus social media and ad tech and things.”
0:18:16 · Andrew Clark · Regulation
Andrew Clark runs Monitaur, a company that provides auditability and governance for machine learning, which the transcript renders as Monitor.ai.
“Andrew Clark is our guest this week. He runs a company called Monitaur.”
0:00:31 · Daniel Faggella · Guest background