In brief
AI is changing how financial firms spot and manage risk, and also how risk spreads through the system. Miriam Fernandez and Sudeep Kesh of S&P Global Ratings argue that generative AI lets firms use messy, varied data for better decisions. But heavy reliance on a few tech providers and fast autonomous systems makes the sector more exposed. Human accountability, transparency and trust have to be built in.
The main tension is between efficiency and exposure. The same digital links that make finance faster also let a cyber incident or a badly managed model spread within milliseconds. Regulators are responding with risk-based rules such as the EU AI Act and DORA, and with more AI-aware supervision. Meanwhile, private institutions are being asked to carry more of the burden of compliance and trust.
Questions
How is generative AI different from the AI banks have used for years?
Fernandez says older machine learning already handled tasks like fraud detection and credit scoring. Generative AI adds the ability to draw insights from unstructured data, such as sensor and climate data, alongside traditional financial data.
What is concentration risk when banks use AI?
Fernandez describes it as the danger of relying on a small group of providers for chips, cloud services, AI models and software. If one of them fails or is attacked, many institutions are affected together.
Why could AI make financial crises spread faster?
Fernandez says autonomous systems, such as algorithmic trading, can become complex and opaque, and they trade in milliseconds. Problems or cyber incidents can then spread through linked banks and firms before people react.
What do the EU AI Act and DORA actually do?
Fernandez says both take a risk-based approach, with tougher requirements where risk is higher. DORA, the Digital Operational Resilience Act, adds duties for critical tech providers, and the EU AI Act requires special risk assessments and adversarial testing for foundation models.
Who is responsible when an AI agent makes a financial decision?
Kesh says humans must remain accountable and he would accept no substitute. He adds that regulators, banks and others need to work out shared definitions of what that accountability means.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Miriam Fernandez said that supervised and unsupervised machine learning have been used in financial risk management for decades, including for fraud detection.
“some types of AI, like supervised and unsupervised machine learning, have been used in risk management functions by decades to either for example uncover hidden patterns in data like it's the case in fraud detection”
0:04:20 · Miriam Fernandez · Machine learning
Miriam Fernandez gave the example of an insurer combining policyholder data with Internet of Things sensor data and climate data to anticipate catastrophes such as river overflow.
“you can consider a company that not only incorporates the analysis from the policyholder, but also could incorporate data from Internet of Things, sensors and devices, and like climate data and climate patterns”
0:05:12 · Miriam Fernandez · Insurance AI
Miriam Fernandez identified a tech oligopoly in chips, cloud services, AI models and AI software as a source of concentration risk for financial institutions.
“the market tech oligopoly that we see around chips, around cloud services, around AI models and AI software”
0:08:56 · Miriam Fernandez · Concentration risk
Miriam Fernandez said banks are digitally linked with clearinghouses, asset managers and fintechs, so a cyber incident at one can spread to others, and generative AI makes such attacks easier.
“banks interact with clearinghouses, with asset managers, with fintechs. And if one of them suffers from a cyber incident, and we know that now with generative AI, it's easier to perform those malicious attacks”
0:09:38 · Miriam Fernandez · Contagion risk
Miriam Fernandez argued that poorly managed autonomous systems such as algorithmic trading could add complexity and opacity and speed up the spread of contagion risk.
“we could see an increase in complexity and an increase in opacity of the workflow. And then this could accelerate the spread of contagion risk because in the end, these are highly fast transactions that happen in milliseconds.”
0:10:24 · Miriam Fernandez · Algorithmic trading
Matthew DeMello said there have been about 15 years of flash crashes driven by technology speeding up the flow of data.
“We've had about 15 years of flash crashes now driven by technology accelerating the flow of data faster than ever.”
0:08:13 · Matthew DeMello · Flash crashes
Matthew DeMello credited Nick Lewis, who leads one of the financial crimes units at Standard Chartered Bank, with describing AI regulation as a shift of responsibility to private institutions.
“Nick Lewis, who leads one of the financial crimes units at Standard Chartered Bank, is through the lens of shifting responsibility.”
0:11:57 · Matthew DeMello · Regulatory responsibility
Matthew DeMello said that in Europe banks are taking on a larger role in tracking know-your-customer (KYC) data and keeping regulators informed.
“In Europe especially, we're seeing banks take on a larger role, tracking know-your-customer data, KYC data, staying ahead of emerging risks and keeping regulators informed.”
0:12:28 · Matthew DeMello · KYC compliance
Sudeep Kesh said the principle of do no significant harm is embedded in the EU AI Act.
“in Europe, for instance, you have this principle of do no significant harm, right? So that's kind of embedded in the EU AI Act, for instance”
0:13:42 · Sudeep Kesh · EU AI Act
Sudeep Kesh said much current regulation is punitive, meaning a wrongdoer pays a penalty to compensate the injured party.
“A lot of regulation right now is punitive in nature. So what I mean by that is that you do something bad and then you have to basically pay a financial penalty to restore the injured party, for instance.”
0:14:12 · Sudeep Kesh · Regulation
Sudeep Kesh said an organization that maintains trust can manage a cyber or AI incident, while negligence erodes trust and can be existential.
“if you maintain trust, it's still manageable. If you're actually negligent, then it tends to erode trust.”
0:16:18 · Sudeep Kesh · Trust
Miriam Fernandez said the EU's Digital Operational Resilience Act (DORA) adds requirements and assessments for critical third-party tech providers to financial institutions.
“We have already seen that with DORA, the Digital Operational Resilience Act, and how, for example, for critical third-party providers of tech for financial institutions, there are additional requirements and assessments to be conducted.”
0:16:58 · Miriam Fernandez · DORA
Miriam Fernandez said the EU AI Act requires special systemic risk assessments and adversarial testing for foundation models.
“with the EU AI Act and the foundation models, for example, they require special systemic risk assessments and adversarial testing”
0:16:58 · Miriam Fernandez · EU AI Act
Miriam Fernandez said regulators are beginning to use generative AI to classify documents and interpret data.
“regulators right now are also starting to use generative AI as part of their processes to classify documents and interpret data.”
0:20:19 · Miriam Fernandez · Regulators using AI
Miriam Fernandez cited John Danielson, a professor at the London School of Economics and specialist in systemic risk, as proposing new supervisory metrics for AI.
“His name is John Danielson. He's a professor from the London School of Economics and specialist in systemic risk.”
0:21:01 · Miriam Fernandez · Systemic risk research
Miriam Fernandez said that if many institutions use the same biased training data, it could lead to harmful market behavior and systemic risk.
“if you're using the same training data and the training data is biased or has some sort of discrimination factors, this could lead to hurting behaviors in the market and therefore systemic risk.”
0:21:36 · Miriam Fernandez · AI bias
Sudeep Kesh said humans must remain accountable for decisions made by autonomous AI systems.
“ultimately, we as humans need to be accountable for these things. And I wouldn't accept any substitute for less than that.”
0:23:12 · Sudeep Kesh · AI accountability
Matthew DeMello identified Daniel Faggella as the CEO and Head of Research of Emerj AI Research.
“On behalf of Daniel Fagella, our CEO and Head of Research, as well as the rest of the team here at Emerge”
0:28:04 · Matthew DeMello · Emerj leadership