In brief
Malcolm deMayo of NVIDIA argues that generative AI is moving financial services from experimentation to production, and that banks must build dedicated AI infrastructure, which he calls an AI factory, to capture gains in productivity, operational efficiency and new revenue. Because AI now works in everyday language, far more employees can use it than could use earlier technology.
The main tension is speed versus caution. Banks have decades of AI experience, strict regulation, customer trust to protect, and legacy systems that consume budgets. Yet deMayo says old CPU servers cannot run the new workloads well, talent is now a bigger obstacle than data, and early adopters will gain an advantage. He also expects early projects to merely repave existing processes before reinventing them.
Questions
What does it mean that 91% of financial institutions are assessing or using generative AI?
Jim Marous cited NVIDIA's survey showing 43% already use generative AI and 91% are at least assessing or using it. deMayo said the industry is well past the hype stage.
What is an AI factory and why do banks need one?
deMayo describes it as a shared, high-powered computing hub where teams like wealth management and treasury can build and run AI. He says older servers handle new AI poorly, and a flexible hub lets banks run workloads wherever their data is.
What is the biggest challenge banks face in adopting AI?
deMayo said talent has just overtaken data as the top challenge, with fierce competition for data scientists and engineers. Banks are responding with training, recruiting and university partnerships.
How can banks share AI learning without exposing customer data?
deMayo described federated learning, where a model is trained separately on different datasets, such as transactions from different countries, and the knowledge is shared without exposing the sensitive data.
Will AI replace bank employees?
deMayo argued the opposite: more productive companies become more profitable and hire more people. He pointed to NVIDIA, whose revenue rose from $29 billion to $60 billion while headcount grew under 5%, and described humans as pilots directing AI.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
NVIDIA says it began working with financial services firms about 16 to 17 years before this episode and that its accelerated compute platform took about 30 years to build.
“We began working with financial services firms 16, 17 years ago. What we do at NVIDIA is we build an accelerated compute platform, and it's taken 30 years to build.”
0:02:40 · Malcolm deMayo · NVIDIA history
Malcolm deMayo says NVIDIA accelerates some large banks' simulation workloads from 20 hours to four hours.
“In some of the largest banks, these workloads take 20 hours to run and we accelerate them down to four.”
0:03:15 · Malcolm deMayo · Workload acceleration
Goldman Sachs has said that its developers accept about 40% of the code generated by generative AI.
“Goldman Sachs talks very much about how they're using generative AI for code generation. And that 40 % of the code being generated is being accepted by their developers.”
0:05:25 · Malcolm deMayo · Code generation
Klarna used OpenAI's technology to handle two-thirds of its customer calls in a single month.
“Klarna over in Sweden using OpenAI's technology, API, to be able to handle two-thirds of their calls in a single month using generative AI”
0:06:02 · Malcolm deMayo · Customer service
Malcolm deMayo says Klarna's generative AI reduced average resolution time from 11 minutes to two minutes.
“reducing the average time to earn resolution from 11 minutes to two minutes.”
0:06:02 · Malcolm deMayo · Customer service
NVIDIA's survey found that 43% of financial institutions were already using generative AI and 91% were at least assessing or using it.
“Your study showed that 43 % of financial institutions already are using generative AI and 91%, when you take that and say how many are assessing that capability”
0:08:15 · Jim Marous · Survey results
Malcolm deMayo says S&P Global hired Accenture to help train 35,000 of its employees in AI.
“S&P Global announced they hired Accenture to help them train 35,000 of their employees.”
0:13:48 · Malcolm deMayo · AI training
Malcolm deMayo says JPMorgan Chase built an internal program to train every new recruit.
“JPMorgan Chase has built a program internally to train every new recruit.”
0:14:42 · Malcolm deMayo · AI training
NVIDIA says over 20,000 startups have come through its Inception program.
“We have over 20 ,000 startups have come through our Inception program.”
0:16:16 · Malcolm deMayo · Startup ecosystem
Malcolm deMayo says over 1,700 companies build AI solutions on NVIDIA's platform, which is sold through partners rather than directly.
“So over 1700 companies are building AI solutions using our platform. So we, we, we actually don't sell our platform directly.”
0:16:49 · Malcolm deMayo · Partner ecosystem
NVIDIA's research found 51% of institutions focused on identifying additional AI use cases, up from 40% the prior year.
“your research showed that 51 % of institutions are focused on identifying additional AI use cases, up from 40 % last year.”
0:20:45 · Jim Marous · Survey results
Malcolm deMayo says NVIDIA built an AI factory, a shared AI hub, with Bank of New York and other institutions.
“an AI factory is something that we've built with Bank of New York and a number of other institutions where they have created an AI hub.”
0:23:32 · Malcolm deMayo · AI factory
NVIDIA worked with Capital One on modernizing models that decide which messages to show customers on its homepage, discussed in a GTC talk the prior year.
“We worked with, and we did a GTC talk last year, Capital One.”
0:28:18 · Malcolm deMayo · Model modernization
Malcolm deMayo says the Titan supercomputer, built in 2012, used about 17,000 GPUs, drew close to 10 megawatts and delivered 20 petaflops.
“In 2012, we built Titan, a supercomputer that took up a tennis court, 17 ,000 GPUs, consumed close to 10 megawatts of power and generated 20 petaflops”
0:32:09 · Malcolm deMayo · Titan supercomputer · 2012
Malcolm deMayo says NVIDIA's Blackwell delivers 20 petaflops of AI performance while consuming about three quarters of a kilowatt.
“Blackwell fits in the palm of your hand, consumes three quarters of a kilowatt of energy”
0:32:41 · Malcolm deMayo · Blackwell · 2024
NVIDIA built ChipNeMo, an AI that works with its chip designers to speed up design experimentation.
“Chip Nemo is a chip design AI that works collaboratively with our chip designers.”
0:33:13 · Malcolm deMayo · ChipNeMo
NVIDIA's NIM packages open-source models such as Llama and Mistral into containers with its inference engines, which deMayo says gives up to a 5x inference boost.
“takes open source models like Llama or Mistral or Cohere or any of them. We NIM them.”
0:35:26 · Malcolm deMayo · NVIDIA NIM
Malcolm deMayo says NVIDIA expected to spend about $8.5 billion on research and development in the year of the episode.
“we're going to spend probably $8.5 billion this year in R &D.”
0:36:49 · Malcolm deMayo · R&D spending
Malcolm deMayo says NVIDIA's revenue grew from $29 billion to $60 billion while headcount rose less than 5%.
“In 2022, our revenues were $29 billion. The next year it was $60 billion. We added less than 5 % headcount.”
0:39:51 · Malcolm deMayo · NVIDIA revenue · 2022
Malcolm deMayo says a chief operating officer told NVIDIA that employees pulling and reading contracts cost their company 4 billion euros annually.
“he told us it's 4 billion euro expense annually.”
0:39:21 · Malcolm deMayo · Contract processing
Malcolm deMayo says one leading generative AI use case in banking is understanding and documenting legacy COBOL code bases.
“Probably one of the most exciting use cases right now in financial services is understanding the COBOL code base that exists in a lot of these large organizations”
0:19:59 · Malcolm deMayo · Legacy modernization