# NVIDIA Executive Reveals How AI is Reshaping Financial Services

- **Show:** Banking Transformed with Jim Marous
- **Published:** 28 Oct 2024
- **Source:** https://thefinancialbrand.com/banking-podcasts/
- **Transcript:** https://podcasts.opennash.com/shows/banking-transformed/nvidia-executive-reveals-how-ai-is-reshaping-financial-services-2ce9cf/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Malcolm deMayo of NVIDIA argues that generative AI is moving financial services from experimentation to production, and that banks must build dedicated AI infrastructure, which he calls an AI factory, to capture gains in productivity, operational efficiency and new revenue. Because AI now works in everyday language, far more employees can use it than could use earlier technology.

The main tension is speed versus caution. Banks have decades of AI experience, strict regulation, customer trust to protect, and legacy systems that consume budgets. Yet deMayo says old CPU servers cannot run the new workloads well, talent is now a bigger obstacle than data, and early adopters will gain an advantage. He also expects early projects to merely repave existing processes before reinventing them.

## Key insights

1. **Accelerated computing shrinks day-long bank workloads** deMayo says some large banks' risk and regulatory simulations took 20 hours and NVIDIA's platform cut them to four. That frees banks to use more data and scenarios instead of making compromises.
2. **Talent has overtaken data as the top obstacle** deMayo previewed NVIDIA's upcoming survey, saying talent acquisition now ranks above data as the main challenge. Banks respond with university partnerships, new recruits and retraining, such as S&P Global's Accenture program for 35,000 employees.
3. **Federated learning trains models without exposing customer data** A fraud model can learn from Spanish, French and German transactions and share the knowledge without revealing the sensitive underlying data. This addresses privacy limits that block cross-border model improvement.
4. **Generative AI is already cutting costs and time** Goldman Sachs reports developers accept 40% of AI-generated code, and Klarna reportedly handled two-thirds of calls in a month, cutting resolution time from 11 minutes to two. These are concrete productivity examples.
5. **Legacy infrastructure is slowing banks' AI adoption** deMayo says new AI applications run poorly on older CPU servers, so banks need an AI factory: a shared compute hub that works across cloud, data centers and colocation. He says this also improves resilience against outages.

## Questions this episode answers

### What does it mean that 91% of financial institutions are assessing or using generative AI?

Jim Marous cited NVIDIA's survey showing 43% already use generative AI and 91% are at least assessing or using it. deMayo said the industry is well past the hype stage.

### What is an AI factory and why do banks need one?

deMayo describes it as a shared, high-powered computing hub where teams like wealth management and treasury can build and run AI. He says older servers handle new AI poorly, and a flexible hub lets banks run workloads wherever their data is.

### What is the biggest challenge banks face in adopting AI?

deMayo said talent has just overtaken data as the top challenge, with fierce competition for data scientists and engineers. Banks are responding with training, recruiting and university partnerships.

### How can banks share AI learning without exposing customer data?

deMayo described federated learning, where a model is trained separately on different datasets, such as transactions from different countries, and the knowledge is shared without exposing the sensitive data.

### Will AI replace bank employees?

deMayo argued the opposite: more productive companies become more profitable and hire more people. He pointed to NVIDIA, whose revenue rose from $29 billion to $60 billion while headcount grew under 5%, and described humans as pilots directing AI.

## Facts

- **Malcolm deMayo** [0:02:40]: "We began working with financial services firms 16, 17 years ago. What we do at NVIDIA is we build an accelerated compute platform, and it's taken 30 years to build."
- **Malcolm deMayo** [0:03:15]: "In some of the largest banks, these workloads take 20 hours to run and we accelerate them down to four."
- **Malcolm deMayo** [0:05:25]: "Goldman Sachs talks very much about how they're using generative AI for code generation. And that 40 % of the code being generated is being accepted by their developers."
- **Malcolm deMayo** [0:06:02]: "Klarna over in Sweden using OpenAI's technology, API, to be able to handle two-thirds of their calls in a single month using generative AI"
- **Malcolm deMayo** [0:06:02]: "reducing the average time to earn resolution from 11 minutes to two minutes."
- **Jim Marous** [0:08:15]: "Your study showed that 43 % of financial institutions already are using generative AI and 91%, when you take that and say how many are assessing that capability"
- **Malcolm deMayo** [0:13:48]: "S&P Global announced they hired Accenture to help them train 35,000 of their employees."
- **Malcolm deMayo** [0:14:42]: "JPMorgan Chase has built a program internally to train every new recruit."
- **Malcolm deMayo** [0:16:16]: "We have over 20 ,000 startups have come through our Inception program."
- **Malcolm deMayo** [0:16:49]: "So over 1700 companies are building AI solutions using our platform. So we, we, we actually don't sell our platform directly."
- **Jim Marous** [0:20:45]: "your research showed that 51 % of institutions are focused on identifying additional AI use cases, up from 40 % last year."
- **Malcolm deMayo** [0:23:32]: "an AI factory is something that we've built with Bank of New York and a number of other institutions where they have created an AI hub."
- **Malcolm deMayo** [0:28:18]: "We worked with, and we did a GTC talk last year, Capital One."
- **Malcolm deMayo** [0:32:09]: "In 2012, we built Titan, a supercomputer that took up a tennis court, 17 ,000 GPUs, consumed close to 10 megawatts of power and generated 20 petaflops"
- **Malcolm deMayo** [0:32:41]: "Blackwell fits in the palm of your hand, consumes three quarters of a kilowatt of energy"
- **Malcolm deMayo** [0:33:13]: "Chip Nemo is a chip design AI that works collaboratively with our chip designers."
- **Malcolm deMayo** [0:35:26]: "takes open source models like Llama or Mistral or Cohere or any of them. We NIM them."
- **Malcolm deMayo** [0:36:49]: "we're going to spend probably $8.5 billion this year in R &D."
- **Malcolm deMayo** [0:39:51]: "In 2022, our revenues were $29 billion. The next year it was $60 billion. We added less than 5 % headcount."
- **Malcolm deMayo** [0:39:21]: "he told us it's 4 billion euro expense annually."
- **Malcolm deMayo** [0:19:59]: "Probably one of the most exciting use cases right now in financial services is understanding the COBOL code base that exists in a lot of these large organizations"

## Transcript

**[0:00:09] Jim Marous:** Hello and welcome to Banking Transformed, the top podcast in retail banking. I'm your host, Jim Marous, founder and CEO of the Digital Bank Report and co-publisher of the Financial Brand. In a landscape where 91 % of financial institutions are either assessing or deploying AI in production, Malcolm DeMayo, Global Vice President of Financial Services at NVIDIA joins us on the Banking Transformed podcast. This episode provides an in-depth look at the rapidly evolving AI landscape in financial services backed by NVIDIA's latest industry survey of global financial services professionals.

**[0:00:47] Jim Marous:** We explore why 75 % of institutions consider their AI capabilities industry-leading or at least competitive and discuss the finding that 97% of institutions plan to increase their AI infrastructure spending in the coming year. Malcolm DeMeo also shares his insights on how institutions can tackle key challenges including data privacy and talent acquisition while implementing AI across critical areas such as risk management, portfolio optimization, and fraud detection. Financial services organizations need a comprehensive AI platform to build and scale AI across the enterprise that empowers internal teams and external partners.

**[0:01:36] Jim Marous:** To that end, executives must prioritize high-yield AI use cases to improve employee productivity while delivering superior customer experiences and investment results. So Malcolm, before we start, can you introduce yourselves to our audience and share a little bit about your background?

**[0:01:55] Malcolm deMayo:** Sure, Jim. Thank you for having me here today. I am responsible for the financial services industry here at NVIDIA. And for the last 30 some odd years, I've been working in the banking industry with various technology companies like NCR, Teradata, EMC, Oracle, and now here at NVIDIA.

**[0:02:16] Jim Marous:** So NVIDIA is not a new name to any of our listeners, but it may be new from the perspective of how you're helping financial institutions navigate the landscape to improve scale and speed of AI implementation. Can you discuss just a little bit about the relationship you have with financial institutions to make them more productive in their AI initiatives?

**[0:02:40] Malcolm deMayo:** Sure. It's a great question. So our journey in financial services is not new. We began working with financial services firms 16, 17 years ago. What we do at NVIDIA is we build an accelerated compute platform, and it's taken 30 years to build. It's very different from hardware accelerators. There's lots of them on the market. But we build a full stack. And that full stack solution allows us to accelerate a variety of workloads like high performance compute.

**[0:03:15] Malcolm deMayo:** These are workloads you find very commonly in financial services. Things like options trading where they're looking to discover price. They're looking to do risk assessments and using back testing to develop algorithmic trading strategies. And in insurance for underwriting, these are Monte Carlo simulations. In banking, the regulatory reports like daily regulatory reports like daily credit loss is all simulated. And we accelerate these. In some of the largest banks, these workloads take 20 hours to run and we accelerate them down to four.

**[0:03:51] Malcolm deMayo:** So we've been working with financial services for 17 years. What we do is accelerate workloads. what that enables them to do is they can do more with less or they can start to rethink how the very processes that we're accelerating, they can rethink them and reimagine them. Because when you think about it, when something takes 20 hours, you make all sorts of compromises on the data attributes that you use and the number of scenarios you run.

**[0:04:19] Malcolm deMayo:** When it takes four hours, you can rethink that and hopefully do a much better job.

**[0:04:24] Jim Marous:** And do more with less in many cases.

**[0:04:27] Malcolm deMayo:** Substantially more with less.

**[0:04:29] Jim Marous:** Yeah. Can you share your perspective on what you think the most significant changes that AI has brought to financial services in just the past year? Because I could go back five years, but that's ridiculous because things have changed so fast and so many elements have come into the marketplace. But what do you see as the most significant change you've seen in banking brought on by AI?

**[0:04:52] Malcolm deMayo:** One of the biggest changes with generative AI is that the programming language is our language. So instead of less than 2 % of the world's population being involved in developing solutions, you now have virtually everyone engaged. Banks have to be very careful. AI is not new to them. This is a decades-old journey. And so they have to move in a cautious manner to make sure, and they're very good at this in terms of bringing in new technology.

**[0:05:25] Malcolm deMayo:** But when you think about the virtually hundreds of processes in banks and the fact that the language of banking is our language, that the impact or the potential for impact of AI is enormous. The initial areas, you know, Goldman Sachs talks very much about how they're using generative AI for code generation. And that 40 % of the code being generated is being accepted by their developers. If you think about when you have thousands of developers, 40 % of their work is now automatically generated by AI.

**[0:06:02] Malcolm deMayo:** They're so much more productive. Think of the impact of the bottom line. But more importantly, think about the backlog of work. that they can now get to. So that's one example. Another example is in customer service, you've probably read about Klarna over in Sweden using OpenAI's technology, API, to be able to handle two-thirds of their calls in a single month using generative AI, reducing the average time to earn resolution from 11 minutes to two minutes.

**[0:06:35] Malcolm deMayo:** So when you start thinking about the impact of customer experience, every survey I've read says, If you improve the customer experience, you have the opportunity to sell them more products. So generative AI is going to be a massive, a tsunami-sized wave of change in banking and in financial services.

**[0:06:55] Jim Marous:** You know, it's interesting because the initial use of AI was a lot in the fraud and risk areas where they were trying to anticipate what was happening or be able to build models that would defend an organization. But it's interesting, generative AI is much more of a two-way conversation, if it were, around what you can test, what you can find out, the learnings you can make along the way. And it really has quite a big impact.

**[0:07:21] Jim Marous:** You know, one thing that came out is you just released a study on the deployment of generative AI solutions in financial services. When you look at that report, what was your biggest surprise from the research?

**[0:07:34] Malcolm deMayo:** The biggest surprise is how engaged financial services firms have gotten. And, you know, we're way past the, you know, the hype. Banks and financial services firms are gearing up. They are building, they're trying to recruit talent, which is a very big challenge for them. They're investing in the technologies and they're experimenting and prioritizing use cases. So they understand that this technology can help them to improve productivity, drive operational efficiencies, and even generate new revenue streams. So that was probably the biggest surprise for me.

**[0:08:15] Jim Marous:** You know, it's interesting. Your study showed that 43 % of financial institutions already are using generative AI and 91%, when you take that and say how many are assessing that capability, 91 % of financial institutions are at least assessing or using generative AI. So what applications have you seen that are driving this rapid adoption, particularly in report generation, back office operation, and customer engagement?

**[0:08:44] Malcolm deMayo:** Well, so a big area is customer service in contact center. The opportunity to improve the productivity of a call center agent is massive. Most of these people have to toggle between five or six systems. They're reading text that's unscannable. And this is after someone's waited anywhere from 10 to 20 minutes to speak to them. So, you know, we're building solutions today that allow AI to listen in on the question, scan ahead. You think about our accelerated compute platform, what you have with every node of our platform gives you four research assistants who can read a thousand books a second.

**[0:09:29] Malcolm deMayo:** So when you think about that power. All right. So now you've got this capability to listen in on the question, read down what the customer's history is, understand all of that in the blink of an eye, be able to suggest to the call to the call center agent how to handle the call. At the same time, that research assistant has access to how good calls have gone and all the transcripts are stored. So the AI can help maybe a newer call center agent be more productive sooner.

**[0:10:05] Malcolm deMayo:** So that's one example. Another example, you mentioned fraud. Being able to embed, so in credit card, being able to embed new capabilities like graph neural nets and transformer capabilities into existing models so that model doesn't have to go back through validation, but it can be more accurate in detecting, in identifying anomalies and reducing the amount of work that people have to do on the back end. And we're seeing a lot of interest and a lot of drive in modernizing models.

**[0:10:40] Malcolm deMayo:** And while I'm on that topic, many banks have models in production. They tend to be machine learning models, and they tend to be very small. And like I had said a bit ago, a lot of compromises were made. The amount of data that they're using and the amount of intelligence in the model, there's an opportunity to upgrade those models with more robust machine learning based models and even start to introduce transformer architecture. So generative AI. So the use cases, there are hundreds of them that are driving adoption.

**[0:11:17] Malcolm deMayo:** And at the end of the day, they fall into three categories, improving productivity, driving operational efficiency and creating new revenue streams.

**[0:11:28] Jim Marous:** You know, it's interesting, Malcolm, we've talked about this before in other podcasts we've done, is that one thing with generative AI is you're not only deploying answers to questions, but you're also collecting, as you referenced just a second ago, collecting information for long-term use. Do you see a time when financial institutions may be able to provide maybe an agent on behalf of the consumer that will look for answers to their questions almost before they're asked.

**[0:12:00] Jim Marous:** In other words, we use our GPS systems on our cars regularly, but they've gotten more and more sophisticated based on how they learn our driving habits. Do you see this happening in the banking world where I may be able to call my financial institution and my financial institution will be able to help me through Genrevee I models to build a better financial wellness platform for me?

**[0:12:25] Malcolm deMayo:** on my behalf? I think so, yeah. I think at the end of the day, the future is hard to predict. But banking and financial services has the canonical use case for generative AI. And it's the ability to train an AI to understand every transaction you've ever had with the bank. So, you know, in theory, at some point in the future, you could have an assistant for every customer, an AI assistant for every customer. Think of a future where you have human employees working collaboratively with AI employees.

**[0:13:06] Malcolm deMayo:** It's not that far off.

**[0:13:08] Jim Marous:** Exactly. And I think we're seeing it in some industries right now to a degree, but in banking, there's regulatory and compliance issues as well as safety issues on that. But it's interesting because as you, I mean, I just use it on a personal basis for my business and And the way that the learning tool helps to learn on my behalf is astounding at times to me. So, you know, while financial institutions are starting to dip their toe or their entire foot or more into the water of generative AI, what common challenges do you see banking organizations face when implementing AI solutions?

**[0:13:45] Jim Marous:** And how do they overcome these challenges?

**[0:13:48] Malcolm deMayo:** The most prevalent, and this dates back to the beginning of computer science, is data. But interestingly, I'll give you a sneak preview on this year's survey. Talent acquisition has just trumped data as the top. So there is a fierce competition to bring in data science and data management engineers. and that that's going to be that's going to continue and we're just going to have to continue to either train they're going to have to train their employees retrain their employees and also augment with new new recruits uh and partnerships with universities that's what we're seeing them do partnerships with universities bringing in new recruits and also training their employees i don't know if you saw that uh s &p global announced

**[0:14:42] Malcolm deMayo:** they hired accenture to help them train 35,000 of their employees. JPMorgan Chase has built a program internally to train every new recruit. NVIDIA has a university. It's available to anyone. So there's lots of places you can turn to to get smart quickly. But the other challenges, like the data challenge, we're building inside our platform capabilities to help. We call it federated learning. If you want to train a fraud model on your Spanish transactions, your Spain transactions, and on your French transactions and on your German transactions, and then share the knowledge without exposing the data, the sensitive data, we've enabled that in our platform very securely.

**[0:15:29] Malcolm deMayo:** And so these are opportunities for financial firms to improve the intelligence of their models, improve the ability of their models without exposing sensitive customer information.

**[0:15:45] Jim Marous:** You talk about the whole issue of talent, but isn't another solution, and bring up NVIDIA, isn't another solution working with partners? Because not everybody's been able to develop their own tools internally or their own solutions internally, but the partnership with organizations that are investing in the talent and are bringing multiple case studies together for improvement in the models, isn't that also an option?

**[0:16:16] Malcolm deMayo:** Absolutely. You know, and there's lots of great partners out there. AI is a full-stack problem, and we've built a full-stack solution, but it's also a team sport. So having the right partners is absolutely critical. And what we've done is built a huge ecosystem. of partners, Jim. We have over 20 ,000 startups have come through our Inception program. And for those that listen to your podcast, if you're a startup, it's really easy to join Inception.

**[0:16:49] Malcolm deMayo:** You get cloud credits, you have access to our smart people and all of our processes and best practices. And we even have built a launch pad for you to run POCs to have real world experience. And by the way, enterprises can use that too. So, you know, from a, from a ecosystem perspective, we try to work with everyone that is building AIs. So over 1700 companies are building AI solutions using our platform. So we, we, we actually don't sell our platform directly.

**[0:17:21] Malcolm deMayo:** It's only available through partners. And where, where we'll directly engage is when, by the way, If any enterprise calls us, we'll try to help them. But where we'll directly engage is where there's a really substantial problem, something that's needle moving for the industry and is going to require. It really couldn't be solved without our platform. There's lots of examples of that in banking.

**[0:17:48] Jim Marous:** You know, it's interesting that as we look at that partnership issue, it really makes it so smaller organizations can really do some pretty phenomenal things that used to be only relegated to the biggest and largest and most progressive financial institutions. And I think we're seeing this in every industry is that generative AI and AI in general really allows those organizations that have the right leadership and the right leadership thinking to be able to do as much or more than their larger partner organizations because of the fact that organizations like NVIDIA and others are allowing them to partner to build these solutions at scale, which is kind of exciting when you think about it.

**[0:18:32] Jim Marous:** You know, how do organizations balance AI innovation with regulatory compliance, especially in the financial services field?

**[0:18:44] Malcolm deMayo:** So that's a great question. Regulations exist to protect the consumer and, you know, or protect our society, right, from financial crime. And so banks, I think, don't get enough credit for the work they do for society. But inside of a financial firm, they're constantly balancing. You talked about how the small companies are more agile. They don't have the technical debt. They don't have the legacy systems. And they also don't have the broad base of customers. Right.

**[0:19:28] Malcolm deMayo:** And and there's something that goes along with that, and it's called trust. So the large financial institutions want to make want to make sure that they protect their customer and they and that they remain the trusted house bank or bank of choice. And at the end of the day, it is a big challenge when a large portion of your budget is dedicated to keeping the lights on, running these legacy systems. How do you innovate? Generative AI is helping.

**[0:19:59] Malcolm deMayo:** Probably one of the most exciting use cases right now in financial services is understanding the COBOL code base that exists in a lot of these large organizations, documenting the business practices and helping them think through that transformation to something more agile and modern. So you're going to see a lot more modernization as digitization continues. It's a big trend in banking. I think it's job one for banks to constantly think about how to innovate in terms of their product offering and the experiences that they're in.

**[0:20:38] Malcolm deMayo:** And I don't think if you look at any 10K or any annual report, it's in every one of them. We're focused on improving customer experience, right?

**[0:20:45] Jim Marous:** Yeah, it's interesting. You know, as we get to money 2020 in Las Vegas and the subsequent ones in Amsterdam and in the Far East, we really, it's a sharing environment. It's an environment where people share ideas, what's going on in the industry, what's being tested, and everybody's showing their wares out there. And your research showed that 51 % of institutions are focused on identifying additional AI use cases, up from 40 % last year. What are some of the most exciting new applications you're seeing in AI in the industry right now?

**[0:21:24] Malcolm deMayo:** Well, so I don't know if they're exciting. You know what? If they make money and if they move the market forward, that's good. In capital markets, we're seeing a massive adoption on the buy side of generative AI to build, to bring together structured and unstructured data and divine or find discovers alpha signals, new trading strategy signals. In massive, massive data sets, some of these companies are approaching exabytes or have exceeded exabyte size data sets. And in regular banking, we've talked about fraud.

**[0:22:04] Malcolm deMayo:** There's a lot of focus on financial crime. And financial crime, there are an army of people working in every bank doing a lot of manual jobs where the tool of choice is search, is a search engine. And in many cases, the software is rules based, which is incredibly manual to maintain hundreds and hundreds and hundreds of rules that have to be maintained by human programmers. And they're not adaptive to change. So being able to take a large transaction model and create a large transaction model that can start to augment some of those transaction monitoring systems that have been in place for years to reduce the amount of alerts and really help the bank focus their resources on the real fraud,

**[0:22:57] Malcolm deMayo:** on the real financial crime. These are exciting. I don't know if they're exciting, but these are real, real needle movers for financial services. If you think about the contact center discussion we had, right, that's super exciting. Creating a digital avatar, a digital human that is working together with the contact center agents or with wealth managers or with bankers to help them prepare, to help them answer questions, to do the kinds of things you were talking about earlier, to help provide guidance and education.

**[0:23:32] Malcolm deMayo:** So there's just so many different use cases that are being looked at. The thing that's one of the things, in addition to data and talent, that's slowing financial services down is they need to transform their infrastructure faster. These new applications don't run on yesterday's CPU servers well. And so they need to move to what we're calling an AI factory. And an AI factory is something that we've built with Bank of New York and a number of other institutions where they have created an AI hub.

**[0:24:07] Malcolm deMayo:** That AI hub is a standard place where a standard, essentially massive compute where their various groups, the wealth managers, the settlement, the treasurers, everyone can build, experiment and progress AIs into production. and they're using a stack that would allow the wealth managers, if they want to deploy this on AWS, if they want to run it in their data center, if they're using our platform, which is available everywhere, makes them more resilient, allows them the flexibility of running it where their data is, not where the compute is.

**[0:24:44] Malcolm deMayo:** And so you think about NVIDIA's mission. Our mission is to make AI available, high-performance compute AI and computer vision available anywhere, in cloud, on-prem, in colo, to every enterprise in the world. And so if they're using that platform, they have the ability to say to a regulator, I'm fully resilient. If there's a CrowdStrike-like thing that takes a cloud region down or takes a data center down, I can move these workloads to somewhere else.

**[0:25:12] Malcolm deMayo:** It's available. I'm resilient. I'm not going to destabilize the system. So I think that it's an exciting time, but financial firms need to move faster to create those AI, to create the AI factories and transform the data center of yesterday, which is where we store things to create a factory which generates insights and generates data and content. And that's the big difference. This is the new industrial revolution, data in, insight and intelligence out. So those who get there the quickest will have a big advantage in terms of being able to create better experiences for their customers, to be able to reduce costs and to be able to generate new creative revenue streams.

**[0:26:05] Jim Marous:** You know, it's interesting, Malcolm, because you talk about the external resilience, but you also referenced earlier when we were talking about talent is that the way of thinking about the distribution of data within an organization has really changed. I come from way far back in banking where whenever I wanted information, I had to go to the person who was in charge of data and ask for specific reports. These were green bar reports. They weren't on the computer.

**[0:26:34] Jim Marous:** and you had to analyze them. What you're really talking about here with generative AI and with the AI as it's being formulated going forward is a completely rethinking of the back office where you're really democratizing the insights and the ability to use the data that's there in new and innovative ways, correct? I mean, it's really making it so every single employee will have the ability to access data for their own uses, correct?

**[0:27:04] Malcolm deMayo:** Absolutely. I mean, when you think about when you think about the difference between just search and generative AI, right, you ask a question on search, you get a thousand documents back that you have to read. You ask a very precise question to generative AI, it summarizes those thousand documents into a concise into what you tell it to do. If you want it in a paragraph, it'll give it to you in a paragraph. Now, impose that model on your own corporate data, right?

**[0:27:37] Malcolm deMayo:** And think about all of the different data repositories across a bank. You now have the opportunity to rethink how you price for customers, all right? If a customer is a single product customer, they may get a different rate than a customer that is across your products. And you now have the ability to understand that quickly in real time, as opposed to, you know, there's just been so many barriers to, in the data retrieval era that we both grew up in, right?

**[0:28:18] Malcolm deMayo:** The store and retrieve. And at the end of the day today, it's now take that data and generate insights and ideas and suggestions. So I'll give you an example. We worked with, and we did a GTC talk last year, Capital One. This is, I think, a good example of how we need to modernize models. These people are some of the best on the planet, some of the brightest minds, and they have built incredible models to determine how to serve messages up when customers visit their homepage.

**[0:28:53] Malcolm deMayo:** When you look at a customer authenticating on your website so you know who they are, but your model's not capable of understanding that they visited on Monday, Tuesday, Wednesday, and Thursday, it just isn't smart enough. replacing that with a smarter model that's capable of understanding what you're looking at, what you're interested in, and pinpointing that and making that offer to you. That's all possible today.

**[0:29:20] Jim Marous:** You know, it's interesting, too, because the old way of asking for information was pretty linear, if you visualize it in that way. And what's interesting today is the most skillful people using generative AI tools are those that learn how to prompt, learn how to ask the questions, learn the depth of the question that can be asked to give you an answer, and learn completely new ways of asking questions. You know, we referenced in quite a bit an author we had on the show that actually went the opposite direction what you would usually do and said, you know, if you were to give certain my chapters only a one star out of five, why would this only be a one star?

**[0:30:02] Jim Marous:** As opposed to asking AI to say, how do you improve what we're doing? It asked it what was wrong with what I've said. And it's a different way of viewing, but the ability to understand how you ask these questions to get the most out of generative AI is really a learning process in and of itself, but it really gives power in a democratized basis to the line people, to the people that aren't used to having to ask these questions or get the answers they really need to get.

**[0:30:34] Jim Marous:** So it's going to be interesting to see how it all plays out because we're all still learning. You know, your survey indicated that 97 % of institutions plan to increase the investment in infrastructure spending. What investments do you see that financial institutions must prioritize as they actually

**[0:30:57] Malcolm deMayo:** prepare for the application of AI? Well, I mean, in fact, it's really basic. They need to build the AI factory. They can do it in their data center, or they can do it in cloud, or they can do it in a colo. We're agnostic as to where they do this, but they need to do it. You need to have the right infrastructure. So that means the accelerated compute platform. And what we've built, Jim, is just very simply is a three-layer cake.

**[0:31:27] Malcolm deMayo:** It's hardware, it's software, and it's development frameworks. And they're available. You can access this through the server vendors, through the cloud providers, and also through Colos. We have partnerships with secondary cloud providers as well. What the accelerated compute platform is, The problems we're solving. CPU, the observation Jensen had 30 years ago, 30-ish years ago, was that CPU performance scaling was ending. And it pretty much has, it's pretty much flatlined at this point. In the last eight years, we've accelerated with GPU acceleration by workloads by 1 ,000x.

**[0:32:09] Malcolm deMayo:** Now, let me give you an example of why that matters, all right, why we need an accelerated compute platform. In 2012, we built Titan, a supercomputer that took up a tennis court, 17 ,000 GPUs, consumed close to 10 megawatts of power and generated 20 petaflops of AI performance. The way to think about 20 petaflops of AI performance is that's, by the way, that's a thousand million million floating points. That's 20 research assistants that can read a thousand books a second.

**[0:32:41] Malcolm deMayo:** All right. So fast forward to 2024. Blackwell fits in the palm of your hand, consumes three quarters of a kilowatt of energy, 10 megawatts to three quarters of a kilowatt tennis court to the palm of your hand and generates 20 petaflops of AI performance. So 20 research assistants that can read a thousand books a second. And the ability that think about the ability that that unleashes in an organization that that is run on the English language or on any on any language.

**[0:33:13] Malcolm deMayo:** Right. Your favorite language. So they need to they need to make the investment. And so we built this three layer cake, that hardware layer. We are the only company on the planet that's creating a new architecture every year. The reason we're able to do that, there's a lot of them, but one of them is that we have been using AI inside of NVIDIA. We built Chip Nemo. Chip Nemo is a chip design AI that works collaboratively with our chip designers.

**[0:33:42] Malcolm deMayo:** It helps them to understand what won't work faster. You know, the experimentation process and invention is a lot of fail. There's a lot of trial and error. There's a lot of experimentation and failure to get to what will work. And as a result, we've built, you know, with Hopper and then Blackwell and then Rubin every year, a new, radically more performant architecture. What does that mean? That means training times will go up, will improve by five times and inference times will improve by up to 30 times, which means you'll spend five times less money or 30 times less money, depending on what you're doing.

**[0:34:20] Malcolm deMayo:** The second layer of this accelerated compute platforms and operating system we call NVIDIA AI Enterprise. And it does three things. First, it abstracts the developer from the hardware so they can program in Python, Scala, SQL, whatever they're used to programming in. And as they build their AI models. Right. The second thing it does, we've optimized it to run on the platform so you get the best performance, which means you can do more with less. Right. And then the third thing it does is it's hardened for enterprise so that you don't have to worry about security patches.

**[0:34:53] Malcolm deMayo:** You don't have to worry about we take care of all of that. And we accelerate all three phases of not just AI, but high-performance compute and computer vision as well. But we accelerate in AI the data processing, the model building training, and the inferencing. And probably one of the most expensive workloads that companies don't know they pay for every day is data preprocessing. This is taking the data from multiple source systems, bringing it together, doing complex joins, doing groupings.

**[0:35:26] Malcolm deMayo:** It's deduping the data. It's compressing the white space out of the data. It's extracting HTML and noise out of the data. And we accelerate and provide tools to help improve data quality. So we also have created a training set of capabilities so that really any of the techniques, whether it's fine tuning or parameter efficient fine tuning or RAG, retrieval augmented generation, any of these capabilities are available in our platform. So organizations, and we've built guardrails, and we've built an inferencing engine called NIM, which takes our inferencing engines, takes open source models like Llama or Mistral or Cohere or any of them.

**[0:36:18] Malcolm deMayo:** We NIM them. We put them in a Kubernetes container with our inference engines, and this gives you up to a 5x performance boost in inference. And these are available on ai.nvidia.com to anyone. You can just go and download them. They're microservices-based. That performance improvement means you spend less money. It means you do more with less. And it also allows you to deploy, because it's microservices-based, it allows you to deploy in minutes, not in days.

**[0:36:49] Malcolm deMayo:** So we're constantly – we're going to spend probably $8.5 billion this year in R &D. We're constantly improving the optimization of our platform. The third layer are these development frameworks. You're going to have humans working with AIs as a natural course of business. We're already doing that inside of NVIDIA. That means you can envision a future where a human is in the loop all the time. All right. But AI is also in the loop working to make that human more productive.

**[0:37:24] Malcolm deMayo:** And the human's the pilot. He's directing them, telling them what to do, giving them instructions. And that's kind of a quick glimpse of where we're heading. Wow.

**[0:37:35] Jim Marous:** A quick list of where you're heading is a very long and really dynamic change in the way we do business. It's rethinking the back office, rethinking the front office, rethinking the final results. So finally, Malcolm, what emerging AI applications or just overall the way organizations are going to use AI excites you the most right now?

**[0:38:03] Malcolm deMayo:** I think the idea that it's just the constant innovation and the fast pace is we're immersed in it. It's just incredible to be part of. The fact that we have the opportunity to truly rethink how things are done, it's very, very likely that first application will look a lot like the way we do things today. We'll just sort of repave the road the way it is, optimize the workflow the way it is, and then realize quickly, wow, we can rethink this and we can do this smarter.

**[0:38:41] Malcolm deMayo:** There are processes in these large financial institutions that have been around for a long time. They're commonly called dead processes. Nobody knows why they still do them. There's an opportunity to clean house and really reinvent, not like you said, the front office, the middle office, and the back office. We had a chief operating officer from one of the brands in our office in Santa Clara in our briefing center. And he was telling us that today they have people and this is they have people that spend their entire day just pulling contracts and reading them and answering questions.

**[0:39:21] Malcolm deMayo:** And he told us it's 4 billion euro expense annually. We had another brand tell us that seven out of 10 of the questions they get in their contacts aren't asks for advice. There are questions, there are problems that need to be solved, but not advice questions, not financial advice questions. When you hear these things, you realize just how impactful AI can be. I don't know if it's exciting. I don't know if that's exciting, Jim, but I can tell you that it is impactful.

**[0:39:51] Malcolm deMayo:** And the opportunity, and this is going to happen in a very controlled, safe, responsible way. But the opportunity to improve efficiency, improve productivity and move the needle. I told you about Chip Nemo. Think about NVIDIA. In 2022, our revenues were $29 billion. The next year it was $60 billion. We added less than 5 % headcount. Think about that operating leverage. Some of that is attributed to our ecosystem. Some of that is attributed to the fact that we're leveraging AIs to do the R &D work.

**[0:40:26] Malcolm deMayo:** that normally would have required hiring more people. So at the end of the day, we're going to continue to grow our people base because as companies become more productive and when they become more productive, they become more profitable, they hire more people.

**[0:40:42] Jim Marous:** Yeah. You know, it's interesting. From my perspective, I think the biggest thing I'm excited about is all we discussed today, a lot of it is internal, is making things more efficient, making things more productive. of eliminating steps that are not needed so that people can think beyond where they do today it's going to be the deployment and the feeling from a consumer or a business standpoint of what this means to them i i think we're just starting to scratch the surface on the deployment i i kid a lot of times on podcasts that i know my financial institutions know a lot about me that i would be surprised they know maybe they know what i they think i'm going to

**[0:41:23] Jim Marous:** do next in there and they may be right or may be wrong, but the challenge is, when do I start to feel the impact that the financial institution is having with the use of AI? And I think we're going to start to see the feel of this. We're going to start to see consumers increasing their trust in their financial institutions because of how they're able to help them now with the deployment of AI and generative AI solutions.

**[0:41:49] Jim Marous:** We're going to see a person that maybe has used voice banking and Bank of America, learning that all the conversations they had in the past are now all combined in a way that makes it so that the solution is at their fingertips as opposed to a step or three or four away. I think that's really exciting because it's where it travels through the organization to the consumer, to the small business, to the end user. That's going to be really exciting.

**[0:42:18] Jim Marous:** And to see the role of NVIDIA and actually the involvement of NVIDIA and organization events like Money 2020 and the ability for you to share what you shared with us today on the podcast around what you're doing to make banking better and what banks are doing to make banking better as well. It's really an exciting future. I think we're probably underestimating what will come out of it, but it may take longer than we think because it is banking.

**[0:42:50] Jim Marous:** Malcolm, thank you so much for being on the show today. I really appreciate it. It was great to chat with you today, Jim. Happy to be here. Thanks for listening to Banking Transformed, the winner of three international awards for podcast excellence. If you enjoyed today's interview, please take some time to give our show a five-star rating. Also, be sure to catch my recent articles on the financial brand and check out the research you're doing on the Digital Banking Report.

**[0:43:16] Jim Marous:** this has been a production of evergreen podcasts a special thank you to our senior producer leah haslidge audio engineer chris fifalius and video producer will pritz i'm your host Jim Marous until next time remember the future of banking includes the use of gen of ai waiting to adopt ai could put your institution at a competitive disadvantage

**[0:43:48] Malcolm deMayo:** Where are we going?
