OpenNashPodcasts

Banking Transformed with Jim Marous

How Ally Bank is Using AI to Reshape Financial Services

29 Apr 2025 · 41m · Sathish Muthukrishnan, Jim Marous · 22 quoted facts

▶ Play episodeOriginal source ↗

In brief

Ally Bank's technology chief argues that a digital-only bank earns trust through security, consistency and simple experiences. Ally built its own AI platform early, tied to a clear goal: every customer should feel like the bank's only customer. It combined cloud data, embedded compliance staff and several AI models to move fast without losing customer confidence.

The central tension is speed versus safety. Ally moved quickly on generative AI after ChatGPT appeared, yet built a layer that strips personal data before it reaches outside models. Sathish also warns that small fintechs can now build quickly with AI. He says Ally must keep re-examining its own business, and that experimentation on problems others call too hard is how it stays ahead.

Key insights

  1. Security came first because there are no branches

    Sathish says a digital bank cannot ask customers to walk into a branch, so protecting data and meeting expectations consistently is how trust is built. Security was therefore the first of Ally's six technology pillars.

  2. Ally built its own AI platform in two months

    Ally AI connects to outside language models but removes personal information, logs traffic and restores the context afterwards. That answered the concern about sending data to outside models and produced a working use case by early 2023.

  3. Using several AI models beats relying on one

    Sathish says Ally AI can connect to many models and combine their answers. It uses different models for content creation, code generation and risk assessment, matching each to the job.

  4. Embedding compliance early avoids rebuilding later

    Ally put compliance, risk and legal staff into requirement writing and design reviews through its technology operating model, called Adam. Sathish says this is a small hurdle compared with building something and then having to dismantle it.

  5. Small AI-driven fintechs are the real competitive threat

    Sathish says a company with fewer than 20 people can reach $100 million in annual recurring revenue quickly. He talks to venture funds to learn where startups are heading and asks how he would rebuild Ally from scratch with AI.

Questions

How did Ally use generative AI without exposing customer data?

Ally built its own platform, Ally AI, that sits between the bank and outside language models. Sathish says it removes personally identifiable information, tracks all traffic, and restores the details afterwards so the response still fits the customer.

Why does Ally use more than one AI model?

Different models are better at different tasks, so Ally picks the best fit for each use case. Sathish says it uses different ones for content creation, code generation and risk assessment, and can combine answers from several.

What does Ally mean by wanting every customer to feel like its only customer?

It is the bank's stated goal, according to Sathish. Customers compare banks with the best experience they have had anywhere, such as Amazon or Uber, so Ally uses cloud data and applications to personalize experiences in real time.

How does Ally handle compliance without slowing innovation?

It involves compliance, risk and legal staff at the start, when requirements and designs are written, rather than after the build. Sathish says this works through Ally's technology operating model, called Adam.

What did Ally learn from its COVID-era auto loan deferral offer?

Ally let customers defer auto loan payments for six months through a fully digital process that rewrote the contract. Sathish says customers who took the deferral still kept making payments, which built loyalty.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

Sathish Muthukrishnan says Ally was started right after the 2008 financial crisis and is the largest direct-to-consumer digital bank in the US.

“Ally was started right after the worst financial crisis the country has ever seen in 2008. We are the largest direct-to-consumer digital bank in the US.”

0:02:01 · Sathish Muthukrishnan · Company history · 2008

Sathish Muthukrishnan says that about five years before the interview Ally still ran on mainframes, used hardware-based networking and had six different apps.

“it would be hard to believe that five years ago, we were still running on mainframes. We still had network that was plug and play, hardware enabled. We had six different apps.”

0:03:13 · Sathish Muthukrishnan · Legacy technology

Sathish Muthukrishnan says about 75 percent of Ally's applications run in the cloud.

“Having all our applications on cloud, now we have about 75 % of our applications running on cloud.”

0:08:37 · Sathish Muthukrishnan · Cloud migration

Sathish Muthukrishnan says about 95 percent of Ally's enterprise data is in a single cloud-enabled data warehouse.

“Having all our data in a single data warehouse that is cloud enabled, I have about 95 % of the enterprise data in the cloud.”

0:09:09 · Sathish Muthukrishnan · Data warehouse

During the peak of COVID, Ally offered auto loan customers a six-month payment deferral.

“So anybody that had an auto loan, we offered them to defer their payments for six months.”

0:10:52 · Sathish Muthukrishnan · COVID relief · 2020

Sathish Muthukrishnan says customers who accepted Ally's COVID auto loan deferral kept making their payments anyway.

“the consumers that took the deferral never deferred their payments.”

0:11:34 · Sathish Muthukrishnan · COVID relief · 2020

Sathish Muthukrishnan says Ally originates about $45 billion in auto loans a year and works with about 21,000 US car dealerships.

“We do about $45 billion a year, have about 21 ,000 U.S. car dealerships that we have a relationship with.”

0:14:48 · Sathish Muthukrishnan · Auto lending

Sathish Muthukrishnan says 90 percent of Ally's Invest customers come from its bank deposit customers.

“And 90 % of our invest customers come from our bank deposit customers.”

0:15:55 · Sathish Muthukrishnan · Ally Invest

Ally's one-click Invest account opening required three pages because of FINRA regulations, according to Sathish Muthukrishnan.

“We said one click, but for FINRA regulations, we had to go through three pages.”

0:15:55 · Sathish Muthukrishnan · Account opening

Sathish Muthukrishnan says Ally created its own technology operating model, nicknamed Adam, that embeds compliance staff in requirement writing.

“we created our own technology operating model. We called it Allies Technology Operating Model.”

0:17:34 · Sathish Muthukrishnan · Operating model

Ally has an internal technology lab that tests new technology with a business outcome and a stakeholder from outside technology.

“we also have what we call an allied technology lab.”

0:22:52 · Sathish Muthukrishnan · Innovation lab

Sathish Muthukrishnan says an Ally board member asked him about ChatGPT on the morning of a board presentation, before he had heard of it.

“One of the board members literally that morning is like, did you hear about chat GPT?”

0:22:23 · Sathish Muthukrishnan · ChatGPT · 2022

Sathish Muthukrishnan says Ally's leadership team spent three days in Seattle meeting Amazon and Microsoft after Microsoft's investment in OpenAI.

“we need to go sit in Seattle. So my entire leadership team and I, in November 2020, not the ideal time to visit Seattle when it's really cold.”

0:23:55 · Sathish Muthukrishnan · Microsoft partnership · 2022

Ally AI removes personally identifiable information, logs transactions and restores the information afterwards when connecting to an external large language model.

“It would remove PII. It would keep track of all of the transactions going in and out. The ability to rehydrate PII so it'll have the context for Ally consumers that are using it.”

0:25:07 · Sathish Muthukrishnan · Ally AI · 2023

Sathish Muthukrishnan says Ally built its Ally AI platform in two months.

“Very simple. We built it in two months.”

0:25:07 · Sathish Muthukrishnan · Ally AI · 2023

By early 2023 Ally had a use case running on Ally AI using GPT-3.5 on its own private VPN on Microsoft Azure.

“by beginning of 2023, we had a use case that was working on Ally AI using 3.5 GPT running on our own private VPN on Azure.”

0:25:41 · Sathish Muthukrishnan · GPT-3.5 deployment · 2023

Sathish Muthukrishnan says Ally uses different large language models for content creation, code generation and risk assessment.

“Our content creation LLM is different from what I use for code generation. and that is different for what I use for risk assessment.”

0:27:26 · Sathish Muthukrishnan · Multi-model AI

Sathish Muthukrishnan says a company with fewer than 20 people can quickly reach $100 million in annual recurring revenue.

“a company with less than 20 people can quickly build something and can very quickly have a $100 million ARR”

0:29:19 · Sathish Muthukrishnan · Fintech competition

Sathish Muthukrishnan says Ally customers with savings buckets save twice as much as they normally do.

“people save twice as much as they normally do when they have a savings bucket”

0:33:17 · Sathish Muthukrishnan · Savings buckets

Ally uses generative AI to create five-to-six-line summaries of its financial education articles on its Conversationally platform, which increased engagement.

“we simply took that entire article and we created a five to six line bullet of this is what the article is talking about. And it was done by generative AI.”

0:36:04 · Sathish Muthukrishnan · AI summaries

Ally has a product called Plans that shows customers how to pay down debt across multiple accounts.

“We have rolled out a product that is called Plans that essentially tells you how you can pay down your debt across your multiple accounts”

0:35:24 · Sathish Muthukrishnan · Debt tools

Sathish Muthukrishnan paraphrased a blog by the founder of Anthropic as saying a world of geniuses in a data center can be accessed through an API.

“you now have access to a world of geniuses in a data center that you can enable through an API.”

0:38:22 · Sathish Muthukrishnan · AI outlook

Chapters

  1. 0:02:01Ally's digital-first origins and transformation

    Sathish explains why Ally started in 2008 and how it moved off mainframes and six separate apps. This sets up the technology strategy that made later AI work possible.

    “Ally was started right after the worst financial crisis the country has ever seen in 2008.”
  2. 0:08:00Six pillars and the North Star

    Sathish lays out the pillars that start with security and ends with his goal that every customer feels like the only customer. It frames the rest of the discussion.

    “My North Star is very simple. I want every consumer of Ally to believe and experience as if they are our only customer.”
  3. 0:10:52Digital advantage: COVID loan deferrals

    A concrete example shows how a fully digital process let Ally help customers faster than banks that required phone calls.

    “My team was able to build and deploy something that was 100 % digital”
  4. 0:17:34Embedding compliance through the Adam model

    Ally brought compliance and legal into design from the start, which reduced rework and kept experiences simple.

    “we embedded our compliance folks with our story writing folks.”
  5. 0:24:30Building Ally AI after ChatGPT

    The story of the board member's ChatGPT prompt, the Seattle trip, and a privacy-protecting AI platform built in two months.

    “we built an AI platform that had the ability to connect to an external LLM”
  6. 0:29:19Staying ahead of fintech competition

    Sathish describes experimenting on hard problems, watching big banks and venture funds, and fearing that AI-native startups could outpace incumbents.

    “What do we know right now is a company with less than 20 people can quickly build something”
  7. 0:33:17Engagement, savings buckets and AI summaries

    Examples show how relationship-based engagement, such as savings buckets and AI article summaries, helps customers meet financial goals.

    “people save twice as much as they normally do when they have a savings bucket”
  8. 0:38:22What's next: a world of geniuses

    Sathish closes with optimism about AI as an accessible resource for small teams, and Jim stresses that culture and a clear destination decide the winners.

    “you now have access to a world of geniuses in a data center that you can enable through an API.”

Transcript

0:00:10
Jim Marous

This is the first of this year's Executive Leadership Series. Last year we interviewed six different individuals from the financial services industry about their transformation. Very candid, very unscripted, very dialogue-based. This year's going to be the same except we have eight guests. Our first guest from Ally, one of the most progressive financial institutions in the banking industry. Also, one of the first that I really saw integrating AI and generative AI within their banking system. They built a very early relationship with Microsoft, which gave them a little bit of a head start.

0:00:53

Our guest today was on my podcast about a year ago and talked about that partnership. We're going to take it to the next level now and talk about what's next. So I'd like to introduce my guest today, Sathish. I love it. Very good.

0:01:18
Sathish Muthukrishnan

Right after Kevin O 'Leary.

0:01:20
Jim Marous

It's a hard act to follow. What a great presentation. I do like the kicks. I was coming close to wearing my red, black, and white ones today, but I couldn't get into the changing room quick enough. Thank you. You know, we've talked before, and I think one of the most interesting dynamics of what you do is having to look forward at the implementation and integration of AI within the Ally Bank ecosystem, both on the customer side, but also back office.

0:01:48

Can you talk a little bit about the beginning of that journey? Because it's not that long, even though it probably seems like years and years for you. and what brought on your integration with Microsoft and what you've done in the last few years.

0:02:01
Sathish Muthukrishnan

Yeah, first of all, congratulations on a phenomenal event. This is my first time here and I can feel the energy. Thank you for having me, Jim. Thank you. Just for people that don't know, Ally was started right after the worst financial crisis the country has ever seen in 2008. We are the largest direct-to-consumer digital bank in the US. So the reason we started was not because consumers needed a bank. They needed a better bank.

0:02:33

So from day one, transformation and making it better has been in our DNA. So when I started with Ally five years ago, I was thrilled to be leading technology and product in an all-digital platform. And Ally had invested in the foundational capabilities, but looking at what is the next thing to do is what led to this transformation that we started. And as you see, consumers are always looking for magic from any institution that they interact with.

0:03:13

So looking back, it would be hard to believe that five years ago, we were still running on mainframes. We still had network that was plug and play, hardware enabled. We had six different apps. For a digital bank, we had six different apps. Consumers were kind enough, we still had a high retention. Consumers were highly satisfied. But the moment of truth was fast approaching. And COVID hit and it accelerated everything.

0:03:50
Jim Marous

Yeah. You know, it's interesting because you talk about your journey at Ally and it's relatively short, but suffice to say, a lot has happened since you joined. When you started at Ally, generative AI was not yet here. AI was usually used for fraud and risk more than anything else in the banking world. What was your mission when you joined Ally from a standpoint of what you personally wanted to achieve for your organization?

0:04:20
Sathish Muthukrishnan

Yeah. The intent behind launching what we called a technology strategy was to make sure that technology will continue to be relevant in an all-digital bank. but more importantly, can we create the differentiation and drive significant business outcomes? For that to happen, our technology strategy had to integrate with business strategy. So we took a step back and said, what can we do to achieve that? And we categorized into six different pillars. You know, physical bank, people walk in, they interact with people, have a discussion, and you start to establish trust and relationship.

0:05:09

But in a digital bank, you don't talk to people. The best way to establish relationship is to give consistency, is to make sure you meet your customers' expectations. More importantly, you protect their data. So security became our first critical pillar.

0:05:28
Jim Marous

Because your brand can break down immediately. Exactly. If you break the cardinal rule of don't mess with my data. That is exactly right.

0:05:36
Sathish Muthukrishnan

That's exactly right. I can't ask somebody to go into a branch and ask them to bank, right? So at all costs, I had to protect my bank. So security became our first transformational pillar. The second thing that we forced ourselves to do, what happens if you have your entire organization focusing on security? You can clamp them down. You can mitigate innovation. So we said we want to be the best bank that offers the best and differentiated experiences for our consumers.

0:06:11

So our second pillar was, how can I drive tremendous experiences? And the third pillar is, how do I know my experience is working? That's when data analytics came in. Measure what consumers do, but more importantly, measure what they don't do. Being a digital bank, I know exactly where the consumers are perusing, where they're dropping off, where they're spending more time, where my experiences are kludgy, and how can I simplify that. So data analytics critically helped. So now I've talked through three pillars.

0:06:49

How do I enable these pillars? I need to transform where all of these experiences run. So in came our operational pillar, migrating to the cloud, driving automation, driving consistency in how we develop and deploy code. And then we said, all of this, for it to be true, I need to preserve my culture. And more importantly, I need to take care of my talent. So that laid the pillars, those pillars laid the foundation for where we transformed our, where we started our transformation and where we are going right now.

0:07:28
Jim Marous

if you're saying it simply, what would be your North Star at Ally today? Because you heard me this morning, and I talked about the fact that one of the mistakes people are making right now is really not knowing what their specific destination has to be, which can change over time, but very much like a GPS system. If you don't know your destination specifically, you may get there, you may not, but it's left up to chance. What is different about the way that Ally's defined your North Star?

0:07:59

Yeah.

0:08:00
Sathish Muthukrishnan

My North Star is very simple. I want every consumer of Ally to believe and experience as if they are our only customer. It's very easy to say, extremely difficult to achieve because what am I fighting against? Consumers are expecting magic, as I said earlier. Their bar for experience is the last best experience they had anywhere, not just with banking. You know, you could have had your best experience shopping at Amazon, flagging an Uber, booking a restaurant, or walking into a restaurant and having a meal.

0:08:37

Now, when you're doing banking after that, they come in with the mindset of, I am expecting the same experience from the bank as I had my last time. So for me to achieve that, my North Star is for my every consumer to behave and believe that they are our only customer. Now, to achieve that is where we have to unpack what we achieved through our transformation. Having all our applications on cloud, now we have about 75 % of our applications running on cloud.

0:09:09

Having all our data in a single data warehouse that is cloud enabled, I have about 95 % of the enterprise data in the cloud. What that allows me to do is learn from consumer behaviors, learn and understand what they're expecting, constantly allow us to get better every day, but have the advantage of applications running on cloud and be able to create those experiences real time so my consumers think that they are our only consumer.

0:09:39
Jim Marous

So there are other digital banks out there, but there's also, like 95 % of the people here, digital plus locational. How do you at Ally compete against or transform to make it so you have a humanized banking experience in a digital world? Because that, as we know, once COVID came, we all tried to do it better. But as you mentioned, the differentiation is, how do I differentiate without emphasizing the benefits that maybe a dual network has?

0:10:17
Sathish Muthukrishnan

Yeah. Isn't that the challenge? Sometimes you feel like, oh God, if I had my consumer in front of me, I can explain what's going on. I can actually explain why they have to go through 40 pages of filling out this application before I even make a decision. But I believe our digital footprint is the advantage that we own. So I'll give you an example. So in the peak of COVID, we saw that our consumers were depressed and we wanted to offer them assistance.

0:10:52

So anybody that had an auto loan, we offered them to defer their payments for six months. I know that. Thank you for being a customer. Now, you can't do that just by sending them an email. You have to rewrite the contract. My team was able to build and deploy something that was 100 % digital, where the consumers came in, looked at the offer, and said, this is what I expect my financial institution to do. So they were able to rewrite the contract digitally, and it allowed them to defer their auto loan for six months.

0:11:34

Compare that to what the other institutions did. They wanted to help out the customers. They didn't have a digital solution, but they offered another solution, which is, can you call us and we will make sure we'll defer it. You call and you're waiting for days to talk to somebody. So it's not an ideal situation. The intent is right, but the execution is missing. Now, what we found out was the consumers that took the deferral never deferred their payments.

0:12:07

They liked the plan B, but they still wanted to continue making the payments for the bank. So it was a tremendous win-win for us, showing up for our customers when they needed us most. And then you're driving the loyalty with our consumers.

0:12:23
Jim Marous

So a lot of your business starts, if I'm not mistaken, from the lending side. And you have since brought in more deposit services and you don't push those. you offer those on a regular basis based on transactions. I mentioned to you that, you know, once we set up this interview, I started saving all my ally emails, but they're all very different from what I get from my traditional financial institutions. Number one, they are more frequent. Number two, they are highly targeted.

0:12:57

Number three, they refer to what I already have with you and talk about what could service me in the future. And there's always an offer of some nature. That sounds like the old pushing out things all the time, but it never feels like that. How do you build that type of communication where you bring all this data together in such a way that it's not selling? It seems like a mentality that says,! We're going to try to make it so that you improve.

0:13:29

you're taking the vested interest in my financial wellness. What is the method that that goes about? Does it start with data or start with communication or a mixture of both?

0:13:39
Sathish Muthukrishnan

Yeah, we have an ambition. We have an ambition to be your best financial ally in the nation. How do we bring that ambition to life is by not giving you banking speak. Even if you go through our products and then consume our products to the experiences we offer, there is not a lot of TNCs. And I have to thank my colleague, Andrea Brimmer.

0:14:08
Jim Marous

She's been a guest on the podcast.

0:14:10
Sathish Muthukrishnan

Yeah, a phenomenal human. And she simplifies the complex like nobody else does. and she forces and partners with technology to make sure that we bring that to life through the experiences that my team, digital team is building. The advantage that I have is the product team that dreams about the features, the UI UX team that designs the experience and the technology team that builds it all part of the same technology organization. So we challenge each other to make sure we simplify the experience that our marketing teams dream up.

0:14:48

And what they dream up is, how can I make this as simple as possible? You said we start many of our relationship through lending, and that is because we are one of the nation's largest auto loan producers in the country. We do about $45 billion a year, have about 21 ,000 U.S. car dealerships that we have a relationship with. We also see a ton of consumers coming to us to open regular banking products, whether that is savings, CDs, et cetera.

0:15:26

We also have an invest platform. So we took it upon ourselves to say, what could be the minimum number of clicks for our deposit customers to open an invest account? And I think we spoke about it when we had our first podcast. And we called it one-click account opening. So if I already know you as a consumer, and if you have shown intention to invest and build your wealth, can I open your invest account through one click?

0:15:55

And that was a challenge that we took on. And 90 % of our invest customers come from our bank deposit customers. So it's a subset of the bank customers that come in. We said one click, but for FINRA regulations, we had to go through three pages.

0:16:11
Jim Marous

but it still was way better than four clicks oh what's interesting I got to think about this that is close to the really it's better than numbers that getting an apple card was which I thought was the epitome of customer experience because you didn't they had already pre-filled nine-tenths of it they never knew me but they pre-filled it and they gave me the approval almost immediately But four clicks, I don't think you can get lower than that.

0:16:42

With government regulation, speaking of government regulation, you talk about internally and Andrea and what she does and her team does from a requirement of we've got to get to this, this, this. How do you deal with what is perceived to be the compliance issues that everybody has to deal with? Because obviously you have the ability to view compliance issues in a different way. Maybe it's bringing compliance in the front end of the game. which we already we've talked about that but on top of that viewing things from a risk management as opposed to risk avoidance perspective not putting as many as you can not making it so nobody can read it yes we all scan screen but the reality is

0:17:24

you don't have to do that with yours correct how do you get it down and stay within what your organization believes to be compliant It was hard work.

0:17:34
Sathish Muthukrishnan

We had to make it systemic. So to make it systemic, we created our own technology operating model. We called it Allies Technology Operating Model. Makes sense. Short for Adam. So we internally fondly call it as Adam. And that is where we embedded our compliance folks with our story writing folks. It was such a subtle difference in when you engage compliance and risk folks to changing their mindset and saying, I want you to be a part of writing the requirements.

0:18:14

So you're not coming after and telling me what I did wrong. Now you're actually guiding me, making me go take the right direction. And we created a small but mighty group within my UI UX organization. What they do is they take the experience. One part of it is compliance, making sure we are building the right code. The second part of it is building out the experiences, and they work with our legal compliance and risk folks. They take the user experience before it is being built and coded and walk through our control folks to say, this is what we are trying to build.

0:18:57

Now, in some instances, we have north of 60 people sitting there opining on the user experience. Ultimately, the compliance, risk, and legal folks are also humans. They also want to contribute to the betterment of the bank. And we have to say, okay, this is the business folks and the marketing folks that tell us what the UI UX should be. I want you to tell me the compliance requirements that are required to overlay this. So there is a little bit of ground setting that happens that you have to overcome.

0:19:33

But that's a small hurdle to overcome as opposed to building something.

0:19:38
Jim Marous

And having to destruct it. You know, it's interesting. If anybody's seen my hurdle of my podcast, watching my podcast, see me just this morning even talking about it. if you have compliance, I worked in marketing my, my virtually my whole life in some way or another. If you let compliance come into the end, they will find something wrong. If you bring me in the beginning and have them part of the solution and vested in the solution, they will be your ally.

0:20:06

I'll put another plug in for you, but it's, it's the biggest thing that can get in the way is the unwinding of thought process. And it's destructive to the culture because it gives them a bad name when they may not deserve it. But even worse, it makes it so that people feel at risk to voice what they'd like to see. I mean, Andrea would be deflated if she didn't have the ability to think this was going to get through.

0:20:32

Correct. But she also, I know from interviewing her, she has enough flexibility in her mindset that says, I'm not steadfast. I'm not saying this way or the highway. But the reality is it's that working together. Yeah. So we bring up to November of 2022, gender of AI gets introduced to the public, but you were already working with Microsoft. Yes, yeah. What let you know that this was going to be one of the biggest changes you had seen at Ally ever since, you know, the implementation of data and AI within Ally was.

0:21:06

What made you move that way and what's kept you on that path? Yeah.

0:21:11
Sathish Muthukrishnan

Yeah. You know, it's the pursuit of what is next. We as a technology team, we're significantly empowered to go do what is right for the future of the company. One of the things that I love about Ally, there is several things. The focus on long-term strategy and the effort, intent, investment, and execution against that long-term strategy never wavers. So as I talked about, we had our cloud strategy. We had our data in the cloud warehouse.

0:21:48

And at that time, beginning of 2022, I had requested to go completely modify our network because we have tremendous amount of data. Our applications are running on cloud. I can't have somebody carrying this data in a wheelhouse, right? Like you need to be software enabled. So we redefined our network. When we did that, we were saying, what else could we do? And frankly speaking, we were thinking about AI and we had launched our chat assistant called Ally Assist.

0:22:23

Yet another example of Ally doing it right, not giving it a proper name, but letting our consumers know this is a chatbot and you can chat with this chatbot. And if we can't answer your question, we'll seamlessly transfer you to a live person, which we have 24 by 7 live support in the U.S. So as we were thinking about it, all of a sudden it coincides with our board meeting. One of the board members literally that morning is like, did you hear about chat GPT?

0:22:52

And I'm like, I'm preparing for the board presentation. I haven't heard about chat GPT. You're supposed to know this already. And then one thing led to the other, and I'm like, oh, my God, this is, you know, you see something, and you're like, this is something that I have to experiment. Just to give a little bit of context, we also have what we call an allied technology lab. The goal of allied technology lab, it's a simple three-part execution.

0:23:21

One is you need to be experimenting in a new technology. And that new technology has to have a relevant business consumer outcome. And then you have to have a stakeholder supporting you from outside of tech. So that was the goal of the Ally Tech Lab. And I asked them, okay, this is a newer technology. Can we start experimenting with it? And we saw this moving really fast, really fast. And then we saw the Microsoft investment in OpenAI.

0:23:55

And we said, we need to go sit in Seattle. So my entire leadership team and I, in November 2020, not the ideal time to visit Seattle when it's really cold. We went there for three days, sat with Amazon, sat with Microsoft, and started asking questions. At that time, now they've all caught up. I felt like Microsoft was way ahead of their game. And I'll tell you what those three days of investment helped me do. I came back to the organization and said, we need to build something using generative AI.

0:24:30

But I see this fast evolving because Amazon has a different roadmap versus what Microsoft has. So collectively as a leadership team, we decided to create what we call Ally AI. because what we knew then was technology is fast evolving, but the narrative is I am scared of you sending data to any LLM because everybody in the world would know. So to avoid those conversations, we built an AI platform that had the ability to connect to an external LLM, but it did a number of things.

0:25:07

It would remove PII. It would keep track of all of the transactions going in and out. The ability to rehydrate PII so it'll have the context for Ally consumers that are using it. Very simple. We built it in two months. Going back to my trip to Microsoft as we were building and showcasing this, we now had to go write a contract. Imagine two corporations writing a contract. Ally, it's a new contract to use LLMs. Microsoft is like, we are not changing our corporate contract.

0:25:41

But leaning in on those conversations and the investment we made over those three days, they were able to accommodate several of the red line that my legal team did. And we were able to move fast. And by beginning of 2023, we had a use case that was working on Ally AI using 3.5 GPT running on our own private VPN on Azure.

0:26:08
Jim Marous

You know, it's interesting because in our discussion about a year and a half ago, it became very clear that you were educating Microsoft as well as Microsoft educating you. They had to learn more about banking. They wanted to be in the banking industry. You wanted open AI or at least a generative AI platform. But we've been now two and a half years, three years into the game. There's a lot of providers out there. Do you use alternative providers as well to build this better platform in your Ally AI?

0:26:43
Sathish Muthukrishnan

Yeah, I'm telling you, if I can go back and revalidate one of the decisions we made was to build Ally AI. Not only it has the ability to API into 3.5, all the way from Flan to Cloud to Bedrock, we have several LLMs that are connected to the single platform. And not only are we able to pick and choose the right fit LLM for, depending on the use case, now I can combine answers from several LLMs and pick what is best for Ally.

0:27:16

For the purpose. For the purpose.

0:27:18
Jim Marous

Fit for purpose. Some are much better at one element than they are at others. Some are better back office providers. Some are better communication providers. That's exactly right.

0:27:26
Sathish Muthukrishnan

Our content creation LLM is different from what I use for code generation. and that is different for what I use for risk assessment. So several use cases using several different models.

0:27:40
Jim Marous

So you are in the cutting edge slash leading edge of what's going on. Who do you look at or what do you look at to keep you ahead of the game personally and your team ahead of the game? Because it's not like most of us, including myself, that are learning from the leaders You have that leading spot, and there's not necessarily somebody out there that is ahead of you. You're trying to catch up with. You're trying to be the leader.

0:28:09

Where do you look to get your inspiration on the what's next?

0:28:15
Sathish Muthukrishnan

I'll categorize it into two things. One is executing, and the other one is experimenting. I get a lot of inspiration from what we are experimenting with. What do we have to experiment with? We have to experiment with the problems that everybody across the globe says it's extremely difficult to solve. You cannot solve it. Or the models are too expensive. Or the models are too complicated. Because what do we know? Six months from now, that model is going to be 100 % less expensive.

0:28:51

It's going to advance so much faster that the problem that I am saying cannot be solved. you will have a way to solve it. So that gives me a lot of inspiration. And we are looking at several complex problems that we would love to solve. Now, who do I look at? I look at all of the big banks. Obviously, you know, I mean, look at what Chase is doing. Look at what Bank of America has done and Wells Fargo has done.

0:29:19

They have advanced their Gen AI journey. They've rolled out Gen.AI to several of their employees, as is some of these fintechs. What do we know right now is a company with less than 20 people can quickly build something and can very quickly have a $100 million ARR, and they're working harder than you to come and compete. So if all of these fintechs went and did a list of, let me look at all of these big corporations that are not performing well and have a poor net promoter score.

0:30:02

A poor net promoter score generally says, I don't like the product that you're offering. And if they said, how can I use AI to restart this business from ground up? I guarantee you somebody's looking at it. So that is my fear. I go back and look at how can I passionately re-underwrite what Ally does, starting with AI, and see how I can beat some of these fintechs coming my way. So how do I get my insight?

0:30:29

I go sit with all of these venture funds, and I talk to them and say, where is your money going?

0:30:36
Jim Marous

I love it. So they tell me. Where do I have to be next? Yes.

0:30:40
Sathish Muthukrishnan

So they tell me and bring me all these companies that they're investing in. And, you know, what I heard an amazing nugget is in the history of investing, they haven't seen as many companies rise to profitability in less than six months as they're seeing right now. And every one of those companies is working on something that AI is using to solve a critical problem for the consumers.

0:31:04
Jim Marous

A couple, several nuggets in that sentence response. One of them is you aren't asking AI to tell me, how do I get better? That's the only one. The one is, what am I doing wrong that I need to improve upon? Correct. I interviewed an author once that was combining a book with multiple authors as part of that. And instead of asking AI, how can I improve each one of these chapters? he asked in each chapter what is one flaw in this chapter that people will go against in other words what what's wrong with this chapter a complete reverse what we were used to doing this was two years ago he did the interview but it was interesting because you go

0:31:46

if you take the reverse you get completely different answers you know exactly how do i improve what i'm doing then they use what you're doing now as the foundation as opposed to what am i doing wrong by my net promoter score. What's also interesting, you talk more and more about not just the experience from a standpoint of ease and simplicity, but we're hearing more and more about the need to generate engagement. I see it in how you're communicating to me.

0:32:11

It's a whole lot different than it was when I got the loan seven years ago. The reality is it seems like you're pushing for the ongoing dialogue that we could have. Why? What is the end game of that? I know it's not the sale by itself. It's more than that. But what is that end game of that engagement that goes beyond simply a good, easy to use app? Yeah, what a great question.

0:32:39
Sathish Muthukrishnan

If we do a thought experiment right now, what is the most personal thing to anybody? It is their finance. And finance drives emotiveness. It makes you happy. It makes you sad. It makes you anxious. And if our ambition is to become the best financial ally for you, it can be transactional. It cannot be transactional. It has to be relationship-based. I need to understand, in most cases, anticipate why you would come to ally. So I'll give you a simple example.

0:33:17

We rolled out what we call savings buckets. Savings buckets is the old school, I have an envelope, I'm saving money. I have multiple envelopes, I'm saving money, right? It's your deposit account, and then it has multiple, multiple savings buckets. It's a very simple mindset shift of you are allowed to name your own bucket. It could be I am saving to buy a pet or I am saving to go on my honeymoon. What we have found is people save twice as much as they normally do when they have a savings bucket, which I translate.

0:33:59

If I translate into emotiveness, we are encouraging and influencing our consumers to achieve their goals twice as fast. What is better than that? Yeah. I mean, banking is a noble profession. And if I can enable my consumers, my customers, to achieve their goals twice as fast, and if technology is the tool that is enabling it, and I'm just giving you one simple example, what is wrong in creating this continuous communication that continues to add value, that showcases what your bank can do to be your best financial ally?

0:34:36
Jim Marous

so we're at a time right now in the economy and in the world where fear is driving a lot of things you talked about the aspirational things I talked earlier this morning about the fact that I had my biggest savings account ever but it was built with acorns where little bits of money were taken out every month in different ways almost like Christmas Club plus plus the Bank of America you know roundup and and it generated the highest savings account I'd ever had not real proud of the In fact, that was my highest, but I saw that as being aspirational.

0:35:07

Today, there's a lot of fear in the marketplace. There's credit that's going off the charts. People are having to pay down low. How does Ally work on that side to actually encourage or work with a customer on their financial wellness journey to unwind challenges that they're facing?

0:35:24
Sathish Muthukrishnan

Yeah, I mean, ultimately, we are a business. You know, we have to be, because we are investing our shareholders' money, we are answerable to our investors. So we need to be responsible in terms of lending money. But what we have done is we have this platform called Conversationally, where we are writing financial articles to educate consumers on how can you be better at credits. We have rolled out a product that is called Plans that essentially tells you how you can pay down your debt across your multiple accounts that you have.

0:36:04

And we are constantly educating them. And we found that this requires a lot of text. And consumers don't have a lot of patience in reading through it. In comes Gen AI. What we did was we simply took that entire article and we created a five to six line bullet of this is what the article is talking about. And it was done by generative AI. If you like it, you can click it. We see significant increase in engagement and the amount of time that they are spending because they now know what the article is going to provide.

0:36:39

That's a subtle way of... If you click to expand it then? That's exactly right. It's a very subtle way of educating our consumers in driving responsible financial behaviors.

0:36:51
Jim Marous

So we're running out of time. What's next? Without giving away any trade secrets, but what do you see maybe in the industry or at Ally? Where do we go from here?

0:37:02
Sathish Muthukrishnan

Yeah. I'll tell you. Let me start with the industry. I feel like what a time to be alive. Yeah. You see opportunities everywhere. You see technology that will help break down those problems and build solutions for your consumers. Your consumers are engaging with your platforms. And their expectations are constantly improving, creating challenges for your teams to go solve for it. So, as I said before, if you open your mind and if you start seeing some of the roadblocks that are creating friction in your experiences and you start to question yourself, why do they exist?

0:37:45

How can I go solve them? And I keep this in the experimentation bucket and solve the hard problems. What is going to happen is your brand is going to become more relevant to the consumers. you are going to become the next best experience that they ever felt. And that's going to deepen the relationship. And ultimately, they'll show up in business results. So what a time to be alive.

0:38:11
Jim Marous

You know, it is interesting because there's only upside, really. I mean, we take care of the downside. We get ourselves in the way all the time with regard to excuses and things that are going on.

0:38:22
Sathish Muthukrishnan

Maybe I'll add, I'll just quickly add. I'll just quickly add the founder of Anthropic. he had in one of his blogs, he had written a big blog. But ultimately, when I was thinking through it and simplifying it, all it said was, you now have access to a world of geniuses in a data center that you can enable through an API. And that world of geniuses can actually help you solve problems that you never had access to before.

0:38:50

You don't need to hire thousands of thousands of people. With a small team, you have access to this world of geniuses called generative AI. And you can ask and ask the help, like you said, even ask it to define the problem as well as help you solve the problem.

0:39:06
Jim Marous

That's a takeaway for the audience, the people who are live in the podcast. You know, I think one of the things to really embrace here is you mentioned earlier that you can be a very small 25-person shop and replicate where Ally is today. Scary easy. It keeps you up at night because it goes, how do I keep ahead of the sheriff? Because the sheriff is coming after us. Yes. How do you build the brand so that people realize you're on their side?

0:39:36

You're their ally. Great name to have. You've leveraged it very well. How do you get people engaged on an ongoing basis so they feel connected to who you are and they feel like you're connected with them? way ahead of the game, but not unachievable, especially if you're a smaller organization who can be nimble. I cannot say this enough that the organizations that are doing best are the largest, and in many cases the smallest. And we have a lot of organizations in this room today and in the conference of various sizes, but the hurdle appears massive, but it takes a culture it takes leadership and it takes a knowing your North Star and what you want to achieve so that every

0:40:28

day when you sign into work you're working towards that goal Sathish, thank you again for being on the show this is a gentleman who knows what he's doing but he's a great person to talk to me because he breaks it down I do encourage everybody to look in the Ally app it's not hard to break down how they do what they do and to understand what sets them apart. Great, great time. Thank you very much. Such a pleasure.

0:40:57

Thank you, Jeff.