In brief
Ally, a digital-only bank, built its own generative AI platform, Ally.ai, on a private copy of OpenAI's models running in Microsoft's Azure cloud. The argument is that banks can adopt generative AI safely and productively if they control the platform, protect customer data, start with internal uses, and treat AI as an assistant that helps staff rather than replaces them.
The central tension is speed versus caution. Ally moved quickly, with legal contracts turned around daily, while stripping out personal data and keeping nothing in the model's memory. Both guests say culture, leadership willingness to take managed risk, and employee education decide success more than the technology does, and they stress keeping humans involved in the work.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Sathish Muthukrishnan began his career at Singapore Airlines in Singapore before moving to the United States to work for United Airlines.
“I grew up in India and started my first job with Singapore Airlines in Singapore and got to a point where I got offered a job to move to U.S. with United Airlines.”
0:02:04 · Sathish Muthukrishnan · Career
Sathish Muthukrishnan worked at American Express before becoming Honeywell's first chief digital officer.
“Then had the opportunity to move into a completely different vertical industrial working for Honeywell as their first chief digital officer.”
0:02:36 · Sathish Muthukrishnan · Career
Sathish Muthukrishnan describes Ally as a digital-native organization, the largest direct-to-consumer digital bank in the U.S. and the largest auto loan originator in the U.S.
“a fully native, digital native organization, the largest direct-to-consumer digital bank in the U.S., and also the largest loan originator for autos here in the U.S.”
0:03:11 · Sathish Muthukrishnan · Ally profile
Sathish Muthukrishnan says Ally was the first bank to offer rewriting or deferring loans entirely digitally during COVID.
“We were the first bank to introduce rewriting or deferring the loans 100 % digitally.”
0:04:49 · Sathish Muthukrishnan · COVID response · 2020
Ally runs its own private instance of OpenAI models on Microsoft Azure that no one else can access.
“We have our own private instance of OpenAI on Azure, which itself is terrific because we know that nobody else has access to that instance.”
0:20:28 · Sathish Muthukrishnan · Architecture · 2023
Priya Gore says the Azure OpenAI service brings OpenAI's models into Microsoft's Azure environment so customers can use them in their own secure tenant.
“The Azure OpenAI service is basically a secure service leveraging all of the great capability that OpenAI has developed and brought forward to the market”
0:19:12 · Priya Gore · Azure OpenAI · 2023
Priya Gore says Microsoft does not use customers' data to retrain the models when they use the Azure OpenAI service.
“The answer is no, because when you use the service in something like Azure, you're able to reason over your own data in a private instance.”
0:19:46 · Priya Gore · Data privacy · 2023
Ally.ai removes personally identifiable information from requests before they leave Ally's firewalls.
“Ally.ai allows us to strip out anything personally identifiable to our consumers or attributable to our consumers before we even send that request outside the Ally firewalls.”
0:23:49 · Sathish Muthukrishnan · PII protection · 2023
Ally's team built a module that removes and restores personally identifiable information and contributed it to the open-source LangChain project.
“we have built a module that will strip out the PII and rehydrate the PII using LangChain that my team has contributed to the source code of LangChain”
0:24:31 · Sathish Muthukrishnan · Open source · 2023
Sathish Muthukrishnan says Ally customer care associates reported that automated call summaries were more than 85 percent accurate.
“It's more than 85 % accurate. It's saving me precious minutes or seconds.”
0:31:52 · Sathish Muthukrishnan · Call summaries · 2023
Sathish Muthukrishnan says Ally had over 205 generative AI use cases in progress at the time of recording, mostly proposed by customer care associates.
“We have over 205 use cases, and this is from this morning, heart of the press, use cases that my team is working on across different businesses”
0:31:52 · Sathish Muthukrishnan · Use cases · 2023
Sathish Muthukrishnan believes Ally was among the first corporate customers, possibly the first, to use a private instance of Microsoft's OpenAI service.
“I believe we were one of the first, if not the first corporate customer to use the private instance of Microsoft OpenAI.”
0:29:25 · Sathish Muthukrishnan · Early adoption · 2023
Priya Gore says GitHub Copilot was Microsoft's first Copilot solution, launched a little over a year before the recording.
“The first co-pilot solution that we launched a little over a year ago was the GitHub co-pilot.”
0:39:15 · Priya Gore · GitHub Copilot · 2022
Priya Gore cites adoption speeds to 100 million users: 16 years for the mobile phone, seven for the Internet, 4.5 for Facebook and three months for ChatGPT.
“it took 16 years for the mobile phone to reach 100 million users. Seven years for the Internet. Facebook, 4.5 years. ChatGPT, three months.”
0:40:23 · Priya Gore · Adoption speed · 2023
Priya Gore says generative AI is best suited to content generation, knowledge summarization, code generation and semantic search.
“generative AI is really great at four things, right? So generative AI won't be the answer to all things, right?”
0:15:36 · Priya Gore · Generative AI uses · 2023
Priya Gore joined Microsoft about six years before the recording, initially in the global black belt organization as its Azure AI leader for the Americas.
“So I joined Microsoft about six years ago.”
0:12:46 · Priya Gore · Career
Priya Gore says Microsoft launched a responsible AI framework around the time it launched its machine learning cognitive API services.
“we also launched an AI framework for responsible AI.”
0:13:16 · Priya Gore · Responsible AI
Sathish Muthukrishnan describes LangChain as the largest open-source framework for maintaining context in generative AI conversations, which he says almost all companies use.
“Generative AI is not able to do that effectively. And the largest open source that allows you to do it is called LangChain, which almost all companies use it.”
0:24:31 · Sathish Muthukrishnan · LangChain · 2023
Ally recently launched 'One Ally,' which brings all its products into a single digital experience.
“we recently launched what we call One Ally, where we are bringing all of the experiences across all of the products into a single digital experience.”
0:50:07 · Sathish Muthukrishnan · One Ally · 2023
Priya Gore says Microsoft was rolling out Microsoft 365 Copilot to clients in an early access program.
“we're starting to roll that out with our clients that are in the early access program”
0:52:31 · Priya Gore · Microsoft 365 Copilot · 2023
Priya Gore says Microsoft was refreshing its AI Business School curriculum and planned to make it available the following quarter.
“We're refreshing that right now. And next quarter, that will be readily available for anyone to leverage.”
0:46:53 · Priya Gore · AI education · 2023
Sathish Muthukrishnan says Ally calls its effort to help communities understand generative AI 'AI equity,' tied to the company's 'do it right' mantra.
“And we call this AI equity. How can we stick to our mantra, which is do it right?”
0:46:06 · Sathish Muthukrishnan · AI equity · 2023