3 Jan 2024 · 25m · Prem Natarajan, Sam Ransbotham, Shervin Khodabandeh · 19 quoted facts
Prem Natarajan, chief scientist and head of enterprise AI at Capital One, argues that successful AI depends on combining three disciplines: product vision, science and engineering. Generative AI is a real inflection point because it follows instructions and adapts without being explicitly trained for each task. That lowers the bar for developers, but it still has to be used responsibly.
The central tension is balance. Natarajan says every AI project needs all three disciplines, but in different proportions. Mature uses like fraud detection lean on science and engineering, while new ideas need heavy product thinking up front. Generative AI also pushes the conversation from AI as a predictive tool toward something closer to a coworker, which requires learning how to instruct it well.
Natarajan says a good idea without science, or good science without reliable engineering, goes nowhere. The mix shifts by project: fraud leans on science and engineering, while brand-new ideas need heavy product thinking first.
Capital One rebuilt its fraud platform around machine learning. Models must act in the time it takes a customer to swipe a card, so real-time engineering matters as much as the science.
Fraudsters adapt, so models must be refreshed constantly. Natarajan says updates once took months and now take days, and newer AI methods help further because they learn more effectively.
These systems respond to plain-language instructions and show behaviors they were not explicitly trained for, which Natarajan calls in-context learning. More people can build with them, though he stresses responsible use.
Asking AI to summarize four papers may highlight what they share, while asking what differs gives a different result. Natarajan says users need to learn how to instruct the technology to get the output they want.
Natarajan says today's generative AI systems adapt to new situations and follow instructions without being explicitly trained for each task. That makes them much easier for ordinary developers and companies to use.
It rebuilt its fraud platform from scratch with machine learning at the center. The models weigh large amounts of data and must reach a decision in the time it takes a customer to swipe a credit card.
It needs product vision (what to build and why), science (the invention needed to make it work), and engineering (making it run reliably in real time). The proportions vary by project, but he says all three are required.
It is the ability of a generative AI model to do a task just from the instructions or examples you give it, without being retrained for that task. Natarajan says the popular way to describe it is that the AI responds to prompts and follows instructions.
The host (label S2) suggests generative AI is more like a coworker you train and converse with than a tool that simply improves predictions. Natarajan agrees it is uniquely exciting but warns against anthropomorphizing it.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Prem Natarajan says Capital One was the first bank, and one of the only major enterprises worldwide, to go all in on a single public cloud.
“It was the first bank, and I think one of the only major enterprises worldwide that is all in on a single public cloud.”
Prem Natarajan says Capital One has more than 100 million customers.
“deliver magical experiences, valuable experiences, everyday conveniences to our 100 million plus customers to help all of those folks.”
Prem Natarajan describes the history of artificial intelligence as alternating 'springs' of optimism and 'winters' of disappointment.
“People like to think of them as AI spring, followed by AI winter, followed by AI spring, followed by AI winter.”
Prem Natarajan says early AI expert systems did not scale because they required too much human input.
“Initially, it was all these expert systems and all of that. And then we said, oh, they don't really scale because they require so much human input.”
Prem Natarajan says probabilistic approaches such as Bayesian models and hidden Markov models later became important in speech and language processing.
“Then the whole probabilistic set of things came in, Bayesian models, then they later became, in some contexts, hidden Markov models and all of that for speech and language processing”
Prem Natarajan says the technical term for generative AI models following prompts and instructions without explicit training for the task is in-context learning.
“And if you become really technical, they'll call it in context learning.”
Prem Natarajan says he is ethnically Tamil and grew up in Maharashtra, India, in a four-language environment.
“My family is ethnically Tamil. Grew up in Maharashtra, where a lot of my immediate friends spoke Marathi.”
Prem Natarajan began his career at BBN Technologies, which he describes as an MIT spin-off with a history tied to ARPANET and pioneering speech and language research.
“And so then I started working at this company called B B N Technologies, yeah, MIT spin-off, ARPANET.”
At the University of Southern California, Prem Natarajan was vice dean of the School of Engineering, a computer science faculty member, and head of the Information Sciences Institute.
“I was vice dean in the School of Engineering. I was a faculty member in computer science, but I also was the head of the Information Sciences Institute.”
Prem Natarajan headed the Alexa AI organization at Amazon before joining Capital One.
“And then I went to Amazon, where I headed the Alexa A I organization.”
Capital One rebuilt its fraud platform from the ground up so that machine learning sits at the center of it.
“We rebuilt this fraud platform from the ground up and basically to use M L at the center of that enterprise.”
Prem Natarajan says Capital One's fraud models must produce a decision in the time it takes a customer to swipe a credit card.
“those models have to kind of activate an outcome in the time it takes our customers to swipe their credit cards.”
Prem Natarajan says Capital One's fraud model updates used to take months and now take days.
“When we first started, it used to take months to update models. We now do it in days.”
Prem Natarajan names retrieval augmented generation as one of the most popular ways generative AI is currently used, including in search tools.
“one of the most popular ways in which these are being used are this retrieval augmented generation style use.”
Prem Natarajan invokes a saying he attributes to Arthur C. Clarke that sufficiently advanced technology feels like magic.
“was it Arthur C. Clarke who said, like, any technology that's sufficiently advanced feels like magic?”
Prem Natarajan calls the idea that artificial general intelligence is imminent the biggest misconception about AI.
“The notion that A G I is around the corner, and I think it's a misconception that works at two levels.”
Prem Natarajan says AI's biggest opportunity is to democratize access to services and resources across society.
“The biggest opportunity for A I is to, in my mind, democratize access to a lot of services, resources, et cetera, across the entire spread of our social spectrum.”
MIT Sloan Management Review and Boston Consulting Group have jointly researched and published on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies.
“Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies”
The next episode of Me, Myself, and AI, the final one of season eight, features Mark Sermon, executive director of the Mozilla Foundation.
“On our next episode, our final episode of season eight, Shervin, I chat with Mark Sermon, executive director of the Mozilla Foundation.”
Natarajan explains why Capital One's all-in cloud strategy gives it a base for applying AI to its 100 million plus customers.
“Capital One has a legacy of being a very tech forward enterprise.”
He traces AI's cycles of hype and disappointment, arguing past breakthroughs like speech recognition became so ordinary we stopped calling them AI.
“have actually become commoditized, which is the true sign of success.”
His path from growing up speaking four languages to BBN, USC and Amazon explains his long focus on language technology.
“I grew up in a four language setting. My family is ethnically Tamil.”
New technology shifts the mix of skills a company needs rather than simply replacing data science expertise.
“it's more of a rebalancing of the resources across the skill spectrum.”
The fraud platform shows how science and engineering combine to make complex real-time decisions.
“in the time it takes our customers to swipe their credit cards.”
Natarajan says the three disciplines are always needed but in different proportions depending on how mature the use case is.
“All of the above, but not all of the above in the same proportion”
Fraud patterns change as people adapt, so shortening update cycles from months to days creates a big advantage.
“When we first started, it used to take months to update models. We now do it in days.”
A research-summary example shows both the power of generative AI and why users must learn to instruct it precisely.
“Can you please give me a half page summary of the key points?”
Natarajan calls near-term AGI a misconception and sees AI's biggest opportunity in widening access to services.
“The notion that A G I is around the corner, and I think it's a misconception”
Generative AI requires organizations to carefully balance product innovation, science, and engineering. On today's episode, a leader in the financial services industry shares his experience with these challenges.
I'm Prem Natarajan from Capital One, and you're listening to me, myself and AI.
Welcome to me, myself and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.
And I'm Shervin Khodabandeh, senior partner with BCG and one
of the leaders of our AI business. Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate. Hi everyone. Today, Sam and I are talking with Prem Natarajan, chief scientist and head of enterprise AI at Capital One. Prem, thank you for joining our show today. Let's get started.
Delighted to be here, Sam and Shervin.
Describe your role at Capital One and the history of how you got there, please.
My role at Capital One, if we just stick to the AI aspects of it, is to build upon, like Capital One has a legacy of being a very tech forward enterprise. It was the first bank, and I think one of the only major enterprises worldwide that is all in on a single public cloud. That kind of transformation takes both a deep belief in the power of technology and a willingness to mobilize the enterprise, if you will, around that kind of vision.
It takes the vision and the willingness to execute and the energy. And that, I think, puts us on a great footing to then harness the power of machine learning, artificial intelligence and all of that. And Capital One has both integrating technology, using technology as a transformative tool, but also in machine learning. And so my role right now is to strengthen that kind of history, build upon that history of early adoption of a lot of technology. You know, we are in this kind of historical inflection point in AI with transformers and generative AI and all of that.
And one way I see my role is to bring the power of all of this new technology to deliver value to the business, to deliver magical experiences, valuable experiences, everyday conveniences to our 100 million plus customers to help all of those folks.
You said inflection point, and I agree we're at an inflection point. Why do you think we're at an inflection point, though?
I mean, this is not the first inflection point, but it does feel historical in some sense to me. In the past few decades, there have been a few such points in colloquial AI history, if you will. People like to think of them as AI spring, followed by AI winter, followed by AI spring, followed by AI winter. And I feel each of those transition points between those eras is kind of an inflection point. Initially, it was all these expert systems and all of that.
And then we said, oh, they don't really scale because they require so much human input. Then the whole probabilistic set of things came in, Bayesian models, then they later became, in some contexts, hidden Markov models and all of that for speech and language processing, et cetera. Those have all been inflection points where we said, oh, this thing. And even though sometimes people feel like AI has always been promising, in many ways in my mind, the previous inflection points in AI history have actually become commoditized, which is the true sign of success.
Like 2025 years ago, using speech recognition in standard industry practice, whether it's for, you know, interactive voice responses seemed novel. Now, all of us kind of expect it is there. And so once it is there all the time, we kind of don't think of it as a I. Honestly, like, we, oh, that's just speech recognition. But there was a time when it was like the forefront of machine learning and AI. And so now this new inflection point, though, I'd say if we think about it as a stack, as a science stack, a capability stack.
We go from being able to take phenomena and convert them into some representation, like a speech signal into the sequence of words. And the next step up is kind of interpreting some of that transduction into something meaningful, like maybe some level of semantic interpretation, et cetera. We keep moving up that stack. Right now, we're in this place where we've built all of these systems. They're showing tremendous capacity to adapt themselves to novel circumstances. But one new thing right now is that they're demonstrating behaviors that they were not necessarily explicitly trained for or designed for.
And if you become really technical, they'll call it in context learning. In the popular literature, we refer to them as, oh, they respond to prompts, they follow instructions. So that part, I think, kind of substantially lowers the bar for their use. You still have to do it in a responsible way, you still have to do it in a thoughtful way. But it lowers the bar for their use where all of us can start using these in our pet projects and in our enterprise-wide initiatives.
So that's the inflection point. I see the developer experience has changed.
And when you say these, you you mean you mean specifically large language models and the whole stack around that, is that right?
Yeah, yeah, from a technical technology perspective, transformers, in terms of their manifestation as a capability, large language models and generative AI. And I think it has the power to transform the developer experience, like your creativity is top and front and center, and you can use all of these resources relatively easily.
Prem, you have a pretty interesting background. Maybe share a little bit about how you got started in technology and AI and your path to where you are today.
Happy. Though I should say the early part of the path is somewhat canonical for somebody with my background. You know, I grew up in India, I did my undergraduate education. I grew up in a multilingual community and society. I grew up in a four language setting. My family is ethnically Tamil. Grew up in Maharashtra, where a lot of my immediate friends spoke Marathi. Hindi was one of the required subjects in class and was a reasonably cosmopolitan neighborhood I lived in.
So there were people who came from other parts of India, spoke Hindi. And then English was the medium of instruction. So, you know, and so it was hard to not spot some aspects of language that are interesting. So if you're just speaking Indo-European languages, you're used to certain verb, subject, object orders. But then you take something like Tamil, and it's not an Indo-European language, it's a Dravidian, and so those orders are different. So, even at a surprising early time, we didn't understand there was actually a subject called linguistics.
We just said, I wonder why we say, come here in this language and then here come in this other language. And there was some spark of, like, curiosity built in early on in that way. And again, I'm not necessarily unique to me, but in my case, it kind of triggered some actions later on. One of the summer internships I did during my graduate school was working on offline handwriting recognition. And that kind of reawakened, I think, my interest in language and its production in some form.
And so then I started working at this company called B B N Technologies, yeah, MIT spin-off, ARPANET. There was a lot of recent modern history in that place, and it had been a pioneering place for speech and language research at the time. And so the next several years were just an incredible learning experience for me. So that was early. Then I expanded the set of things I was interested in. It led to computer vision, other areas, and all of that just happened to be a good thing for today's world, where our interest is.
A I. When we talk about it now, we talk about it in terms of multi modality, reasoning and things like that. I guess also wanting to constantly work on new problems while still maintaining some connection with old problems allowed me to increase the surface area of what I was doing. And then I went to University of Southern California as a faculty member and also as an administrator. I was vice dean in the School of Engineering. I was a faculty member in computer science, but I also was the head of the Information Sciences Institute.
And then I went to Amazon, where I headed the Alexa A I organization. A fantastic learning opportunity there too, contributed and learned how to scale to massive scaling. Then I wanted to go back to my original roots, where I was also building end to end solutions for end users in enterprise and Capital One. Now going back to the technology forward lean, big investment, support from the very top for being at the frontier of technology. All of that just felt like an exciting place to just come and build.
Yeah, it's wonderful, and it does. I realize you might not be at liberty to talk about all the magic that's in the works, but are you seeing a future where that composition of the team that does A I is changing and evolving? And maybe moving away from the hardcore data science a bit more towards other skill sets, like engineering and prompt engineering and design and human centering design and things like that.
And see, I think with every wave of technology, whether it's A I or something else, it's more of a rebalancing of the resources across the skill spectrum. When something new comes about, you need new skill sets in your enterprise. And then maybe it helps improve the productivity of certain other things, but then you need these new things too. So basically, the overall enterprise is producing more through a rebalancing of these things. So, you know, people learn new things, et cetera.
I would say, coming back to the thrust of your question, though, we are opening up a whole new set of possibilities, right? In terms of what can be done, whether one of the most popular ways in which these are being used are this retrieval augmented generation style use. If you look at something like your favorite search tool today that uses generative A I, they're using some form of this. And those things allow us to become better, faster at things that we might do routinely, things that you might not enjoy doing.
But when it comes to certain things around decision making, et cetera, I think that end of data science still remains in where you're bringing in your domain expertise to use these technologies to deliver more value in that domain. But I think the fact that these are more scalable, more adaptable, more capable of learning, able to consume massive amounts of context, makes that investment that much more valuable because you get so much more performance out of it.
Hey, when you think about what something costs, if it costs a plus b, and the cost of a part goes down a lot, then the overall a plus b goes down a lot, and you can do a lot more of it. Can you give us some examples of the kinds of things you're doing at Capital One? Earlier, you said magical. What's something magical that you've got going on?
What could be magical is things that anticipate my needs or things like that. So, but leaving aside kind of that speculative future aside, I was also just going to sort of like, you know, was it Arthur C. Clarke who said, like, any technology that's sufficiently advanced feels like magic? So it's also in that technical science fiction context that I was kind of saying magical. But coming to this other question that you have about how are we doing it?
I'll give you one high level kind of abstract conceptual thing and then something very specific as well. At the abstract level, I think we see tremendous potential here to harness all of these advances in A I to deliver better experiences for our customers. I mean, Capital One has a whole portfolio of offerings for customers. And so we see a real opportunity to deliver continuously better experiences for our customers. In that sense, I think what will happen is A I will become more and more central to how we deliver value for our customers, how we run our business, et cetera.
Now, on a specific example, let me talk about our fraud platform. We rebuilt this fraud platform from the ground up and basically to use M L at the center of that enterprise. And also to make it efficient so that we can make complex real time M L decisions, massive amounts of context being consumed, massive amounts of data being used. And in order for it to be really useful to our customers, those models have to kind of activate an outcome in the time it takes our customers to swipe their credit cards.
So it's both a feat of science, but it's an even more impressive feat of engineering. I like the fraud example because I think it brings together the know of all the different disciplines we need to bring. I think the best work here will be at the intersection of folks with solid product vision who are envisioning the use cases, the folks with the science vision to translate that product vision into saying, what is the invention that is required to enable that science?
And then folks with the engineering have to say, I can do all of this, I can do it reliably, it will work time in and time out, and it will work in real time all the time, and you can count on this, et cetera. So it's just like something that exercises all the muscles of a complex, multidisciplinary org.
I like those three cases. I mean, if you think about any one of those three, if you didn't have it, it wouldn't be worth doing. You could have great science and great engineering, but if you got the wrong idea, you're not going to go anywhere. But having all those threes together, as you point out, big part of that. Where is the challenge? Which one of those three is the hard one? All the above?
All of the above, but not all of the above in the same proportion in every instance, right? If you take something like fraud, which is a very highly developed use case in the sense that just from a use case perspective or a product perspective, we kind of understand the whole shape of this application. Well, there the balance might be in the science and engineering, like, you have to do really, what are the new invention I'm going to do?
There might be other cases which, you know, obviously, as you might imagine, we are not ready to talk about yet. But where things don't exist, where you're envisioning the future, and this might fall in that bracket of what's the new magic that will happen in the future, right? There, I think all three of these have to be engaged, but there has to be a lot of product thinking up front. What's the use case, what is the impact it will have, who will benefit from it, what will be the business impact, what will be the customer impact?
So all of that. And it helps for the entire enterprise over time to start having more and more of a product mindset, right? And so everybody's thinking in that way. So I'd say, Sam, that it's all three, but not always all three in the same proportion in every instance. I'll give you an example. In the case of fraud, if you get deeper into this thing, you know, the world is constantly changing. And so you build a something that models fraud, et cetera.
But, you know, the world adapts itself to everything. Right? That's the amazing thing with humans. We learn and we say, oh, these patterns no longer give me the outcomes I wanted. I have to adapt how I do this stuff, which means the models also have to be updated regularly. When we first started, it used to take months to update models. We now do it in days. When we think about it from a product perspective, sometimes it's about the high level application fraud.
But then, if you're really thinking deeply down and say, what are my developer partners experiencing? What is the friction they're experiencing on a day-to-day basis? That might stand in the way of me as a company getting maximum benefit from this technology. Oh, look, how much effort it takes to update these models. Ah, can I bend that curve? Well, it turns out some of these new technologies help us bend that curve too, because they lend themselves to more easy updating and and things like that, because they learn so much more effectively.
These are all the ingredients of magic, right? I mean, what you're talking about, what's what's going on in the industry overall, your own unique background, with language being a pretty enduring thread. And then, of course, the mix of both the theoretical and the practical and the entrepreneurial and then the support over there. That does sound like magic. And so, I guess the the question I have is give a glimpse of the kinds of things that makes it magical.
Can you, like, give us a teaser on any of this?
I can try. As a lead-in to the answer to that question, look, I think these are still kind of very early days, right? Because generative AI, as I said, it has been showing properties that are not entirely designed. The fact that you give it an instruction and it follows it, right? Things like that. So I think it behooves us to be responsible, especially responsible, thoughtful in doing some of these things. Because once something can do things that you haven't designed, like you really want to think through things.
So imagine that you're reading, I don't know, you have a thirty page paper to read, right? Well, how about something that you say? Can you please give me a half page summary of the key points? Now, part of it might be that half page summary may miss some of the key things, but just as somebody who wants to keep up to date with a lot of stuff that's going on, right? To me, that feels like a pretty magical thing, like just as somebody who wants to keep abreast of a lot of stuff that's going on.
The next step could be, hey, here's a collection of four papers that I've been looking at. Can you give me a summary of what's different between these four? Who now? That's like really getting exciting, right? Like, because it's one thing to read a paper in depth and understand what's there, but now I have to read four papers to kind of zoom in on what's different between them. So I think in those kind of relatively open use cases are the seeds of the kinds of things that you could do
That's right.
for other kinds of applications where those kinds of things are particularly valuable, but gives you a sense of the power of this technology. Now, in order to do it right, remember how I said, Can you tell me what's different between these? I could give you the four papers and say, Can you summarize these four papers? And it might actually just focus on what's common between them because it might think that's what's important. Or it might make the statistical inference.
Let me not say think. I don't want to anthropomorphize this too much, right? But if I give it a specific, tell me what's different. So now I have to develop some level of understanding of how to instruct this technology to get the output I want. It's doable, and that's where gaining familiarity with these things becomes useful.
And that's sort of what I wanted to tease out of you, because I I also feel like it's a technology, but it's different in many ways. Whereas if I think about some of those AI models that you were talking about, you know, with fraud, AI was a tool and it would dramatically improve the efficacy and speed and accuracy of predictions. It feels like here, generative AI is more than a tool. It it maybe is a co-worker.
You're you're sort of having a conversation with it. And like a new coworker, you're training it and you're you're seeing unintended consequences, just like you would. I I know you're not trying to anthropomorph- anthrop- okay, I won't Anthropomorphize. say that. Yeah, but uh, no, I think there's something very, very unique.
Yeah, there is something unique, something uniquely exciting. Yeah.
Let's transition. Prem, we have a segment where we will ask you some rapid fire questions. So just give us the first thing that comes up in your mind. What's the biggest misconception that people have about A I right now?
The notion that A G I is around the corner, and I think it's a misconception that works at two levels. One is that there is the belief that there is a shared understanding of what is a G I and kind of the fear around it. I think it's appropriate to be very guarded and very concern and very thoughtful about how, as a society, we should respond to it, et cetera. But I think as humans, we've overcome so many things, so I just feel like we will prevail here too, you know?
Prem, talking about artificial general intelligence here versus a more narrow definition of A I that shows up in so many of the things that we're doing. What do you see as the biggest opportunity for A I right now?
The biggest opportunity for A I is to, in my mind, democratize access to a lot of services, resources, et cetera, across the entire spread of our social spectrum.
What was the first career that you wanted?
Oh, what was the first career that I wanted? That's interesting, right? It was actually a lawyer.
That's consistent with language.
Consistent with language. I also felt like many of the role models at that stage in my life that I looked at, especially in the Indian context where I grew up around the time of the independent movement. Disproportionate number of them were lawyers or teachers.
So where are we using too much A I? Where are we making this hammer fit all the screws?
I don't know that a particular pattern stands out to me, but I'll say this. I think when something is working really, really well, right? Like, for example, my tap, you know, if it's working finely well and then I can do a touch, you know, nowadays you have these touch taps, that's awesome. I think if if you get to a point we're saying tap, turn on water tap, uh, like that feels like, you know, so I'd say there are things where I think as humans, we can say, is A I improving, reducing the friction I'm experiencing in my life, making things easier, or does it just feel like, you know, strange?
Like, so, okay,
Technology for the sake of technology. So what's one thing you wish A I could do now that it cannot?
Oh, that's an easy one. I wish it could make me an awesome singer. I love singing, I love music, I don't have a singing voice, and so if A I could transform
I think it can.
Yeah, I think I think we might be there.
I would like to have an A I attachment to me that I go out on a karaoke and I sing and everybody's like, man, this this guy is belting it out.
Good. I really think this framework you mentioned about the idea of product combining with science, combining with engineering, and how all those pieces fall together and are necessary, but have balance, different balance in different situations. That alone will probably resonate with lots of our listeners. Thank you for taking the time to talk with us. We've enjoyed having you. Thanks.
Thank you.
Thanks for listening. On our next episode, our final episode of season eight, Shervin, I chat with Mark Sermon, executive director of the Mozilla Foundation. I'm excited about this one. Please join us. Thanks for listening to me, myself and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MITSMR and BCG.
You can access it by visiting MITSMR dot com forward slash AI for leaders. We'll put that link in the show notes, and we hope to see you there.