Why CEO Matt Garman is willing to bet AWS on AI
Decoder with Nilay Patel · 13 Jan 2025
Source: https://www.podtrac.com/pts/redirect.mp3/pdst.fm/e/pscrb.fm/rss/p/mgln.ai/e/257/traffic.megaphone.fm/VMP1155018731.mp3?updated=1736548375
Transcript: https://podcasts.opennash.com/shows/decoder-with-nilay-patel/why-ceo-matt-garman-is-willing-to-bet-aws-on-ai-121cb2/ (via OpenNash Podcasts, machine-generated)

As soon as autumn rolls around and you swap your iced lattes for pumpkin spice, that's your signal to start thinking about the holidays, the decor, the gifts, the shopping lists, but more importantly, the holiday treats, cheesecakes, cookies. Is there anything better? And when the recipe calls for cream cheese, there's only one option Philadelphia, the original cream cheese, made with fresh milk and real cream. This holiday season, whether you're going all out or keeping it simple, make it really philly good.

Find your favorite holiday recipe at creamcheese dot com.

Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow none of them talk to each other. That's where Odoo comes in, an all-in-one business management software that brings every part of your business together. From sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system.

Try for free today at odoo dot com slash vox. That's O-D-O-O dot com slash vox.

Hello and welcome to Decoder. I'm Nilay Patel, editor in chief of The Verge, and Decoder is my show about big ideas and other problems. Welcome to a new year. We're starting 2025 off with a good one. Today, I'm talking with Matt Garman, the CEO of Amazon Web Services. Matt's actually the new CEO. He just took over last June. You might recall that we had his predecessor, Adam Selipsky, on the show just over a year ago.

That makes this episode terrific decoder bait, since I love hearing how new CEOs will decide what to change and what to keep going after they've settled into their role. And Matt has a really interesting perspective for that kind of conversation, since he's been at Amazon for twenty years. He started as an intern and was AWS's original product manager. He's now the third CEO of that division in just five years. And I really wanted to understand both his broad view of AWS and where it sits inside the cloud computing industry, which he had a pivotal role in creating.

You'll hear Matt say that most companies are still barely in the cloud, and the opportunity remains massive for AWS, even though it's been the market leader for years. If you're a product manager or an aspiring product manager, you'll catch Matt talking about these things exactly like the PM he was from the start, only now with the broad view from the CEO chair. But just acquiring new customers isn't really the game anymore. Like every cloud provider, Amazon is reorienting its entire infrastructure for the world of generative AI.

That includes more than 8 billion in funding for Anthropic, a huge push to build its own AI chips to compete with Nvidia, and even nuclear power investments as the energy demand for AI continues to grow. In fact, after Matt and I had this conversation, the company announced 11 billion dollar investment to expand its data center operations in Georgia. What was most interesting to me is that Matt's perspective on AI as a technology and as a business opportunity is pretty distinct from his peers, especially those who are more incentivized to hype up the capabilities of AI and what it might be able to do.

I really pushed Matt on Sam Altman's claims that we're close AGI and on the precipice of machines that can do any tasks that humans can do. If that's true, a lot of things are going to change, and companies like AWS will have to change with them. I also wanted to know when any of this is going to start returning or even justifying the tens of billions of dollars of investment that's going into it. Matt's answers to both of these things were pretty candid.

It's clear that Matt and Amazon are far more focused on how AI technology turns into real products and services that people want to use, and less about what Matt calls puffery in the press. One note before we start, we recorded this episode just before the holidays, so I asked Matt about Netflix, one of AWS's biggest customers, and whether it would hold up while streaming live events, especially the NFL games that was set to air on Christmas Day.

Turns out, Netflix did just fine with those, but the answers were pretty interesting, so we left them in. Matt still checks in on his big customers, even as CEO. Okay, AWS CEO Matt Garman, here we go. Matt Garman, you're the CEO of Amazon Web Services. Welcome to Decoder.

Thank you, thanks for having me.

I am very excited to talk to you. You're like a perfect decoder guest. You are, I believe, the first product manager at AWS. You started as an intern, and now you're the CEO. We have a lot of listeners who want to be on that journey, so there's a lot to talk to you about just in that. You're also the new CEO. We had your predecessor, Adam Selipsky, just a little over a year ago, you're, what, six months on the job?

So there there's a lot of decoder stuff in there, just how we're how we're changing the organization, how you're thinking about it. And then, obviously, we we're gonna talk about AI. It's gonna happen. I hope you're ready for it.

I'm ready for it. Shoot far far away. I'm happy to go wherever you want.

All right, but I I actually wanna start with a very hot button, deeply controversial topic. Are you ready?

Great, far away.

Okay, it's Jake Paul. I I wanna start with Jake Paul. My understanding is Netflix is like the prototypical AWS customer, right? They started in AWS, they made a big bet on AWS. Uh, is they're still a customer, right? They haven't left AWS.

Yeah, yeah, yeah, Netflix is a great customer of ours, absolutely.

Okay, so they just had the live stream of Jake Paul fighting Mike Tyson. You can think anything you want about those two men fighting each I was hoping other.

Mike would win, honestly, just, you know, So was I. Uh as as I think probably most were, but that's okay. It was fun to see him out there.

you, you've just set off a million more conspiracy theories about this file, anyhow. Uh, told you it was controversial. Uh, all right, so the the stream was pretty glitchy, I think everybody agrees on that. Uh, I watched it, it degraded to to like 360 p at some point for me. Uh, Ted Sarandos was just on stage at a conference. Uh, they said the demand is 108 million people globally, and here's what Ted said about that stream.

Quote, We are stressing the limits of the internet itself that night. We had a control room up in Silicon Valley that was re-engineering the entire internet to keep it up during this fight because of the unprecedented demand that was happening. You're the CEO of AWS, you're the internet. Did they have to re-engineer the internet for the Jake Paul fight?

Uh, you gotta ask Ted about the, uh, the the the piece I think about that where they were stressed was they they do run their own CDN, um, and so you can ask Ted about that too. But that's the part of it that was really stressed is the the CDN part of that, uh, is really the part that they and they they have, uh, Netflix has their own homegrown CDN that they use, and that's the part that I think was stressed.

So I don't know the details of exactly where they were, um, running into barriers, but it wasn't in their AWS, uh, infrastructure, it was the in the Netflix controlled part of their infrastructure.

Yeah, their their CDN is real fancy, right? They've got boxes and ISPs and everything. I was just curious because what we're about to talk about in in a huge way is how providers like AWS can meet just the growing demand for compute everywhere and then get it to the people who need it. And it feels like most people in 2024 take video streaming for granted, but it's still pretty hard.

Yeah, it is, and I think in particular, like, there's a couple things around that that are challenging, right? By the way, it's like it's a super hard thing that they did. Number one is their first time doing a live a big, like, scaled live stream like that, and so the first time is actually what's hard. You know, there other people have done that before. We will stream Thursday Night Football and and other places like that that have figured out how to do things like that, but it's not the first time, right?

And so it's the I am very I'm sure that the next time I think they have a Christmas Day game, they'll probably work out some of those kinks and and figure that piece out. And so, um, the first time you do it, you'll find those bottlenecks, and and it's true about any compute system where you have an order of magnitude more, and they obviously have shows that have streamed more, but they're spread across more time, right?

And so it's this single spike up where everybody comes in a thirty minute window, and if it's outside of what you plan for, right? If they planned for, I don't know what their numbers were, but 150 million and they got 180 million, that it was outside of what I think they thought their their upper limit was. And we've seen this before in AWS, and we've seen this in Amazon the first time we probably did Prime Day, we probably had issues across that too of just people hitting the website and other things.

So the first time you do events like this, it's a learning process, and so I I think it's probably, um, overstating to say they had to re-architect the whole internet, I think, but it really is that that key spike where a lot of, uh, applications are just not, particularly when you own the infrastructure. And this is one of the benefits of the cloud, by the way, is you get to ride on on, you know, the law of large numbers, where any one spike is not necessarily, um, doesn't overwhelm everything else.

Uh, Netflix obviously has a huge number of customers, and so my guess is they'll be much more prepared for next time, but it's a good, uh, it's a good learning for anybody, even at much smaller scale, that when you're planning on an event that is matur- that that could has the potential to be materially more than your average baseline, there's always risks of that there are some scaling factor that you don't anticipate. So it's it's not a surprising problem to me.

We've seen it over and over again, and it's it's part of one of those problems that the cloud helps to solve. But even in the cloud, like, planning is required, and you have to think about how you scale ahead of it and things like that.

When you were at home watching the fight, did your did your pager go off?

I was texting back and forth to our support teams to make sure we were supporting the Netflix team as, uh, as much as possible. Yes,

How often does that happen to you as you use the internet and you think, boy, this is probably running on AWS. I I better make sure it's going fast.

uh, mo- more back in the day when we, you know, when we were scaling and learning back in two thousand seven and two thousand eight, where, you know, we we kind of were learning how to scale there. I think we today we're we're often at at a broad scale, and so everything, you know, lots, lots of things on the internet and lots of things around the world run on AWS and and we usually run pretty reliably, so le- less than it used to, for sure.

Do you do you have Down Detector bookmarked on your on your on your laptop?

I don't, no.

That down, uh, we gotta get the CEO of Down Detector on the show. That is a fascinating service across the board. Let me ask the decoder questions, cause I think this theme of we are going to be more reliant on cloud infrastructure for compute in the world of AI. And that's gotta get to all the people and hopefully make everybody some money and generate some useful products and services. That's the theme. And I think whether or not we can stream people punching each other and whether or not we can stream AI, the the problems there are in the general sense the same.

But I wanna ask the decoder questions first, so we can I can understand how you are solving these problems, having been at AWS for so long. So you are taking over for Adam, who's on about, uh, just a little over a year ago. He stepped down about six months ago, you took over. You've been there a long time. You started as the the first product manager of AWS, which is a pretty wild place, uh, to begin a career and end up as a CEO.

How are you thinking about AWS, the organization, right now?

There's a couple things that I'm thinking about, you know, one, it's I have been here for eighteen years, so I've been fortunate to learn a lot of the different parts of the business and seen it from the early days until we are now. And, um, you know, I think over eighteen years, we've grown to be 110 billion dollar business, growing at 19%. So that's that's that's great, and we're just at the early stages of what that business can be.

And so I'm pushing the teams consistently to think about how do we innovate faster, how do we think bigger, and how do we support our customers? And as we think about the potential of AWS being 200 billion or 300 billion or 500 billion dollar business, or or whatever size it gets to, you know, I think we continuously just wanna think about what are the organizational structures, what are the m mechanisms we use, what are the ways that we supported customers that worked to get us to 100 billion and may not work at two or 300 billion.

And and I think some of that is just thinking about how do we scale at that? And how do we think about supporting customers in a great way? How do we think about scaling our services in a great way? How do we think about continuously innovating across many different paths? And as you think about, we have to really innovate along our core. What's important, like the the thing that got us here around compute and databases and storage and networking.

But we also have to innovate around AI, we have to innovate around some higher level capabilities, we have to innovate around analytics, and we have to innovate around helping customers who maybe are less technically savvy, take advantage of the cloud where they may not be at Netflix level sophistication, um, which is obviously a very sophisticated technology team, but want to take advantage of some of the the cloud capabilities. And I think we're continuing to think about how do we keep pushing that envelope on helping more and more customers take advantage of of what we have.

And, you know, I think what's one of the things that I spend a lot of time thinking about is how do we organize so that our teams don't lose agility and speed as we get bigger? That's some of where I'm thinking about, and it's nothing that's broken today, but I think it's kinda looking around corners to where when the business is is twice as big as it is today, how do we make sure that we continue to execute and run as fast as possible?

Can I ask about that piece of the puzzle, just where the next new customer comes from? Obviously, when you started at AWS, they were all new customers. Now, most huge companies, they at least have an idea of what they might do with the cloud, whether or not they're using AWS or something else. Or, uh, we have a lot of CEOs who come on here and say, Look, I need to have multiple clouds so that I can get I can go do rate negotiations with all of them.

Fine. There is a new class of companies, I think, that assumes they don't need an any software support. They're just gonna go hire a bunch of SaaS vendors, and they will just run their business and they use the SaaS products however they want to use them. And it seems very unlikely that they will become AWS customers themselves, right? Because they they've outsourced a bunch of business functionality to a bunch of other software vendors. And I'm just wondering that that's a new class of potential customer, right?

Like that yeah, well, that kind of business didn't exist until kind of recently.

it's true, and I think that it's probably there's, uh, there's subtlety in there. So I'll take a couple of those one at a time. Number one, uh, we do have a lot of large customers that are running in AWS in the cloud today, and a huge number of them still have massive amounts of their estate on premise. And so there's a huge amount of growth for there. You can even take our largest customers, many of them only have ten, twenty, thirty, 40% of their their workloads in the cloud.

And so there's a massive amount of growth just helping them get to seventy or 80%, or whatever that that number is gonna be, and and doesn't even presume you get to a hundred. And so I think there's a huge amount of business there. I think there's a huge amount of business in customers that only have 1%, or, you know, rounding to zero of their estate in the cloud, cause they're still running on-prem workloads, and they're whether it's IT or or core business pieces.

And some of it is just they're running it in data centers. Some of that is the workloads haven't yet moved to a cloud world. Think telco networks broadly, right? Most telco networks still run in traditional telco networks. And there's a handful of customers, like the dishes of the world, who have kind of thought about and and have moved to building, cause they they got to start from zero and have built it in the cloud and get the benefits of that agility, but most haven't, right?

And so there's a lot of, uh, think about all of the the compute that happens in a hospital today is mostly in the hospital, right? It's it's and and these are just examples of where there's an enormous amount of compute that could take advantage of these broad scale cloud systems that still haven't yet moved there. So there's a huge amount of potential in those additional businesses. There's also, just as you think about new customers, every single year, there's a huge number of startups that are jumping that are they're created from scratch, and they they all start in the cloud too.

So there's there's still lots of greenfield opportunity for us. And I think your observation about companies leaning more into SaaS is is super interesting, and that's why there's such a focus for us. And so it's why we focus on deep partnerships of how do we make sure that AWS is the best place to run SAP? It's the best place to run Workday, it's the best place to run ServiceNow, it's the best place to run, you know, keep going down the list, right?

And so those SaaS ISVs have always been a really important customer base for us. And increasingly, you see us build capabilities that make AWS even more powerful for SaaS vendors. And so, uh, just at re:Invent, we announced a capability called Q Business Index, where you can basically have all of your SaaS data pulled together in a single index that's owned by the enterprise and controlled by the enterprise, but you can share across SaaS products. And so I think there's things like that, and you'll see more things like that where we can help customers not just say, okay, my data is in a bunch of these SaaS islands and I can't get benefits across them.

And you know, I don't think customers won't be an AWS customer because they're still gonna have a data lake of their own data, they're still gonna have their own applications, they're still gonna run their own websites. There's other things that customers are still gonna wanna do. And so I, you know, I think more of their applications will be in SaaS as opposed to self-managed software, for sure. And I think, uh, I it's it's hard to imagine many customers that won't have their own compute, storage, database needs also.

When Adam was on the show, I asked him, What w- what's the point of the airport ads? Like, who doesn't know about AWS? And his answer basically tracked with what you're saying. There's still a lot of customers who we need to get to even think about moving to the cloud, and that's why there's there's internet football ads. Is that your answer? Is it when you get off the plane and you see the AWS logo, you're like, I'm gonna get that guy?

I mean, look, you can make that argument for lots of ads, too, like, who doesn't know that Coca-Cola exists, but you still see Coca-Cola ads? And so some of it is keeping it top of mind, some of it is also, you know, I think, uh, if you think about the advertising that we do with, uh, together with some of the sports networks, whether it's NFL or F1 or others, a lot of what that does is to help connect the dots.

You may know that AWS exists, but helping see that, like, in a context that you understand, which is football or F1 or or or Bundesliga or whatever the the sport is, how we're helping do analytics for that sport. It really is one of those things that helps customers connect the dots. And so it's not just an ad that says, Hey, AWS exists, but it is connecting those dots that says, okay, if we're able to do analytics that can see how fast a football player can run or see what the chance is that an F1 car can pass.

It really helps customers just connect the dots as to where we might be able to help their business too. And it also opens the door for us to do that next deep dive where we can really dive in and understand that, and and we find that that connection point is actually quite valuable, even if people know that AWS exists already.

I do love the idea of some CEO coming to you and saying, I need a win probability meter for my team every minute of the day in real time.

That's fair.

Uh, let me ask you about Telco for one second, just because Telco has long been a particular fascination of mine. Uh, Dish obviously started from scratch. They announced very loudly that they were gonna use AWS as their cloud provider, that they wanted to do all the compute they needed to do for five G and all that stuff, to run that network in the cloud. The compare and contrast that to the other telcos. When Verizon was launching five G, for example, uh, they told me that they were gonna build a competitor to AWS because they needed the compute at the edge to run the network anyway, and they might as well just sell the excess capacity in their data centers to

customers and say it would have lower latency or whatever you get from being very much at the edge. Did that pan out, or are you saying, okay, that didn't work, and I can go conquest those customers now? I can go get Verizon or ATT, or whoever else on the network.

Well, Verizon was a little bit different. It was actually a partnership with us where we were talking about potentially selling some of that compute space together at the edge. And I think that technology is probably a little bit ahead, and I still think that there's an interesting eventual win there. But I think the idea was a little bit ahead of the technology of, like, really low latency compute at the edge, mostly because a lot of that latency was taken up just in the network, like, and so it's it's hard to get that benefit of a, you know, a small latency gap.

But, you know, I think a lot of look, if you go back fifteen years, many companies were thinking that they would just go offer the cloud like it looked like it was easy, right? And they they said, Oh, it's just a hosting thing. I have a data center, I can sell that. And so I think most companies today, outside of the handful of three, four companies that are really in the space, don't really think that they can go provide a a real cloud offering.

Like, it's hard. Uh, and and particularly, you know, there's niche offerings that you can offer in particular slices. But I think increasingly, we view and and together, we view this as a partnership opportunity where we can go add value together. And so I think our partnership with Verizon is great. We look at how we can go add value together, and over time, we'd love for more of the broader network. Cause you look globally and you're starting to see other telcos start to lean into this model of okay, maybe more of the the core can be run in AWS, and then maybe, you know, so that that part is okay, that can be run in central data centers.

And so we're starting to see more core. And then you think about, can the RAN be run in AWS? Maybe, yeah, it can. And actually, they're starting to see that piece in there. And so, you know, I think it'll be a a transition over time, but I do think that as we add more value there and show that we can give programmability to their networks, that we can show that there's scale to the networks, that you can actually show benefits on patching and other things like that, where there's a lot more flexibility there, I think you'll see more and more telcos leaning into to cloud-based place deployments.

I am sure your partners at the traditional telecom companies appreciate your support in their retconning of their promises around five G. You're doing great. There's a real split here. Just I I hope people can can hear it. We're talking about still trying to get customers to come use cloud services. Step one, move some of your compute out of the basement of the hospital and into the cloud. And a lot of companies aren't there yet, and it seems like you you you perceive that there's still opportunity there.

Then we're gonna in a minute, we're gonna talk about AI, which is the absolute cutting edge of how do we even run these companies? What do these computers even do? How does the cost work out? How are you structuring the organization to deal with that split? Don't have your own servers in the basement versus turn your decision making over to some agentic AI system that we're gonna run for you.

Well, in in some ways, it's a a much stronger carrot. If the pitch is, hey, run the exact same thing that you're doing, but do it a little bit more efficiently and and a little bit less expensively, that is less of a value proposition than if you can do something that hasn't been possible before. And so I think that's why many of the workloads that you've seen move to the cloud already are the super scalable ones, or the ones where they need lots of compute, or the ones where they have a really large footprint.

Because they see s like the wins are are enormous for those type of customers. You know, for a server running in a basement of a of a hospital, you know, maybe they can save a little bit of money, or maybe they can save a little bit of IT work or whatever, but it's the the value proposition may not be there unless we can really deliver a lot of value. You're not gonna be able to get a lot of the value that's promised from AI from a server running in your basement.

Like, it just like, not possible, right? The the technology won't be there, the hardware won't be there, the the models won't live in there, et cetera. And so it's it's in a in in many ways, I think it's a tailwind to that cloud migration because we see with customers, forget proof of concepts, like you can run a proof of concept anywhere, and and and I think we've, you know, the world has proven over the last couple years, you can run lots and lots and lots of proof of concepts.

But as soon as you start to think about production integrating into your production data, you actually have to have that data in the cloud too, so that the models can interact with that data and you can actually have it as part of your system. And I do think that that is going to be a a tailwind over the next couple of years as people really want to have these agentic systems, they really want to have their data in a secure environment, but integrated into a, uh, AI workflow.

And you you can't orchestrate an AI workflow pointing it at a mainframe. It just, uh, like, it's not gonna be possible. And if you have the data going back and forth to some model, you're you the security and and control of making sure that that IP stays with you is is risky, too. But if you move the whole data into a secure cloud environment and you have a a modern data lake that that has all your data, your application is working there, and you're co-located with where the model can run and all the controls can run and the guardrails can run.

And you can have a a RAG index that's nearby, too, that takes advantage of all that data. That's when you can really start integrating it into your actually production applications, and that's where you're gonna see a lot of the wins that are really meaningful. Not just kind of like a cool, Hey, that's neat that I can have a chatbot, but really integrated into how your workflows change and your and your how you can do business changes.

And so I I I have seen early signs that that is, you know, to your to your question on organization, they're very complementary. They're not it's not a or B, it is kind of all pushing in the same place. And so we'll have to have different capabilities, we'll have to have different motions to help all of that. But I do think that that move of getting your data into a cloud world, it's it's kind of a necessary condition to having future really, really successful, deeply integrated AI, I think, into your business processes.

We need to take a quick break. We'll be right back.

Support for the show comes from Quo. You know, when you're juggling five things at once and your business phone rings and you just let it go? That call might have been your next best customer. That's why today's episode is brought to you by Quo, spelled Q-U-O, the business phone system built so you never miss a call. All your calls, texts, and voicemails live in one place, so anyone on your team can pick up a conversation, see the full history, and respond fast.

More than 90,000 businesses rely on Quo to stay reachable and look professional every day. Setup takes minutes, and you can keep your existing number and add teammates as you grow. Quo's optional built-in AI agent handles after-hour calls, answers questions, and even books appointments, even when your team is offline. Money is on the line. Always say hello with Quo. Try Quo for free, plus get 20% off your first six months when you go to Quo dot com slash decoder.

Spelled Q-U-O dot com slash decoder. Support for the show comes from Shopify. Starting a business requires you to be resourceful, and the more you flex that muscle, the more you start to see opportunities for growth. For example, launching your business with Black Friday approaching means you can take advantage of the extra online traffic. Shopify is there to help you make it all happen. Shopify is the platform for a record-breaking, reliable holiday selling season for businesses at every stage.

Their pre-built website themes and AI site builder make it easy to launch a beautiful, professional-looking store, and you don't ever need to touch a line of code to do it. They also have a mobile app that lets you run your store from anywhere. Their AI e-commerce expert Sidekick can help you manage almost any business task with a single prompt, including holiday discounts. With Shopify, you can easily sell everywhere people shop online, in person, across social media, and even in AI chats.

If you're ready to launch your business before Black Friday, go to Shopify dot com slash decoder to start your free trial. That's Shopify dot com slash decoder. Go to Shopify dot com slash decoder today to hear your first Support for the show comes from Granola. Have you ever left a meeting thinking that was productive, and then ten minutes later, when it's time to take next steps, you're at a loss. Or worse, you've been heads down the entire meeting, furiously typing notes instead of actually listening, and now your brain is fried.

Meetings are a mess. Granola fixes that. Granola is an AI-powered notepad that makes your meeting notes useful. It capture what happens in your meetings and turns it into clean, structured notes with decisions and action items pulled out and made easy to find. And the best part, Granola works through your device's audio, which means it integrates seamlessly into the video conferencing tools you already use. No setup, no awkward bots, it's just your normal meeting with superpowers. You get to actually listen instead of frantically typing every word, and still walk away knowing exactly what was decided, who's doing what, and what comes next.

Once you try it on your first meeting, it's hard to go without. You could try it totally free. Just head to granola dot ai slash decoder. That's granola dot ai slash decoder to get your time back. Go to granola dot ai slash decoder and try it on your next call.

We're back with AWS CEO Matt Garman, getting into the classic decoder questions about structure and decision making, and what that tells us about where Amazon is going next. So this leads right into the classic decoder question, How is AWS structured now? What's the org chart?

What do you mean? So say more about that, what do you what? Just where is our org structure?

Yeah, how how have you structured AWS? I mean, you're you're new, so I imagine you might change it, but how is it structured right now and how are you thinking about changing it?

Uh, I will say that an org structure, uh, number one, it's a living thing. And so, you know, whatever I tell you today may not be true tomorrow, and and I think you have to be agile there. But what we broadly, how we think about structuring our teams, um, I think is is pretty well documented in in the industry around Amazon, which is we want single-threaded teams that can focus on a particular problem and move fast.

What that means is you really want a team who can own a problem and not be matrixed across ten different things where they have to coordinate a bunch. In some ways, I think about it like a computer program, where when you have a big monolithic computer program, it's very efficient as long as that monolithic computer program is small. And as it gets bigger, and then you have multiple people working on that program, then you get a mainframe and it's very slow and it's you can't iterate on it and you can't move fast.

And so what you do is you decouple, you then build services, and they talk to each other through well-defined APIs, uh, and then you continue to decouple those programs, you continue to refactor, right? That's how you build modern, uh, technology systems. And you can think about containers as the current way of doing that, which is really small, independently running systems that can kind of talk to each other through, again, through, um, APIs. Now, if you think about org structure, it's not that dissimilar from that.

If you think about how do you have teams that can real run really fast, and there is gonna be coordination. But what you wanna do is minimize that coordination tax as much as possible. And so if you have kind of a well-defined API between them effectively, right? Which is like, I I build a service over here, you build a service over here. We can go innovate occasionally, we'll our teams will get together and make sure that we broadly know what our vision is, right?

We wanna know what the thing is that we're running towards. But then I can go and my, in particular, my service or my organization or whatever it is, or my feature can run independently and not have to, you know, have a coordination. Like if high level, if the EC2 team and the S3 team had to talk every time they were gonna go launch a feature to make sure it worked together, we would move really, really slow.

But we don't, and so the teams can move really fast, and then we make sure, like we have it's kind of part of the leadership and the and the product leadership team to get together and say, okay, we think going after this space is super important. And some of that is customers are going on to this use case, and so broadly, we're gonna have to go after this thing. But we can still then g have the teams go out and run fast.

That is a organizing principle that and and then there's other parts of the organization where we have teams that run kind of the the data centers and other global and and some of those are our separate teams. But if you think about the product and organizing around the product and technology, that's how we think about it.

This question is always bait for Amazon executives in particular, because Amazon executives are raised in a culture to think exactly in this way and describe the company as a series of microservices. But how is AWS structured?

Yeah, just like that, I mean, it is like, even more But like, how so than Amazon.

go th- go through it. What what are the services? Like, how do you think about how do you think about allocating the teams to the services?

There's 200 different services, so I'm not gonna go through all of them, but, uh, but that is it. And and and we'll continually refactor and we think about them. And and so from a a technology point of view, we think about a compute service, right? You can you can think about EC2, and then you can think about EC2 networking, and then you can think about how do we make sure that it's optimized around containers? And then all the way down to the bottom, you think about, you know, how do we have teams of of ten to twenty people that are are focused on a subcomponent of that that that is is fully separable.

And so, you know, we have f- thousands of developers that are all organized on that principle. Sometimes we'll move them around organizationally, but it's not really the org structure. Like, the key piece is that really ownership at the bottom. The the top part is just how are you efficient in management and how do you make sure that you're managing the teams well and and and doing that high level coordination bit, and that's actually where you move around.

But at the core, those teams are are pretty solid, and and as you find a new opportunity, you spin up a new team that goes after it and figure out, you know, where does it make the most sense in org structure? But but at the core, like, that is the organizing principle is we have those small teams and, um, and we continue to drive them. So that's it. And then and then we organize the our sales and and go-to-market teams and marketing teams kind of separate from that, but from the core product side, that's how we think about it, and, uh, and it works well for us.

I think the the positives are are, um, you know, look, there's pros and cons to any organizational structure. From our from our side, the pros significantly outweigh the cons from the con side. Um, sometimes, and and you've heard, I'm sure you've heard this criticism and feedback of AWS, which is sometimes it seems like it's not perfectly consistent, or, you know, this XYZ feature is not supported across every single service yet, or things like that. And and that is the downside of that organizational structure is your fit and finish across every single service is not always perfect.

And sometimes it takes a little while to catch up to all of those things, which is, you know, expected as you're having a thousand different teams kind of run at different paces on on different things. But the trade-off is we get to move really fast, we're super agile, we can respond to customer feedback really quickly. Uh, and I think that is the other secret to that, that it's not just an organizing principle, but it is also you teach those teams to really listen to the customer.

Like, re you know, and truly, I'm sure every leader you have on here says they listen to their customers, and I don't believe that they they like. Amazon does a really good job of actually internalizing that all the way down to to every individual contributor. And we think about, how do we go solve customer problems? And when you're small and you're agile and you can make decisions, you can actually go solve customer problems really fast, um, in your area.

Those things play on each other and are helpful.

You did start as a product manager, as a product Technically a manager,

technically an intern when, uh, when AWS before AWS

that's true.

launched in 2005. But yeah,

but as a PM, right, you're you're running some product, you're pri- thinking about the customer a lot. What were the frustrations you had as a PM that you think you can now reduce as the CEO?

it was a very different business back in the day. I was the product manager for all of AWS. So, uh, so, you know, I, yeah, exactly, the same job now. Uh, and I I kid, there was there was a couple other product managers at the time, too. But the the frustrations now are then are and now are are also similar, uh, different. It's obviously a different scale that we're operating at. But one of the things I was frustrated at back in 2006 was I knew a ton of things that we needed to go deliver for our customers.

Like, I just had a huge list, and it was all about prioritizing that list. But I wish that we could deliver them faster, um, and do more. Uh, and even at the scale that AWS is today, that's still true. Like, I wish we could do more and do it faster, and that's part of why focus on that organizing principle of how do you make sure that that you can get out of the way of the teams to move fast.

Um, and so my job today is a little bit more of, how do I remove those barriers and help teams move fast? But that's it. I think it's a lot of we wanna make sure that we're innovating, we wanna make sure that we're we're leaning ahead. You know, some of the challenges we have today are are different than we had in 2006. 2006, we had to answer the question of like, why would a bookseller ever run my computers?

And that question we get less and less today, actually, rarely. I don't think I've gotten that one for a while. But now we have to deal with scale, and you have to think about enterprise requirements, and you think about, how do I meet auto requirements? And how do we support governments? And how do we think about scale? And, um, how do we make sure that we have enough electricity in the world and all of those kind of questions?

But all good problems for us to solve so that we can take them on so that customers don't have to.

This is the other big decoder question, and it's gonna lead us right into AI, cause I think you have a lot of decisions to make here. Amazon famously has the one-way door versus two-way door decision making framework. Everyone kind of applies it differently. Every Amazon executive I've talked to they hold on to that idea and they apply it differently. What's your decision making framework? How do you make decisions?

Well, part of my job is to make the the one-way door decisions. I think that framework is it's a it's a useful one to think about. And just to clarify, in case you're not aware of it, it largely is that's how you go fast, is you try to des- define what are those decisions that are they? They can be important decisions, by the way, I think sometimes misunderstood. That's like, what are the important decisions and not important decisions?

It's not that. It's, uh, you want the people that are owning those teams at the edges of the organization that really own those products to make important decisions, cause they they know best about their product. But there are also decisions that could be undone if we decide that it wasn't the right thing to do. And then the the bigger kind of, you know, I'm gonna go invest $1 billion or or some decision, or I'm gonna launch a new service that is hard to kind of pull back or is painful to pull back.

Those are the the one-way door decisions that I think we want to have a little bit more inspection on. And even those, though, I think we are, you know, I think we're trying to figure out how do we make those faster too, and and uh, enable a a broader swath of people to make those. But you asked how I make decisions, you know, I think for for better or worse, uh, my my take is I am rarely, uh, if ever, the expert on any particular subject that we're that we're working on.

And whether if we're working on compute, if we're working on storage, if we're talking about hypervisors, if we're talking about sales compensation, if we're talking about, you know, power contracts that we're signing, if we're talking about, you know, whatever it is across technology, if we're talking about go-to-market efforts, if we're talking about marketing, I I am, uh, almost never the expert in the room on that. And so I make sure that I listen and leave space for those experts that spend all of their day thinking about that to weigh in as to how they've come up with their recommendation, how they think about what we should do.

And then the part that I bring to that is to, one, take a view of a non-expert and ask some questions and understand how they're thinking about the problem. And then also help connect the dots to the other part of the organization that they may not have visibility into and understand if there's trade-offs that they've maybe not thought about. Because, you know, they're making a marketing decision and didn't know about a new product that we were delivering over there.

And so I try to make sure that as an organization, we've connected those dots and then ask the right sets of questions. And then if, look, if there's a tiebreaker decision, I'll have to do it so that we can move fast. Cause I don't I think the the the place we don't wanna be in is sit there and just debate forever. We at some point you need a tiebreaker decision, and that's kind of what I I view my job as doing as well.

All right, so I think this does bring us straight into AI, because this is a bunch of decisions that everyone has to make, and the outcomes are, I would say, still uncertain. As an industry, everyone is telling me this is the core enabling technology of the next generation of computing. This is a a platform shift is the phrase that a bunch of CEOs have used with me. Do you think AI is a a platform shift? Do you think it it's that big of a deal, or is it just another suite of capabilities that AWS will offer people?

It's a good question. Uh, I'll I'll start with, I believe that that AI is incredibly transformational, whether you call it platform shift or not, I'll get to in a second. But I think it's an incredibly transformational technology that more than kind of, you know, look, these things come around every decade or so, and I think it is one of the technologies that has the ability to be completely transformational, whether it's transforming industries, transforming companies, transforming jobs, transforming workloads, like I think, or for workflows.

I think it has a real potential to have a material impact on every single piece of of how we think about work and life and user experiences and and and the like. I I'm a full believer that that is true, and and and I I think there's a timeline question, right? Like, is that gonna be in the next twelve months, or the next twenty four months, or the next five years? But I I do think it is gonna happen, and it's gonna be it's gonna have a real change on a lot of pieces of business.

Platform shift is an interesting question, cause platform assumes that, like, AI is not yet a platform, and I think that that is a more open question. It's a it's a huge enabling technology. And whether you build on that AI or that AI is just kind of embedded in everything that you build with and is a core component of what you build with and how you think about it's a tool that is really meaningful and impactful, I think, is it remains to be seen as to exactly what that that means, but it is a it is a transformational technology

Wait, can I can I make that that, um, simpler? Can can I put that on a spectrum for you just to to make this more concrete for the listener? Do you think AI is more like multi-touch, or do you think it's more like the iPhone?

I don't know if it's really like either of those. I would bet Well, cause that it is.

multi-touch is like, you can't make an iPhone without multi-touch, but that doesn't imply that we're all gonna start using multi-touch

it's not like multi-touch.

phones, right?

I don't, uh, it's not like that. I I th- I don't know if it's like an iPhone either, though. It may be more akin to the internet disruption. That's what I'm saying, like, I don't know if the internet is a platform per se. It's like a a a ch- a shift is the how you deliver an application. So maybe it's platform, but it is, I think it's more akin to like that, where there will be fundamental shifts in into, like, how you deliver products and offerings and services, and and do your work on a daily basis.

And so the internet has been hugely transformational in how you do your work on a daily basis, right? You used to sit there on a typewriter or, you know, I don't know, write memos or or or do whatever, and now you're on a computer all day, and you're you're interacting on either SaaS applications or you're emailing people, or, you know, there's there's a just a fundamental connectivity. And I do think that that AI is more akin to something like that, where it kind of has that fundamental shift into how you're gonna get work done.

Yeah, I think you and I are both about the same age, and you describe the typewriter workforce with the same sort of like, I think that's what it was like. Um, I never had a I never I never had a job like had a job like that. that. It's the same way I did. I'm like, I think the typewriters, people had them. Um, the timeline thing you brought up is really interesting, right? What is the timeline for this?

Uh, and it's particularly interesting to me, uh, because I get a bunch of AI CEOs coming on the show telling me, uh, what their timeline for AGI is. So, Sam Altman, uh, recently said AGI will be possible on current hardware, and OpenAI is making a lot of noise about AGI for a variety of reasons that we can unpack at a later time. Uh, Mustafa Suleiman, who is the Microsoft AI CEO, was just on it, and he said, I don't think we're gonna get to AGI on current hardware, but maybe within two to ten years.

And he said, Uh, we definitely are not gonna get there on Nvidia GB200s. You run data centers, you have a bunch of Nvidia chips in these data centers, you are developing your own chips, which I want I want to talk about. Where do you see yourself playing in that debate? Is it one of these vendors is gonna light up AGI on someone's data center, and I hope it's AWS? Is it I'm building this hardware to enable that to happen?

Is it this is what everyone's talking about to goose their stock prices, and I just need to sell more capabilities to more customers.

Well, number one, it's a it's a literally impossible question to ask because there is no definition of what AGI is. So when you reach it is also an impossible definition, cause I don't I don't know. You can't, you can't define when you reach an undefined thing. What I would say is I think that it's just a continuum, and I think that AI we'll call it AI inference, right? The the ability to go do work is going to continually get more capable over time, and I think that there is a long road of this to get much, much, much more capable over time.

And it's gonna get much less expensive to run over time, which I think then explodes the number of ways in which people will find ways to to make it useful, whether it's running agents, whether it's doing other workflows, whether it's long running, uh, reasoning tasks. Like, I think there's a whole host of of things that you can imagine. And and so there's just a continuum of of where eventually the the things land and where you're able to ask the the computers to do more for you at at lower costs.

And I think hardware platforms are gonna play a big part of that. I think software algorithms are gonna play a big part of that, and they'll be, um, you're gonna need both of those. A And so, you know, I I don't know when you reach AGI, I don't know what that means, but I do think that the next generation of compute will be, you know, it's gonna deliver somewhere between and and whatever the the current generation is gonna that we just announced with Trainium two, and eventually, uh, with, uh, Blackwell and and GB200s, I think will give customers a two to four x boost in compute capability per dollar.

And, uh, you know, we announced Trainium three, which will give another two x compute boost to compute by the end of of twenty five. And so that is gonna help. That you'll continue to get more and more, and you're gonna be able to do bigger and bigger things, and you're absolutely gonna need algorithmic improvements as well, which many of the the teams, ours included, are very focused on doing.

But just straightforwardly, if OpenAI declares that it has achieved AGI, which it seems very much poised to do, it will have done that on a bunch of Azure data centers. Do you think AWS needs to credibly claim, Oh, we can do that too, to compete with Azure? again, they've, um, again, I mean, they def-

you can be clear.

they've defined AGI down, to be clear, but they're gonna say Yeah, I it when they understand there are contractual say it.

terms that they're working through, but, um, there's they have some motivation of reasons to do that, from my understandings. But, uh, it's not about declaring anything. Like, it is just a let's go figure out what you as a customer like. This this is again, I am less interested in puffery in the press and more interested in how I can help customers achieve actual outcomes. And so it's fine, like you can there, there can be marketing statements, there can be, like, I have the biggest compute cluster in the world, I can be, I am, I have AGI.

Like, okay, at at some point, I wanna help a bank figure out how they can reduce the amount of fraud that they're seeing, or improve the speed of which they can approve loans, or or whatever the thing is that actually goes and helps a business. I wanna help biotech find cancer cures faster and better, and and figure out how they can significantly shrink and or improve the f- uh, efficacy of of what they find. Like, and so those, to me, are interesting and useful outcomes.

And so if if you tell me like, Hey, can you help a customer find cures for cancer faster? Awesome, like, that is a thing that I'm focused on. Uh, uh, whether was that AGI that did it or not, I I don't, I don't know. It's like, you I'm not interested in that per per se, I'm more interested in, can I actually help our customers deliver value to their businesses? And a little bit less on the like, can I, like, have a stake in the ground around marketing?

Cause I think at the end of the day, customers actually care about that first one, not the second one.

we need to take another quick break. We'll be right back.

Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow none of them talk to each other. That's where Odoo comes in, an all-in-one business management software that brings every part of your business together. From sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system.

Try for free today at odoo dot com slash vox. That's O-D-O-O dot com slash vox.

Support for the show comes from CrowdStrike. AI is already fundamentally changing how companies build products and run their businesses. If you listen to the show, you probably already know this. But while AI drives innovation, every new model and agent also creates new cybersecurity vulnerabilities. That's why CrowdStrike built AI Detection and Response, or AI DR. AI DR helps organizations discover where AI is being used, monitors AI activity at runtime, and helps teams detect, investigate, and respond to threats involving AI systems.

As companies adopt AI, security has to be built in from the start. More than 70% of the Fortune one hundred trust CrowdStrike to protect their businesses. Now, CrowdStrike is helping secure the AI era. You can learn more about AI detection and response at crowdstrike dot com slash decoder.

Support for the show comes from Framer. If you wanna create a website that looks great, feels handcrafted, and comes together fast and seamlessly, then look no further than Framer. It's a complete website platform, not just a builder, so teams can launch and keep improving their sites in one place. It helps creators, teams, and businesses ship production-ready sites faster than ever while getting every detail right. You're able to prompt, inspect, edit, and publish in one place at a whole new pace.

On Framer, agents and humans work in tandem. Agents bring speed and scale, while people bring taste, judgment, and control. And that all takes place in the same workspace where the real site is designed, managed, reviewed, and published. Thousands of businesses, from early-stage startups to Fortune five hundreds, are choosing to build their websites in Framer, where changes take minutes instead of days. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer dot com slash decoder for 30% off a Framer Pro annual plan.

That's framer dot com slash decoder. Rules and restrictions may apply.

We're back with AWS CEO Matt Garman discussing the industry's massive investment in AI and where Amazon is putting its resources. I think this leads right into the the next piece of the AI puzzle that I'm seeing unfold. And it it's really like, where should the investment go? Is it training new models, which might be hitting a sort of scaling law problem and getting less capable at a slower rate than they were before with every successive model, or is it in inference, which is what you're describing?

Hey, we can bring the cost and speed of inference down on the existing models and make cheaper, better products, more cost effective products. Where is your emphasis right now?

Like, I don't think you can pick one or the other. You absolutely like the the world is gonna deliver more capable models, and they are expensive, and they require a lot of compute, and it's an it's an area of investment for us, and it's an very investment for many of our customers. And it's I I think it's the right area of investment for a lot of those, because I do think you you don't get more capable smaller models if you don't have the large model to start with.

Like it is, it is, that is just how it works. You don't, you can't come out with something that's a really, really powerful small model if you didn't also build a frontier model or start with frontier model. So you have to have those large frontier models, and I think we're gonna need those to be more capable. And there's a lot of innovation in inference and how you can drive cost down. And some of that is a systems problem, some of that is a hardware problem, some of that is algorithmic problem.

You can think about model distillation. There's a whole bunch of techniques that you can do to get these smaller, faster inference models, which I think are gonna be hugely impactful and important to delivering real value to enterprises. I think you you go talk to customers now, and they are no longer interested in bright, shiny AI proof of concepts. They want something with real ROI associated with it, and the way you deliver great ROI is you either have more value and or less cost, and I think both of those are gonna be important to keep raising the level of of ROI that you can deliver on.

And and so if, as we think, there is this massive ability to c transform organizations, you have to keep increasing what models can do and decreasing how much they can cost. And so I d I don't see how you pick one of those, I think you have to do both.

If you had to pick one, would you? It it sounds like you would pick inference, right? Because that's where the products are getting built.

Yeah, well, which I'll tell you, is, uh, uh, in my, uh, keynote at in, uh, in reInvent, I talked about another thing that I like to to do at Amazon and we d we do here, which is, uh, we refuse a thing that we call the tyranny of the or, which is forcing someone to pick A or B stifles innovation. Because it means that you don't go out and invent how to do A and B. And so, uh, you can't pick.

I'm just telling you, like, it is not an A or a B chance, it's an A and B, and we have to push our teams to figure out how to do both, which is enable bigger training, and we have to lower the cost of that, by the way. It can't just keep scaling linearly, which is all part of the the silicon invest, uh, investments that we're making and and networking and things like that. Which is, how do you make the cost to train these really large models lower so that you can train bigger models?

And I think we have to make that investment, we are making that investment, and it's a huge area of opportunity for us because today it's too expensive to continue to ramp at the rates of of the cost of the infrastructure. And so that's a big part of Trainium is investing in how do you get the cost down, uh, for training. And I think the inference side has to to to drive cost down too, which I think is incredibly important for the adoption side of it.

So you you have to do both. There's they won't work if you just do one side.

I did watch your keynote, and you are welcome for that alley-oop on the tyranny of four. Um, and it was coming, cause I wanna ask about trainium. This is a huge investment, right? You've been at it for several years. You announced Trainium two at reInvent that has additional capabilities in training and in inference, right? It's designed to be good at inference, so you can use the same chip everywhere. Building these chips is a huge investment, and you are up against dedicated chip companies.

You're up against AMD, which is also making a huge investment. You're up against, uh, Microsoft, which is making its own investments. You're up against Nvidia, which is the leader and has a huge head start, not only in the chips, but in the software ecosystem around the chips. How are you thinking about that competition and that investment?

it's less a competition and more a an addition of choice, cause I do I don't think that it is, uh, GPUs

Oh, by the I should I should probably point out that Google has, I think, a pretty advanced data center and AI capability of its own.

that's right. And and so it turns out, we've been making chips now for over a decade. Um, so we're we've been making silicon chips, our own custom silicon, for more than a decade. And so we have one of the most experienced teams in the industry doing this. And so we're it's not a new thing. It's not like we dove in here and said, we have no idea what we're doing, which is, by the way, some of those others are learning it for the first time, not not Nvidias and, of course, AMDs and and and Google's been doing it for a little while, too.

I think Microsoft is is pretty new to this space. But we think that that that is a big advantage for us is we understand how to do this at scale, we understand how to do it in the cloud. I think we have some advantages in that we don't have to do it for a broad set of customers. You know, we have to deploy our chips in exactly one environment, right? We have to deploy them in an AWS data center, we have to do them in in deploy them in exactly one server.

We don't have to support a a whole OEM infrastructure, we don't have to support a set of different drivers and a bunch of different things. It's just in our environment, and we know exactly what that's gonna look like. And we think it's a choice. We don't think that it has to meet every single use case for every single customer. We think that NVIDIA GPUs and AMD GPUs and and others are gonna be s super interesting. Like, they have good platforms, they all both of them have very good teams that are executing really, really well, and I think they will continue to do that.

I don't see any reason why they wouldn't, and we plan to be a great partner of theirs for a really long time and support that and offer it to customers for when it's the right technology choice for their use case. And we think that we can offer interesting choice, and we've done it with Graviton. We've proven that we can launch a processor at broad scale that is very useful for a set of workloads, a lot, a broad set of workloads for our customers.

And it doesn't in Graviton's case, it doesn't mean we don't buy a ton of Intel and AMD chips and offer those to customers. We, of course, do, um, and those are growing businesses for us as well. It's just more choice and it, we think that choice makes AWS a more attractive platform for customers because they have more choices than they do other places. Uh, and so that additional choice is nice, and there means part of that choice is we wanna really lean in, make sure it's the best place to run NVIDIA GPUs and and AMD and Intel and and others.

And so we think we think it's a but it's a big opportunity for us. And and if you do think, which we do, that AI is gonna disrupt all of those different industries, it's a massive opportunity where it's not one player is gonna be the only compute platform that all of those things run in over time, and we think that we have an opportunity to to build some of that and provide differentiated choice for customers that choose to run AWS.

Chips and chip investment is a long-term decision, right? You're you're making decisions now, you're allocating capital now that might not pay off for a decade or more. Do you think that model training is hitting a scaling limit? That it's gonna plateau the way that some people are saying it's plateauing?

I I think people like to talk about scaling laws cause again it sounds fun to talk about, but I th- I think that it probably just means there has to be more levels of invention, like I think if you look over, uh, in any technology ramp, you kind of see a, uh, one particular technique ramping like this, and then it slows down, and then somebody says, Oh, how about you try this? And then it goes back up again, and then you try something else, and and so there's gonna have to be software and algorithmic changes, I think it's not a blind, like, dump more data and more compute and, you know, close your eyes, and you get a bigger model

next, you know, next year. You're gonna have to have smart people looking at it and driving it and figuring out new ways to to help that, but that doesn't mean that you've hit a limit. I think it's just you're gonna have to keep innovating on different ways, and you think about again, eh, think about number 1, how how long and it was a longer than a decade that people were saying that we were hitting Moore's Law scaling limits.

That was just, you know, can you take 17 nanometers and make it 15 nanometers and 13 nanometers? And you're saying like, Okay, there's gonna be a limit. Number 1, they had to figure out technology to get past a couple of those, right? I remember it somewhere around 10 nanometers, people were like, I don't think you can get past this. And now we're building 3 nanometer chips, right? And so you keep getting smaller because there's new technologies in there, you had to figure out how you deal with interference, and you had to think about how you think about actually stacking the memory and different structures of the chips and other things like that, but you work through those, and in the

meantime, you kinda figured out how to do more compute on an accelerator, like a GPU, that then gave you, you know, a huge step change in compute, and and so no longer people are worried about, Did you, you know, are we hitting the limits of what a 17 nanometer Intel chip from 10 years ago was doing? Right? Now we're we're orders of magnitude more compute than that, Well, hold on, And, uh

hold on. I mean, this is this is the real limit. One company figured that out. TSMC figured that out using E an EUV machine from one company in the Netherlands, right? And they're the provider for everyone, which means you are now asking TSMC for capacity in competition with NVIDIA, in competition with Apple, in competition with Qualcomm, Wi- in competition with AMD, even to some extent in competition They they figured With Intel, right?

Out parts of that, I mean, they figured out the the the layout chip, and by the way, like, CC and the team did a fantastic job of figuring it out, so, yes, but the the world figures But, like, It out, right?

but Intel famously did not figure this out, right? It did, Like, I mean, But but, that's where they Um, but Are right now. but others Um, but I'm saying

have, right?

Right now, the the bottleneck in the chip industry in the investment is one company can provide this product. Do is that something that you actively think about, like, do they have the capacity to let us compete with?

Uh, I think I mean, they're making lots of investments, and I think they're they're scaling, and I think others are looking to to catch up in that space too, and they have a great lead, and this is also true in technology and has been for a long time, is that somebody jumps ahead and figures out a gets a lead, and it's a benefit for them for a while, and others catch up. And I think you you can look at some of the the HBM memory and some of those other fabs that are that are coming up, and they're they're they're catching up and finding other new ways to do that, and, um, and there will be other inventions that leapfrog

over time. But obviously fabs are hugely capital-intensive, uh, investments and others like that, and so I am sure that others will eventually find new and and different ways to innovate around that too. It has always been true in technology.

Are you making any bets on any non-TSMC fabs?

I wouldn't have anything to announce there, but we we partner with lots of folks, we partner with Samsung, we partner with Intel, we partner with others that have their own fabs as well, and, uh, and buy lots of of other stuff from them, from from memory to to CPUs and other things like that, so we buy parts from lots of different fabs around the world.

The other big constraint is power. You have said 2 to 3 generations from where we are in AI, we're gonna need 1 to 5 gigawatts of power, which about a medium city. This led you to talk about nuclear power and how we're gonna need that. That's a big deal, right, to say, okay, we're gonna need so much AI capacity that we're gonna build nuclear power plants. Microsoft has said the same thing, other companies have said the same thing.

Is that still where your mind is? This is gonna be so successful that Amazon is gonna go try to build some power plants?

Yes. It is, we've we've made, uh, significant investments there, and it's a range of things, by the way, it's a it's a portfolio, and this is not a new plan for us. Over the last 5 years, we have commissioned more renewable power projects than ever each of the year each year for the last 5 years, we've commissioned more than any company in the world. And that's bringing on new power into the grids, you know, and whether the new solar farms or the new wind farms, and now we're adding nuclear to that.

And so it's it's just a portfolio of that. I think the world is gonna need more carbon free energy, and compute and data centers are a big portion of that, and we're pushing hard to make sure that the world has enough sources of that, and I do think that nuclear will be an important component of that plan over the next couple of decades. And so we're you know, we're excited about, uh, small modular reactors. I think that that it's a technology that's a little ways away, by the way, it's not it's solved for the next couple of years, but past 2030 and and beyond, I think it could be a very important component one, cause you can actually put it

near where you need the power to be. Another of the bottlenecks that we run into is is around transmission. It's not just power generation, but it's transmission. So you can have a a a solar farm out in the desert, but if you don't have transmission to get it to where your data centers are, then it it doesn't do a lot of good. And so those those are both problems that need to be solved, and and it's not just data centers, it's, um, it's electric cars, it's electrification of all of our businesses.

Like, there's a bunch of these things that are gonna need to happen, and so I think nuclear's gonna be an important part of that, but both small modular reactors, and I think the world's gonna have to build more of these large, you know, industrial scale nuclear plants as well. I think a lot of people's heads are in the, like, you know, that was scary back in the 50s when the technology wasn't as safe. Today, it's a it's a very safe, um, scalable technology, but it's, you know, it's it's something that we have to keep spending and and scaling.

We're gonna have you back for another full hour on nuclear power plans. Uh, that's a whole rabbit hole that I I I definitely wanna talk about at some point in the future, but we're we're running out of time here, and I just wanna ask kind of the biggest question of all. This is a lot of huge forward investment. You're designing chips, we're investing in TSMC's capacity, we're talking about nuclear power plants, we're building bigger data centers, there's an $8,000, 000,000 investment in Anthropic to help build a data center and then run Anthropic and Claude.

When is any of this gonna make a dollar? Right, cause you need a you need a product in the consumer or the enterprise that throws off enough margin at enough scale to fund all of this investment and still make money for the the people making the product, and ideally, the people paying for the product are using it to make more money on the other side. Like, the economics of this are still very unclear to me unless you are Nvidia.

When does all this make a dollar for you?

Yeah. Well, uh, A AWS is a is a is a nice profitable business for for Amazon,

Right, you've got the margin So to spend on it, but it's not like it has And so to return.

It's, you know, I think, look, it's and and for customers, they are increasingly looking at it this way. Like, it's not just us, it is, um, you like and I and I said this a little bit ago. If you talk to customers, they are very focused on how do they have ROI positive AI projects? And I think the cloud has already proven, you know, across a broad swath of industries to be ROI positive for for many different industries.

I think we're we're kind of like moving your data to the cloud, your compute to the cloud, you gain agility. That, I think we've proven that we can deliver great ROI for customers in in moving to the cloud broadly, broad speaking, and taking IaaS side. And so what we're increasingly seeing customers say is I wanna see the ROI of of these AI projects. And I do think that that is a an important shift where it is not cool, it's not just the cool, it's not just the glam, you know, the the shiny object factor.

It is, uh, how do I make sure this makes sense? And we are spending time with customers thinking about that, how do you work through the the use cases that are enabled today that can deliver real value? And some of those are are are kind of broadly reported around things like modernizing your contact center, and and we think Connect is a is a great offering for customers to do that, and we're actually seeing huge number of customers move to Connect in a in a cloud contact center to take advantage of many of those AI capabilities.

Um, you see some of that in in optimizing some of your back office projects. And I think increasingly as the, uh, agentic workflows really get much more powerful and and as we think about collaborative agentic workflows and longer running agentic workflows, you're gonna see more and more value come up through these, and and as the models get more capable, you're gonna see more value coming up through those, and so it's on us, it's incumbent on us to make sure that these are very profitable for end customers to go and, uh, and implement.

But let me just put that in a in a framework that makes it maybe a little bit sharper, right? You've been at AWS since the beginning. AWS started and I'm I'm gonna flatten this narrative, and you can correct me for it being a little too flat, but just in the flattest possible way. Amazon is building a bunch of these services. Hey, we have excess capacity. Hey, we wanna build microservices for our own components. We can resell those.

So you get a bunch of benefit along the way of just building Amazon, and you can turn that into a business. AI right now, it feels like there's a bunch of ideas for products that might be useful inside Amazon, outside of Amazon, for AWS's customers, whoever. But it requires a massive amount of forward investment, right? It's not just we're kind of doing it anyway. It's much more, Hey, there's a huge opportunity here. We need to leapfrog ahead and maybe get some more customers, and maybe there's a platform shift or whatever it is.

We all see the huge promise, and that is happening at a subsidy. And that subsidy Uh, yeah, so I seems dangerous.

Just it's it's not it's not the right characterization of it, so there's a couple things I would say is number one is AWS was never about excess capacity about about Amazon. It just just, like, math doesn't work, right? It's you can imagine, like, I've heard that narrative, it sounds nice, and as soon as Christmas time comes around, if I take have to take Net Netflix's servers away so that we can support retail traffic, like, that doesn't really work as a business, so, like, it just it was never like, uh, like, the it just it is that was never the the the idea, nor the intent, nor the goal of AWS.

A and we we built the businesses from scratch, right? They weren't, like, reusing AW like Amazon components. We we we learned from that. They were an incredible early customer to learn from the components that they would need, but we built them from the ground up to support a broad range of customers. And so AWS itself was a big investment by Amazon to go after a a broad new business. As you think about now, I we had Amazon as a big customer of ours, for sure.

And and they were a super helpful customer for us to learn about what large enterprises would need from services like AWS and continue to be. Uh, I think AI is is not that dissimilar. Like, Amazon needs AI, and and I think if you you you mentioned that you watched, uh, my reInvent keynote, Andy was up there for for 25 minutes talking about all of the cool things that that Amazon, the the rest of Amazon, is doing with regards to AI, and you're talking about Rufus, or talking about how we're thinking about our supply chain and fulfillment centers and and across the whole scope of an Alexa, we actually that business desperately needs AI capabilities to, again, reimagine our business, get

more efficiencies, deliver new experiences for customers like we are, uh, Amazon is customer number one for a bunch of these capabilities, and so if AWS can build them and Amazon can take advantage of them, that's fantastic, and and both of those things are true. So, yes, it's a big forward investment, but we also have Amazon still using these, and we're in a different place now. When we started in 2006, you know, we had 0 external customers, and we now have a million external customers, or or multiple millions of external customers, right?

And so that is a huge customer base that is ready and willing and excited to buy and use the products that we have, and so that investment, it is forward investment, but you also have a really big base that you can amortize across and go offer it to, um, which makes that investment thesis a a little bit easier to get over.

Alright, so I'm gonna ask you the same question again to wrap up. With all this context, when do you think all this investment will become ROI positive?

Uh, I mean, I think it's ROI positive. Well, uh, it depends what you mean by ROI positive. I think there's a lot of investment in the world. Um

Right, but this is a lot of investment in AI across the industry. When do you think it's gonna start If if re returning?

you think globally, I think it's ROI positive now. I think the question is, when does it become more evenly distributed? Um, look, I think the hardest question of that, honestly, is for the model producers. I think that's the the single hardest question. I actually think it it very quickly today, or if not today, very soon, is gonna be ROI positive for the broad swath of customers using AI and building it in, right? Banks and insurance companies and pharmaceuticals and others, like that you can make that ROI positive story today, and I think it will continue to get better.

And I think for infrastructure providers like like NVIDIA, of course, it's very ROA- or they're they're Yeah, they're They're very

doing fine.

positive now. I think the question is when does the cause the the the folks that are making the most huge investments are are the ones that are building foundational models from a software perspective and then reselling those foundational models. It's a good question. I don't know the answer to that of when that that investment kind of fully pays off for an OpenAI or an Anthropic or a you know, I think Amazon and Google, uh, probably have a different math of when you can make those pay off cause you you get kind of internal usage of them from your own use.

I don't know that, but, uh, but there's a lot of smart people investing in a continuing to put investment in a in a broad swath of AI companies, so, you know, and and you have to believe, which we do, that there is a a massive economic benefit from many of these AI capabilities that are orders of magnitude bigger. And I do think it it really does play into that math equation. As inference gets cheaper and more capable, that there are multiple orders of magnitude more inference to be done, and that is kind of when it and it ultimately starts to pay off, I think, for a lot of those model providers and, uh, and in a in a huge, massive

way.

Alright, you are clearly in the weeds of all these products, which is fun to hear. Uh, let's end here. Last question. When you're trying out all these AI products, which is the one that you use that made you think, okay, this investment is worth it?

Hm, that's a good question. Um, I don't know that there was any one product that's, uh, that I got excited about. At the first time, the first product that I ever used that said, hey, I think this is real, is just like everybody else. Is like, I think ChatGPT was just a transformational product. Uh, it was a it was a great UI, and it really unlocked for everyone what was possible. So that, you know, the first time that I that I really kind of realized that this was, uh, gonna take off, right, and we were making investments internally, but I think we were hopeful that they would get there.

Um, I think that's the first one that I used that I was the that I really understood. Now it's hard cause I use thousands of them, and I think all of them are really cool, and and I think there's a lot of startups from people that are building AI products that are making new proteins, which is incredible, to folks like Perplexity that are making search engines that are much more interesting to contact centers, to banking applications.

Like, there's a whole host of them now that are are incredible. I think Amazon makes some, and and many of our partners make many, so the those are all incredible, but it really was like, uh, just just like the rest of the world, I think ChatGPT was the first one that really kind of helped solidify it.

Got it. Very diplomatic answer. Matt, this was great. You gotta come back. I really enjoyed this conversation.

Great. Thanks for having me.

I'd like to thank Matt Garman for taking the time to join me on Decoder and thank you for listening. I hope you enjoyed it. If you'd like to let us know what you thought about this episode, or really anything else you wanna talk to us about here in the new year, drop us a line. You can email us at decoder at the verge. We really do read all the emails. Or you can hit me up directly on Threads or Blue Sky.

We also have a TikTok for as long as there is a TikTok. Check it out. It's at DecoderPod. It's a lot of fun. If you like Decoder, please share it with your friends and subscribe wherever you get podcasts. Decoder is a production of The Verge and part of the Vox Media Podcast Network. Our producers are Kate Cox and Nick Statt. Our editor is Callie Wright. Our supervising producer is Liam James. The Decoder Music is by Breakmaster Cylinder.

We'll see you next time.

Sai quante cose puoi fare in 15 minuti? Puoi scegliere una canzone da aggiungere alla tua playlist, scaldare la cena, leggere qualche pagina del libro che hai sul comodino, chiamare tua madre e scegliere il tuo prossimo hobby, che poi mollerai. Oppure puoi donare il sangue. E così salvi 3 vite in 15 minuti. Donare il sangue è un gesto semplice e il suo valore è enorme. Dona vita, dona sangue. Vai su doneilsangue.salute.gov.it Gli operatori di magazzino Amazon hanno accesso a soluzioni di lavoro flessibile.

Come Antonio, che oggi deve andare a prendere sua figlia a scuola. Per lui, questo è il suono più bello del mondo. Anzi, forse è questo il suono più bello del mondo. Soluzione di lavoro flessibile in Amazon. Per esserci nei momenti che contano. Si applicano criteri di idoneità e condizioni specifiche.