Matt Garman said Amazon Web Services had grown into a $110 billion business growing at 19%.
“I think over eighteen years, we've grown to be 110 billion dollar business, growing at 19%.”
Person
Matt Garman
2 episodes · 49 quoted claims
Matt Garman said Netflix runs its own homegrown content delivery network and that this part, not its AWS infrastructure, was stressed during the Jake Paul-Mike Tyson stream.
“Netflix has their own homegrown CDN that they use, and that's the part that I think was stressed.”
Matt Garman said Amazon probably had problems the first time it ran Prime Day because of traffic hitting its website.
“the first time we probably did Prime Day, we probably had issues across that too of just people hitting the website and other things.”
Matt Garman said that in 2006 AWS had to convince customers why a bookseller should run their computers.
“2006, we had to answer the question of like, why would a bookseller ever run my computers?”
Matt Garman said AWS had no external customers when it started in 2006 and now has a million or more.
“When we started in 2006, you know, we had 0 external customers, and we now have a million external customers, or or multiple millions of external customers, right?”
At its re:Invent conference, AWS announced Q Business Index, which pulls a company's SaaS data into a single index that the enterprise owns and controls.
“just at re:Invent, we announced a capability called Q Business Index, where you can basically have all of your SaaS data pulled together in a single index that's owned by the enterprise and controlled by the enterprise”
Matt Garman said many of AWS's largest customers have only 10% to 40% of their workloads in the cloud.
“You can even take our largest customers, many of them only have ten, twenty, thirty, 40% of their their workloads in the cloud.”
Matt Garman said AWS's Verizon arrangement was a partnership to potentially sell edge compute space together, rather than Verizon building a rival cloud.
“Well, Verizon was a little bit different. It was actually a partnership with us where we were talking about potentially selling some of that compute space together at the edge.”
Matt Garman said AWS has about 200 different services.
“There's 200 different services, so I'm not gonna go through all of them”
Matt Garman said AWS organizes product work into teams of ten to twenty people, each owning a fully separable subcomponent of a service.
“how do we have teams of of ten to twenty people that are are focused on a subcomponent of that that that is is fully separable.”
Matt Garman said that under Amazon's one-way door framework, big investments and hard-to-reverse service launches get more executive inspection than reversible decisions.
“I'm gonna go invest $1 billion or or some decision, or I'm gonna launch a new service that is hard to kind of pull back or is painful to pull back.”
Matt Garman said the newest generation of AI hardware, including AWS's Trainium2 and Nvidia's Blackwell GB200s, should give customers a two to four times boost in compute per dollar.
“we just announced with Trainium two, and eventually, uh, with, uh, Blackwell and and GB200s, I think will give customers a two to four x boost in compute capability per dollar.”
Matt Garman said AWS has been making its own custom silicon chips for more than a decade.
“we've been making chips now for over a decade. Um, so we're we've been making silicon chips, our own custom silicon, for more than a decade.”
Matt Garman said Graviton showed AWS can launch its own processor at broad scale, while AWS still buys many Intel and AMD chips.
“We've proven that we can launch a processor at broad scale that is very useful for a set of workloads”
Matt Garman argued that people once thought chips could not shrink below about 10 nanometers, yet 3 nanometer chips are now being built, so scaling limits can be overcome by new techniques.
“I remember it somewhere around 10 nanometers, people were like, I don't think you can get past this. And now we're building 3 nanometer chips, right?”
Matt Garman said AWS partners with Samsung, Intel and other companies that have their own chip fabrication plants.
“we partner with Samsung, we partner with Intel, we partner with others that have their own fabs as well”
Matt Garman said that in each of the previous five years Amazon commissioned more renewable power projects than any other company in the world.
“each year for the last 5 years, we've commissioned more than any company in the world”
Matt Garman said AWS is excited about small modular nuclear reactors, which he sees as a technology still a little ways away that could matter after 2030.
“we're excited about, uh, small modular reactors. I think that that it's a technology that's a little ways away”
Matt Garman said power transmission, not just generation, is a bottleneck for AWS data centers.
“Another of the bottlenecks that we run into is is around transmission. It's not just power generation, but it's transmission.”
Matt Garman rejected the idea that AWS began as a way to resell Amazon's spare capacity, saying the math does not work.
“AWS was never about excess capacity about about Amazon. It just just, like, math doesn't work, right?”
Matt Garman said the hardest question about AI returns is for the companies producing foundation models.
“I think the hardest question of that, honestly, is for the model producers. I think that's the the single hardest question.”
Matt Garman said ChatGPT was the first AI product that convinced him the technology was real, thanks to its interface.
“I think ChatGPT was just a transformational product. Uh, it was a it was a great UI, and it really unlocked for everyone what was possible.”
Matt Garman did a business school internship at Amazon in 2005 and joined Andy Jassy's new technology business there, which was AWS before launch.
“I did my business school internship at Amazon, and, uh, as we were looking around for projects, uh, I had talked actually at the time to Andy Jassy”
Matt Garman returned to Amazon full time as effectively the first product manager for AWS and says he has been there eighteen years.
“I came back full time, uh, effectively as the first product manager for AWS, and I'm now on year eighteen”
Around 2003 Jeff Bezos mandated that Amazon move from a monolithic software stack to services, which Garman says led to AWS.
“Jeff Bezos basically mandated across the company that in order to move from a big monolithic stack that wasn't gonna scale anymore for Amazon, we had to move everything to services.”
Garman says an early AWS virtual Linux server arrived in thirty seconds, when getting a physical server normally took about six months.
“it auto scaled and you got it in thirty seconds, which was pretty awesome at the time, where normally it was gonna be six months or something for you to go get a server”
Around 2008 or 2009 AWS staff visited financial firms in New York, including Goldman Sachs and JPMorgan, who were fact-finding about cloud computing.
“We went on a trip to New York, and actually, there's a bunch of financial services com companies, the Goldman Sachs' and JPMorgan's”
Garman says AWS deliberately targeted the hardest workloads, such as JPMorgan Chase and US intelligence agencies, so that regular companies' objections would shrink.
“if I can solve the, you know, JPMorgan Chase running in AWS, if I can run solve the US. Intelligence Agency running in AWS, like the the reasons for regular other companies are diminishingly small.”
About a year into working on AWS, Garman told a friend he thought it could become a billion-dollar business.
“Oh, how's that AWS thing going? And I was like, You know what? I think this thing could be a billion dollar business.”
Garman says about 85% of computing workloads still run on premises, by most estimates.
“I think 85% of workloads are still running on prem today by most estimations, somewhere in that range”
AWS won a secret cloud contract with US intelligence agencies, competing against incumbents such as HP, IBM and Oracle.
“we went after the the intelligence agencies for the US. government, and we won that contract and it was secret”
Garman says IBM sued over the AWS intelligence contract, which made the win public, and the agencies then publicly called AWS the most technically sophisticated option.
“so then it became public that we won this deal. And then the intelligence agencies went out to public and said, no, AWS is the most technically sophisticated”
Garman says AWS has invested in AI for about ten years and began building its own processors five or six years ago.
“we've been investing in AI broadly for the last ten years, and that's why we've started five or six years ago investing at the infrastructure layer and building our own processors”
Garman says AWS's AI strategy assumed there would be many models of different sizes and purposes, which customers would combine.
“We thought that there was gonna be a lot of models for a lot of different purposes, and there'd be big models and small models, and people would want to combine them in new and interesting ways.”
Garman says Microsoft has benefited from OpenAI's innovation, and that Microsoft recently listed OpenAI as one of its biggest competitors.
“Uh, Microsoft's done a good job of co-opting, uh, uh, OpenAI's innovation, although in their last, uh, in their last, I saw recently, they listed OpenAI as one of their biggest competitors now.”
Garman says Amazon's Titan embeddings model is the most popular embeddings model in Amazon Bedrock, used for building search indices.
“the Titan embeddings model is by far the most popular embeddings model that we have inside of Bedrock today for people that are building search indices”
AWS made Meta's Llama 3.1 available on Bedrock on the day it launched, and Garman describes a tight partnership with Meta.
“we just launched, uh, Llama 3.1, uh, on the day it launched, and we have a really tight partnership with Meta”
Garman says Adobe built its Firefly models on top of AWS.
“we see folks like Adobe building Firefly as a all built on top of AWS, purpose built for their own thing”
Garman says Amazon Aurora, AWS's managed database, is 100% compatible with Postgres, which he cites as a way to avoid customer lock-in.
“even where we have something like Aurora, which is our kind of managed database, it is 100% Postgres compatible.”
AWS partners with Scale AI to help customers label data for fine tuning or pre-training.
“we have a deep partnership with Scale AI to come in and help you label your data”