Going Full Send on AI, and the (Positive) Impact of AI on Jobs, with Kevin Scott, CTO of Microsoft

Microsoft, the BMF productivity cloud and gaming company, has taken a massive bet on AI. Everyone's paying close attention to its partnership with OpenAI, and the technical community has been amazed by its release of some of the first truly useful and broadly deployed AI products, such as GitHub Copilot. Its full-on attack on web search with the new LLM-powered Bing chat is making its incumbent competitors dance. Today on No Priors, we're thrilled to speak with Kevin Scott, CTO of Microsoft and the driving force behind their AI strategy.

Kevin's leadership, both at Microsoft and prior at LinkedIn, Google and AdMob, as a technologist, is especially inspiring to me, given his distance traveled from his childhood home in rural Central Virginia. In 2020, he published a book, Reprogramming the American Dream, about making AI serve us all. Kevin, welcome to No Priors. Thanks so much for joining us.

Thanks for having me, guys.

Can you start by sharing with us some of your story? How does one go from a farming community in Virginia, where your parents didn't attend college, to CTO of Microsoft?

Uh, I don't know. I think it is a very unlikely journey. Uh, it's like, certainly not a thing that I, uh, I ever could have imagined. I I think part of it is I was just super fortunate to be wired like a nerd, uh, and growing up when I grew up. So, you know, when I was a teenager in the early 80s, uh, personal computing was, uh, was happening, and like, that was the thing that I happened to fixate on.

Um, and even though we were relatively poor, I managed to, you know, scrape together enough bucks to get myself a personal computer that I could have and just tinker with all the time. And it was, uh, like a, it was a RadioShack color computer too, like one of these things with chiclet keys that you, uh, you actually connected to a television. Like I had it hooked up to a thirteen-inch TV, and it had a cassette recorder that you, uh, stored and loaded your programs on.

And, you know, and and it was just the thing that I was obsessed with, and I I stayed obsessed with computers, uh, from then on. And it was just me trying to find a path at each step where I could work on the most interesting thing that someone was, uh, dumb enough to give me permission to go work on. A and and again, it's a it's a lot of luck. Like, there's no way you can, uh, plan a path from rural Central Virginia to CTO of Microsoft.

But, you know, I I think it does help to have a high level vision in your head for what it is that you want to do. Like, just knowing what you're aiming for always helps.

What was that vision for you? Besides, like, you know, obsessed with computers, wanted to work on them.

Yeah, I, I more or less had two of them. So the first vision I had when I was a teenager was I wanted to be a computer science professor. So I just looked at what computer scientists did and thought, This is the most amazing stuff I've ever seen. And I went to a science and technology high school. Yeah, and and the way that it worked where I lived is, uh, like a really rural area. And so the science and technology, uh, it was a governor's school, so it was centrally located in, uh, each high school in these four or five counties that surrounded the Governor school got to send two students each.

And so I was one of the two students that got selected from my high school to go to this thing. And my computer science professor there was this guy, uh, Dr. Tom Morgan, uh, and like, I just sort of felt like he opened up this entire new world to me. Like, it was just thrilling to learn all of this stuff. And I was like, Yeah, I wanna be like Dr. Morgan, uh. And and a lot of this stuff for me is about, you know, like, who those influential role models have been, uh, in your life.

And so as soon as I, like, met Dr. Morgan, I was like, Oh, I should just go be a computer science professor. And that was the path I was on until I was about 30 years old. Um, when I, you know, I was a compiler optimization and computer architecture programming languages person. And I got pretty disillusioned with what being a computer science professor actually was relative to what I wanted to do. Like, I just wanted to have a lot of impact.

And my perception at the time, when I was making these decisions, was that you can have a lot of impact as a computer science professor. Um, and and the impact was actually great, but it wasn't the impact that the system appreciated. Uh, so th- so, like, the impact that you can actually have is inspire students to go pursue these careers, and they will go on to do much greater things than, uh, than you've done yourself. And like that, that, to me, was the greatest impact.

But it was the least appreciated part of being a computer science professor. Uh, like, back in the, you know, 2000s, uh, when I was, you know, making these big decisions. So I decided to leave, and I didn't at the time know what next actually was gonna be. Like, it had been my mission for almost 15 years at that point, and like, I was a little bit lost. Uh, and I saw that a bunch of my academic buddies, uh, were all working at this startup called Google.

Um, and I didn't understand why they were working at Google. Like, Google was, you know, like some little box and you typed a keywords in and it gave you ten things. Like, how is that hard? Uh, but but, you know, there was Holzla, who was a compiler person, and Jeff Dean, who was a compiler person, and Alan Eustace, who was a compiler person. Like all of these people who, you know, who I went to conferences with and whose papers I read.

And uh, I was like, all right, well, maybe I should send my resume in. And like, I sent my resume in and, uh, got called to do a bunch of interviews. And they, uh, like it. It was the best interviewing experience I've ever had because they they took what must have been every compiler person in the company at the time and put them on my interview panel. And I was like, Oh my god, this is amazing.

Uh, like, I had the best day interviewing there, and I got this job offer and I went, Um, I I got this choice. They just started Google New York, uh, which was the first office outside of Mountain View. And they were like, you can come to Mountain View, or you can go be, you know, the 10th person in this, uh, New York office. And my wife and I wanted to live in New York more than we wanted to live in Mountain View, and so, uh, that's what we did.

And after I got there, this is where the new mission came in. I just so we, we were hiring these brilliant, brilliant people at the time, and and the way that we did hiring was kinda crazy. It's like, all right, well, if you're smart, just come work here. And like, we have no idea, like, what exactly it is, uh, you're gonna do. And you, like, came in and you sort of sorted yourself out. And we had these people who were so accomplished and so brilliant, and they would come in and choose to work on things that that just were gonna have no impact at all.

Like, they, they were intellectually very interesting, but they were just sort of silly in that they were never gonna connect with anything that moved the needle for the company. Which was exactly the problem I was trying to get away from. Uh, you know, in in being a, like, a research computer scientist, uh. And so I sorted myself out, like I found a, like, a pragmatic thing to go work on. Like, you know, we, like, I, I won't go into the details of what it is.

But, you know, like the whole team won a Google Founders Award, which was a big deal, uh, for like, solving this, like, very, sort of unsexy problem with a bunch of very fancy computer science. Um, which was one of the things I think Google, uh, Google did really well. And then I was like, Okay, well, I should just go help more people sort themselves out as well. And that's when I became a manager. And then it, from that point on, it was all about, like, Hey, I want to, I, I, I wanna help as many engineers as I possibly can.

Uh, like, make sure that their work lines up with something that's, you know, both interesting and meaningful.

I think that, uh, it's actually pretty under discussed the degree to which early Google had so many academics actually running important parts of the company. I think Ur's is a great example, and I think there's others. And so I haven't actually seen anything like that since, until maybe now, more recently, at OpenAI, there's more academics. Or, you know, you feel like the research community's popping back up again, but it's been maybe a decade or two since that's happened.

yeah, I mean, I I think that that that's actually a really, really great observation. So when when I go sit in open AI, it really reminds me of early Google days, and it's about the same size Google was when I joined. And so, like I I, I couldn't figure it out for a while, and I was like, Wow, this is like, really giving me, like, you know, early Google nostalgia. A And, you know, the the conclusion to draw from that is like, not that they're the same companies, or they're trying to solve the same problem.

It's just sort of the energy of the place and like who they've chosen to hire. And like,

Yeah, yeah, it's the first time I've seen, like, string theorists getting hired again into computer science roles, you know, since Google days. Yeah,

y- y- you and I, uh, like, probably both work with Jonathan Zunger, who, uh, works at Microsoft right now. And like, I, I remember, like, it's like, all right, Jonathan's working on this big distributed file system stuff, and like, what's his degree? Oh yeah, he's a like, sh- string theory guy.

So a big part of your mission for, you know, the last decades has been, um, helping string theorists and other engineers, um, figure out how to be how to be useful in their orgs. The other the other part seems to be, of course, like, um, actual technical direction, right? Deciding, like, what's worth investing in? And and you've worked on machine learning products for a really long time, like ads, auctions at Google, recommendations at LinkedIn, et cetera, et cetera.

Was there a moment when you decided, or you realized personally, that AI should be a key technical bet for Microsoft?

Yeah, yeah, I mean, I, I've been at Microsoft a little over 6 years now, so almost six and a half years. Uh, and like, pretty pretty quickly, it was obvious that AI was gonna be, like, very, very, very important to the future of the company. I think, you know, Microsoft already understood that before I got there. And then it was just how do you focus all of the energy on the company on the right thing? B- Because we had a lot of AI investment and a lot of AI energy, and it was sort of very diffuse, uh, when I got there.

Uh, so no lack of IQ and actually no lack of capital spending and everything else. But it was just, you know, kinda getting peanut buttered across, uh, a whole bunch of stuff. And so the the thing that really catalyzed what we were doing is, I mean, maybe this is a little bit too, uh, too technical. But like, you know, we before before I got there, the technical thing that had been happening with some of these AI systems that to me was very interesting is transfer learning was starting to work.

So, like, you were going from this mode of, you know, the flavor of statistical machine learning that I cut my teeth on, uh, like in my first projects at Google. Which was, you know, you have a particular domain of data and like, you have a particular machine learning, uh, model architecture that you are, uh, you know, you're training in. Like, a particular way that you're gonna go do the deployment and measurement and whatnot. And it's all, like, you know, siloed to a like, a, uh, like, a use case or a domain or an application.

To seeing AI systems that you could train on one set of data and use for things, uh, for multiple purposes. And you saw a little bit of that with some of the cool stuff that DeepMind was doing with reinforcement learning with, uh, you know, play transfer, uh, across some of the gaming applications that they were building. But, like, the really exciting thing was when it started working for language with Elmo, and then, uh, you know, BERT, and then RoBERTa and Turing, and, you know, like, a bunch of things that we were doing.

And that was the point where there were so many language, uh, based applications that you could imagine building on top of these things. If it continued to get better and better, and so we were just sort of looking for evidence that it was gonna continue to get better and better. And as soon as we found it, like, we just started, like, all in. That was everything from doing a partnership with OpenAI to, uh, you know, like, at one point, I seized the entire GPU budgets for the whole company.

And I was like, we will no longer peanut butter these resources around, like, we will focus them, uh, cause it's all capital intensive. It's like we will just allocate these things to things where we have really, really strong evidence-based conviction that, like, a particular path is gonna benefit from adding more capital scale.

I remember, uh, it must have been like five years back. Now we were at dinner, and, you know, now GPU capacity is the talk of the technical town, right? But you were like, I asked you what your, like, most pressing issue was, and you're like, how am I gonna spend on GPUs this year? and how I'm gonna distribute those GPUs?

yeah, yeah, and it and it was, and it has been, it has cer- certainly hasn't gotten any easier. But, yeah, I mean, so Elad, like, I think, you know, the question you were asking is like, how we decided to do the open AI partnership. And so, like, the the reason that we did the partnership was twofold. So one is with transfer learning actually working, you can imagine building a platform for all of this stuff. So that you you're building single things where you're amortizing the cost of the things across a whole bunch of different applications.

And because we have a hyperscale cloud, uh, like, one of the things that I was really, really, uh, interested in, and like, beyond interested, like, it felt, you know, just like an existential thing is how do you make sure that? The way that you're building your cloud all the way from, you know, your computing infrastructure, your networks, uh, your software frameworks and whatnot. How can it really serve a whole bunch of interests beyond your own? And so, like, we felt like, in addition to the high ambition things that we were doing inside of the company, that we needed, uh, like, high ambition partners.

And when we looked around, like, OpenAI was clearly the highest ambition partner that was in the field, you know, and I think still their ambition is just breathtaking. And what it is that they're trying to accomplish. And so that was one thing, and then the second thing was like, you know, they, they really had a very similar vision to the one that I had about, like, these things were evolving into platforms. And, uh, like, we were able to because we were so aligned on vision for the future.

Like, we could figure out how to do a partnership where, uh, like, even though, like, there's just a ton of difficult things and like, you know, I I think there's probably some conservation law of, you know, the stress from difficulty. So it's n- not like it ever goes away, but like it, it, it's stress in service of a common goal. And like, that's the thing that make good partnerships work.

I I think one of the stunning things about the partnership, in some sense, was the timing. Because if I remember correctly, Microsoft made its first investment, or its sig- its first significant investment in OpenAI right after GPT two launched, or right around GPT two, and this is before GPT three. And there was such a big step function between the two of them that I think it was less obvious in the GPT two days that this was gonna be as important as it was.

And so I'm a little bit curious, like, what were the signs that made you decide that this was a good partnership to have versus building it internally versus, uh, you know, usually as a larger company, there's the old like buy, build, partner kind of thinking. And so I'm just sort of curious, like, how how you all decided to to partner in this moment in time, where it's very non-obvious and you invested a large sum of money behind that.

yeah, there w- a and and I like, I don't wanna, uh, like, have revisionist history and like, paint a rosier picture than there actually was. So there there was a huge diversity of opinions inside of the company on the wisdom of doing, uh, doing this. And so Satya, uh, like, has this thing that he talks about, uh, like, no no regrets, uh, investing. So like, there are things where you do the investment and like, there are multiple ways to win and like you, uh, you even win a little bit when you lose, uh.

And so thi- this was one of those no regrets things in that, like, the very, very worst thing that could happen is we would go spend a bunch of capital on, uh, computing infrastructure, and we would learn, uh, like, what to do at very high scale for building these AI training environments. And you know, you, you'd have to believe something very strange about the world of AI that you wouldn't need, uh, advanced computing infrastructure. Um, and then there were just multiple ways where, you know, like, and and we had a bunch of evidence, uh, that, you know, we had gathered ourselves and that OpenAI had that gave us, w- you know, which unfortunately I can't talk about, uh, but like,

that gave us, you know, pretty reasonable confidence that scale up was actually working. I you, you've probably seen the, uh, you know, the famous, uh, open AI compute scale paper where they sort of plot on the log scale, like, how many, you know, petaflop days, or, you know, whatever, the, you know, unit of total compute they were using on that graph. That shows, uh, you know, from 2012, when we first figured out how to train models with GPUs through, you know, I think the the plot ends sometime in 2018.

Uh, yeah, that that we're, you know, basically consuming ten times compute, uh, more compute every year for, like, training state-of-the-art models. Uh, a and so, like, you know, y- y- I just had super, super high confidence that, uh, we were never gonna get to the point where we're like, all right, we got enough compute. Uh,

It was a very bold move. I I think it's very striking, all the amazing things Microsoft has done over the last few years in terms of just incredibly smart strategic moves. That at the time didn't seem obvious, and now are just in hindsight, you know, really brilliant. I guess a more recent move is your announced a collaboration with NVIDIA to build a supercomputer powered by Azure, um, infrastructure, combined with NVIDIA GPUs. Could you tell us a little bit more about your supercomputing efforts in general, and then maybe a little bit more about those collaborations, both the both NVIDIA and, um, OpenAI on the supercomputing side?

yeah, so we built our f- the first thing that we called an AI supercomputer. Uh, I think we started working on it in 2019, and we deployed it, uh, at the end of that year. And it was the computing environment that GPT-3 was trained on. And yeah, we we had been building a, uh, like a progressively more powerful set of these supercomputing environments, uh, like we'd built them in a way where, like, the biggest environments.

Just because, you know, they're they're very capital intensive things, uh, tend to get used for one purpose. But the designs of these systems, like we, we can build smaller stamps of them and they get used by lots of people. So, like, we have, you know, tons of people who are training, you know, very big models on, uh, Azure compute infrastructure, both folks inside the company and, uh, you know, partners who can come in. And yeah, it.

It was a thing that was, like, not possible to do before, where you could sort of say, like, hey, I would like a I would like a compute grid of this size with like this powerful network, uh, to do my thing on. And so, you know, NVIDIA's been, you know, our compute and network partner since they bought Mellanox, uh, you know, for years now. And and the thing that makes that work is generation over generation, like you're just getting better, uh, you know, uh, price performance from the systems.

Um, and and we work super closely with them, uh, like defining, you know, what the hardware requirements need to be, um, you know, in the coming generations of GPUs. Uh, cause like, we have a pretty clear sense of where models are going and like, what model architectures are evolving towards. Um, and so, yeah, I mean, it's just been a super good, super good partnership. Um, yeah, like, we're, we're deploying, uh, hopper now at scale, and you know, like a bunch of the features of hopper, like, you know, eight bit floating point, uh, you know, ar- arithmetic, and you know, a bunch of other things are like things that, uh, you know, like, we've been planning for for a while.

Yeah, I guess one one last question on sort of this, both supercomputers w- as well as platform side of things is. I'm a little bit curious how you view the world shifting in terms of closed source and open source models, and, you know, the mix that'll exist. Because obviously, from an Azure perspective, lots of people are running open source models on top of Azure right now.

Yeah, I mean, I it. It is an interesting thing that people are framing it as some kind of binary thing. Like, I think you're gonna have a lot of both. Um, like, we, we, we still don't see any reason to believe that you're going to want to not build bigger models. Uh, but like, we, we just know in our own deployments. Like, if you look at things like Bing Chat or Microsoft 365 Copilot or GitHub Copilot, you you end up using a portfolio of models to do the work.

And like, you use it for performance and cost optimization reasons, and you use it for, uh, you know, just sort of precision and quality reasons. Uh, sometimes, um, and so there's always this, you know, melange of things that you're, uh, that you're doing, and it's never either or. I'm I'm actually really excited by what's going on with the open source community. I I think, you know, my biggest question mark there is like, how you go deal with, uh, like, all of the REI and safety, uh, safety things.

But like, if you look at the technical innovation inside of the open source community, like, it's really, you know, thrilling. And like, you know, we, you know, like, we're doing some cool stuff right now. Like, I was just playing around yesterday with, uh, that 12 billion parameter, uh, Dolly 2.0 model from Databricks, uh, which, like, runs quite nicely on a single machine. And like, you know, I'm still enough of a dork to like, love playing around with things that run on single machines.

Like, it's, you know, really, really impressive work.

Yeah Yeah, it's super cool. How? Um, how do you think about that from the context of enabling AI for your business customers, um, outside of your core product? So is there a specific sort of B2B AI stack that's coming? Are there specific tools coming? To your point, there's safety, there's analytics, there's fine tuning that, you know, there's so much stuff that you could potentially provide. I'm just sort of curious how you think about that.

Yeah, I mean, I don't wanna turn this into some kind of weird marketing spiel, but, you know, we we have this point of view that we started with this assumption that AI is gonna be a platform. And the way that people are going to most usefully make or, or the way that people are gonna make most use of the platform is by building tools that assist people with jobs. So it's like less about these fully autonomous scenarios and more about, uh, assistive tech.

And so the first thing that we built was GitHub Copilot, which is a coding tool. Uh, it's thing where you can sort of say in natural language what you would like a piece of code to do, and it emits the code. And then you, uh, y- you as the developer, uh, like the same way that you would take a suggestion from a pair programmer. Like, you scrutinize it and code review it, and, you know, decide whether or not it makes sense for your application.

And, you know, and like, that w- that was the first version of GitHub Copilot. It does a bunch of other things, uh, now. And so the the thing that we have observed is this Copilot pattern is actually pretty, uh, you know, pretty generic. Um, and and we we built a bunch of Copilots, uh, since then, and the way that we built them. Like, there's a, there's, there's a Copilot stack that looks almost like one of these OSI, you know, networking, uh, diagrams.

And it starts with a bunch of user interface, uh, patterns that you have, uh, like, there're now an emerging plugin ecosystem for, uh, like, how you extend the, you know, the capabilities of a Copilot for things that you can't natively get out of the model. And then it is a whole stack of things, uh, you know, sort of an orchestration mechanism. Like LangChain is, uh, you know, one of the popular open source, uh, orchestrators, but like, there are a bunch of open source orchestrators.

Like, we have, uh, one that we've developed called Semantic Kernel that we've also open sourced. There is this whole fascinating world right now, uh, that didn't exist nine months ago, a around prompt construction and prompt engineering. And so, like, there's an entire art form and a a set of tools that that people have access to to design a meta prompt, which is sort of the standing instructions to the model to, like, get it to, uh, conform itself to the application context, uh, that it's in.

Uh, like, you have these new things, uh, you know, like new software development patterns, like, uh, retrieval augmented generation. A RAG is, uh, like, you know, we were doing this before it had a name, uh, on it, uh, and s, you know? So it's basically a way to, like, take the prompt that's flowing from the application and to inject context into the prompt that will help the model better respond. Um, and then there's a whole bunch of safety apparatus that you have, uh, so that looks a lot like filtering on both the way down.

Uh, as the prompt flows through the stack, all the way down to the model, as well as, you know, as it flows back up. Uh, so what things are you not gonna let a let the application or the user, uh, send all the way down to the prompt? Because it's gonna get a bad response back? Or like, you know, uh, what things are you gonna filter out at the last, uh, minute? Because, uh, like, it is a bad, uh, response that has gotten all the way through, um, you know.

And sometimes, like, you have multiple round trips through this cycle before you, like, bubble the thing all the way back up to the user to get them the response that they need. And so, like, you know, we, we have a point of view about what all, what this stack looks like. You know which Microsoft tools exist that will help people, uh, build these things, and like, what special things you have to go do in the context of an enterprise?

To, like, answer the actual direct question, uh, where you know safety and data privacy, and like understanding, you know where the flows of data are, and like which plugins can be enabled, and like which can't. Uh, like, a all of those things, uh, like, I I think, are getting built out right now. A and and like, the other thing too I'll say is like, we'll build some of this stuff. And like, the community is gonna build a tremendous amount of it.

Because, like, there's never been a platform or ecosystem where one company builds all of the useful things. Like, that's just nonsense, uh, like, it's just never happened, uh. And and to me, it's the sort of super exciting thing to just see all of the energy that's happening right now. Like, I, I just like, immediately before this call, I was doing a review with, uh, Microsoft Research, and it's just amazing to watch MSR, uh, which is so many researchers there have pivoted what they're doing research on to, like these AI adjacent or AI, uh, like, on point things, uh.

And it feels a little bit like what MSR was like when I was an intern there in 2001. Uh, where you know, y- you had s all of these super bright people who, like, had the tiniest little glimpse of what the future must look like that no one else had. Because it was the point where the PC was racing to ubiquity and like, they were just all orienting their research around, like, what that little glimpse was.

Uh, that, like, maybe they had the earliest, uh, peek at, and it just is, like, feels magical.

Yeah, it's massive realignment of the research community right now, sort of in real time. It's it's very exciting to watch.

I I mean, and and it's awe inspiring. I mean, it's just crazy. It's hard to keep up, like, super hard. Like we went from. I mean, th- this has been the biggest surprise for for me is like, I just didn't realize that GPT-4 and ChatGPT were gonna catalyze as much of this as they have. Um, like, we sorta kinda been expecting a bunch of this stuff. You know, ChatGPT was a ten month old model with a little bit of RLHF on top of it.

And, you know, like, by by, you know, admission, like, you know, not a beautiful user interface. It was just sort of a way to get something out there. Because, uh, you know, you needed some some practice with a handful of things before the big GPT-4 launch was, uh, was coming. And, like, no one really knew that it was gonna blow up this way.

And it's only five months old. That was only five months ago, which is shocking. I think everybody forgets how little time has passed.

Yeah, just shocking. And but but it is the open source community and like the the, you know, big tech community. I think at its best is like, you know, everybody is sort of realigning to, like, what I think is, you know, unlike some of the other, you know, faddish things that have happened over the past, uh, handful of years. Like, I don't think this is a fad, like this is, this is real.

Yeah, um, I, uh, launched my new fund about six months ago with this AI focus, and a few weeks later, ChatGPT comes out. And I'd say, even the people who are very prepared, like, hopefully, somewhat prepared to go, like, try to keep up or be part of that, um, massive shift, like, feel constantly upended. But it is, it is very. It's the most fun time to be in technology in decades.

yeah, it look, it's al- it's also, I will say, a disconcerting time, uh, to be in technology. Because so many things are changing at once, it's changing at a pace that, you know, you probably like e- even me, like, I, I'm, I'm. I think I might be in one of the better positions, uh, to, like, feel like I'm kinda in control of what's going on and like, I'm not in control at all, uh, like of the pace, uh.

And so it must really be disconcerting to folks, you know, trying to keep up with everything that's going on. And in some cases, like, it's forcing people to change their worldview about things like worldviews that they've held for a really long time. I think it's, uh, honestly, harder for some machine learning people than it is, uh, you know, for, like, a brand new entrepreneur who's, you know, just looking for an interesting thing to go do. Because it it is a very different way for a machine learning team to do its work.

A And it's like, been hard, you know, even for some of the people at Microsoft who have had plenty of time to think about the transition to, like, get adjusted to, like, this new way of doing things.

I wanna ask you one more question, um, that is sort of advice for people making the adjustment in a certain sense. And then, uh, you know, talk about your book, talk about the macro and such. Uh, Microsoft has a, um, unbelievably wide portfolio of products. And now you're on the other side of all the infrastructure questions, figuring out the, you know, organization of adoption of all these capabilities into that portfolio, right? Um, I talk to, you know, friends who run large companies, start a large companies all the time that are also figuring out how to do this.

How do you? how do you organize that effort? What what advice do you have for them?

I think you have to be, you have to remember that some things have changed and some things haven't changed at all. Um, a and so, like, one of the confusing things that I think there is for folks, uh, that that many people get wrong is like, models aren't products, uh, and infrastructure isn't a product. And so, you know, you, you, you need to very quickly understand what it is this new type of infrastructure and this new platform is capable of.

But, uh, that does not mean that you get to not do the hard work of understanding, like, what a good product is that uses it. It like, one one of the things I tell a lot of people is probably the place where the most interesting products are, are where you've made the phase change from im, uh, from impossible to hard. So, like, something that, like, literally, you couldn't do at all before, this technology exists, has become hard now.

Because, like, the the things that have gone from impossible to easy are probably not interesting. A And like, the the my my frivolous example of this is, uh, when smartphones, uh, you know, came on the market fifteen, sixteen, seventeen years ago now, like, it's, uh, yeah, 2007, I I guess, was iPhone launch, right? So, uh, sixteen years ago, almost, um, and then a year later, you had the App Store. So, like, the first apps were, like, things that had gone from, uh, impossible to easy.

Um, and like, they just, you know, n- we we barely remember them. Like, there were all these fart apps. There was like, you know, uh, like this app I had on my phone at one point that was called the woo button. You pressed it and it, like, did a woo like rick Flair. Uh, like, tho- those are those are not businesses, like, they're just, you know, sort of like these explorations that people are doing. Like, the the things that have made the smartphone platform are the hard things that, uh, like, went from impossible to hard.

They they also are s kind of the non-obvious things, like they weren't even the things that the builders of the platform imagined. Like, you know, we, we, we don't even think the the original applications on these platforms, like the things that launched, uh, when the platform first launched, like, those are not the interesting things anymore. Like, your smartphone is way more than just a SMS app and a web browser and a mail client. Uh, like, the thing that makes it interesting is TikTok and Instagram and WhatsApp and DoorDash.

And and like, these were all these hard things that people had to go build now that they were possible. And so, like, I think that's thing number one to, you know, hold in your head, either as an entrepreneur or as a business that's trying to adopt this stuff. It's not like how I go sprinkle some LLM fairy dust on my existing, you know, products and do some stupid incremental thing. And and like, you know, and I shouldn't even call it stupid, like, maybe the incremental things are fine, uh, but like, the really interesting things are are are non-obvious and very not incremental.

And so that that is the hard thing for us is you have an entire group of people who are smart and like, they can see all of the things that are possible. And so the har- the the challenge is to steer them towards, like, the hard, meaningful, uh, you know, sort of interesting, non-obvious, uh, things that are possible. Like, not the, you know, like things that are incremental. That, you know, just gonna burn up a bunch of GPU cycles and prevent you from, you know, and a bunch of I product IQ that will prevent you from doing the things that really matter.

If we we sort of zoom out to, like, non-technical audiences, you wrote a book in 2020, Ro- Reprogramming the American Dream. Can you describe who you want to read the book and and what you hope they'll take away from it?

I when I wrote the book, it was not for people like us. Well, the the so, the premise of the book is that I. I grew up in rural central Virginia. My, you know, dad was a construction worker, his dad was a construction worker, his dad was a construction worker. Um, you know, my maternal grandfather, like, ran an appliance repair business and had been a farmer earlier in his life. So the the thing, the thing that was true for everyone who was in my life, uh, like, you know, neighbors, members of the community, is like, you know, they're just smart, entrepreneurial, ingenious people.

Using the best tools that they could lay their hands on to go do things that mattered to them, that, like, created opportunity for them, and, you know, sort of solve problems for, you know, their their communities. Um, and I believe that, like, particularly this platform vision of AI, where it's sort of getting cheaper and it's getting more accessible all the time. You know, like things, you know, like the stuff that we were chatting about, um, uh, you know, a few minutes ago about what I did at Google.

Like, I, you know, came in with a graduate degree, I was mathematically sophisticated, uh, and yet to do the thing that I. The first project that I did, which was a, you know, like a machine learning, uh, classifier thing in 2003, uh, 2003, 2004. Like, that was, you know, stacks of, like, super technical, you know, research papers, and you know, uh, this elements of statistical machine learning, you know.

Like, you read it cover to cover, and then you go write code for six months. Like, high school student could do the whole damn project in four hours on a weekend now. Uh, like, it's just, you know, uh, like, w- and and what's happening? Like, that aperture of who can use the tools is just getting bigger and bigger and bigger over time. And so, like, the book was trying to get people to be inspired by this notion that, uh, like, don't be daunted and intimidated or scared by, uh, by AI.

Like, go embrace it and like, try to plug it into the things that you're doing. And like, maybe, you know, w- we, we've got a shot at having more equitable distribution of, you know, who's benefiting from the platform as it emerges.

If you were gonna add an update chapter for the last few years where so much has happened, what, what would you focus on?

Well, uh, it's. It's really interesting how much of it I think is still true. And like, I had this anxiety the whole time that I was writing the book that I was gonna. By the time I had the manuscript in and it hit the presses, that all of it was gonna be out of date. Like, the real problem I had is like, by the time it hit the presses, we had, uh, we had, uh, a global pandemic, and, uh, it literally w- it, it hit the presses, uh, the week that everything shut down.

So, like, you literally couldn't buy it, uh, like, Amazon wasn't delivering anything other than the essential packages, and every bookstore in the country was closed. So, I mean, it's a little bit surprising, uh, you know, to me, like, how many of the, you know, ideas that, you know, we have a platform, platforms getting more powerful, it's getting more accessible. Um, like, actually, the unit economics of it are getting better. Uh, you know, like, what you can do for, you know, a per token of inference, uh, is like getting higher.

Uh, you know. So, like, I know everybody's like in this frenzy around GPUs and like, which is this very expensive, uh, expensive thing. But like, all of this optimization work is happening where you're you're able to squeeze more out of the compute that you have, and the compute's getting cheaper. So, yeah, I mean, the update that I would, I would add is that, and and it may be an update that I do, like, it probably won't be this book, but like, I'm, I'm sort of contemplating, uh, like, writing, uh, something right now.

I, I do think that that the public dialogue around AI right now is missing so many of the opportunities that we have to go deploy the technology for good. Um, yeah, like a all of the articles, uh, that you, you know, you read in the newspapers are, you know, around the responsible AI stuff, which is important, and like the regulatory stuff, which is important. Uh, but, you know, we should have a few articles in there as well about, uh, Sal Khan's TED talk, which is just amazing, like, Unbelievably good.

And just for, you know, folks who may not have seen it, which they should go see, is like, you know, his problem is perfect for AI. So it's this two sigmas problem, this idea that students who have access to high quality, individualized instruction perform substantially better than those who haven't, like, controlled for everything else.

Just for our listeners sake, the two sigma problem was the study by a guy named Benjamin Bloom, which showed that your average tutored student performed above 98% of students in a control class, which is one teacher to thirty students, like a normal American classroom, uh, with reduced variance, which is amazing.

Yeah, and if you believe that that's true, then you can also believe that every student, uh, every learner in the world, uh, deserves to have access to that individualized, high quality instruction at no cost, which seems like a reasonable thing. And then when you think about how you go realize that in the world, like, the only way that you can realistically do it is with something like AI. And so there's so many problems that have that characteristic where we can all agree that, like, it is a universal good to do this.

And then if you think about how to do it, like, you must conclude that AI is, uh, like, part of the solution. Like that, that is the, you know, the reason I get up every morning and deal with people yelling at me about, like, give me my GPUs, uh, you know, for for the fifth year in a row. Um, I is because of of things exactly like that. And it doesn't mean that when you talk about that and you're hopeful and optimistic about those things, or even hopeful and optimistic about all of the things that you know, venture, venture back companies are gonna go do.

Or like, the way the businesses are gonna reinvent themselves that you are also say, you know, f- you know, giving the middle finger to, you know, the responsible AI concerns, or, you know, the things that people care about on the regulatory front. Like, you can care about both of those things at the same time. But, like, the the thing that I can tell you is like, there is no historical precedent where you get all of these beneficial things by starting from pessimism first, like, pes- pessimism doesn't, doesn't get you to optimistic outcomes.

Yeah, it seems like, to your point, a lot of the dialogue is really lacking from global, um, education equity, global health equity. Like all these things that AI as a platform should be able to produce, because it's it's cheaper, it's personalized, it can do things at the level of a of a human in many cases, in terms of being a great teacher or great, you know, physician's assistant, et cetera. And so it really feels like that message is lost.

And, you know, I think a lot of people don't mention enough how we're almost hopefully gonna enter this golden age if we let this technology actually bloom and be useful. I guess the the question that I always have on my mind relative to all this stuff is, given the capabilities that AI continues to accumulate, how do you think about twenty years from now? In terms of the best roles for people? And in particular, I think about it in the context of my kids.

I'm like, okay, normally, two years ago, I would've told my I would've told my my kids, go study computer science, it's the language of the future. What do you think is the right advice to give people, you know, in terms of what to study? And, uh, that that will be the things that will be most durable relative to to the changes.

Yeah, I I think so twenty years is a tough time horizon, you know? And I and I I think if any of us are honest with ourselves, like, if you rewind twenty years and you sort of imagine the predictions you would've made then, like, would you have gotten here? And like, nope, nobody would. But like, I I think, you know, there there are just some sort of obvious things, like my my daughter, for instance, like, has decided she wants to go be a surgeon.

And like, I think surgeon is like a pretty good job. Uh, like, we, we do not have a, you know, sort of robotics exponentials right now, uh, like we've got a cognitive, uh, exponential. A and so like, I think all of the like, the world is just sort of full of these jobs where, um, you know, really, you know, affecting change on a physical system, like doing something in the physical world, like all of those things, uh, like, we will need probably many, many more of them than we have right now.

Like, particularly in medicine, like nurses, surgeons, physical therapists, people who, you know, work in nursing homes. Like, we, we have a, you know, rapidly aging population, and so like, the burdens on the healthcare system are going to get much higher. And, you know, I do think that AI is gonna have some pretty substantial productivity impacts, uh, but like, you know, maybe it's just enough productivity impact to, like, you know, make room for all of the other net new things that we will have to have there, you know?

And so I I think, you know, we get this weird thing in the United States where, like, we apportion less dignity and respect to, like, jobs like the ones that my dad had, uh, than we should. Um, you know, and J- I lived in Germany for a little while, and Germany's a little bit different on this front. Like, you can, you know, you, you can go be a machinist in Germany, and, you know, like, that's a really great career and something that your parents are, you know, celebrate.

So, like, I think there, like all of these careers, like, you know, electricians and machinists, and, you know, solar installation technicians, and I, I mean, just so many things that we're gonna need. Like, especially because we're gonna have to rebuild our entire, uh, power generation and distribution system, like, in our children's lifetimes. Um, so, like, all of those jobs, I think, are super important. And then I, I would argue even that all of the creative stuff like that, that we do, uh, there's going to be probably more need for that in the future than less.

Even though the tools that we're using to do the creative work, whether it's coding or making podcasts or whatnot, are gonna help us be better at it. And the reason that I say that is humans are just extraordinarily good at wanting to put humans at the center of their stories. So so, like, we, we right now, we could be, you know, making, you know, Netflix shows, uh, you know, like, not Queens Gambit, but Machine's Gambit, uh, like, about, you know, a a fleet of computers play playing chess among themselves, uh, because they're all better than the than the very best human.

And like, nobody wants to watch that, like, y- the technology's probably good enough right now. Where you could have superhuman Formula One closed, uh, track drivers and Formula One cars that could, you know, do things that humans can't do. Nobody wants to watch that. Um, yeah, and like, you even go back before computers, like, forklifts are stronger than people, you could like, go have a strongman or a strong person competition. That was about like, which forklift could lift the most weight?

Like, nobody cares about that, like, we care about humans, like, what are we saying, what do we care about, like, what are we trying to express to everyone else? Like, and nothing about that's gonna change nothing.

I think that's why people watch the Real Housewives of Dubai. So, uh,

Yeah, and and so, like, and and I don't again, like, I d. I don't wanna paint too rosy a picture. Uh, every time you have a major technology platform or paradigm shift, like, there's disruption. But like, what we know from every one of these disruptions is you have, like, actually, a surprising degree of need for human occu- occupation. Uh, like all of the, you know, the industrial revolution predictions about, you know, four hour workweeks and we're all gonna live lives of leisure is bullcrap.

Um, yeah, like, just hasn't happened. And I think some people may say it hasn't happened because, you know, the system, you know, the system, uh, like, doesn't want it to happen. But like, I think a lot of it is because, uh, like, we actually like doing things,

yeah, and there's a lot to do, I guess, uh, on that note, what? What are some of the areas you're most excited about going forward in terms of the coming year of AI? Or, you know, big research areas or big product areas, or things that, you know, you're very optimistic about?

I think, sort of two things, just, um, I think this will be the maybe the great first year of foundation model deployments, where you're just gonna see, like, lots and lots of companies launch, lots of people trying a bunch of ideas. You're gonna see all of the big tech companies will have substantial things that they're, uh, that they're gonna be building. Um, you know, like, and I I got predictions about what other folks will do. But, you know, like, it will touch all of Microsoft's, uh, product portfolio.

Like, the way that you will interact with our software will be substantially different, uh, by the end of this calendar year than it was, uh, coming in, uh, and I think that'll be true for everyone. I think it it changes some of the, uh, nature of the competition that you've got between big tech companies. And like, I think it creates new opportunities for small, uh, tech companies to come and drive wedges into, you know, get footholds and do interesting things.

Um, you know, one of the things that Sam Altman and I have talked about a lot is like, I I suspect that this year, like the next 1 trillion dollar company, gets founded. Um, it won't be obvious which, uh, you know, which it is, but like, um, we're, we're overdue, like, long overdue. And and then I think, you know, what you're gonna see technically this year is, uh, I. I do think that you will have things like, um, the Red Pajama project, uh, is like this a and?

And there're gonna be a bunch of others like it, uh, will make really good progress on building more capable open source models. Um, yeah, and hopefully, hopefully the community will help build some of the safety, uh, safety solutions that you will need to accompany those things when you d deploy them. Uh, but like, technically, I think y- you're just gonna see amazing progress there. And then, like, it's just, yeah, the the frontier will keep expanding out.

Um, you know, we, we don't have o- openAI doesn't have GPT V and wide distribution, but like, it'll get to wide distribution at some point in the not too distant future. And so, like, you'll have these like mul- very powerful multimodal models. The same way that having GPT-4, uh, emitted, like, all of this exciting energy around new things that you could do with it. Like having a having a model that can, like, take visual inputs, uh, and like, reason over them, like, will also emit a whole bunch of new things, uh, that are gonna be very exciting.

So I don't know, like that, that's like, I I just think the the the theme of this year is gonna be like progress and activity, like a almost too much to track. Like, I, I'm gonna need a I'm gonna need a co-pilot just to, like, pay attention to all of this stuff and like, make sure I'm not missing important things. Cause I feel like I'm I'm at the and you all, as investors and as people who are watching this closely, must feel the same thing.

It's like, um, how do I make sure I don't miss, like, you know, the next important thing? How do I see it as soon as humanly possible?

Actually, just to make completely sure, if you are starting the next 1 trillion dollar company in our listener base this year, please call me and Rod and and Kevin too. Um, wrapping up now, is is there anything else you would wanna touch on, Kevin?

Um, I look, I think the dialogue that we're having right now around regulation is actually really quite important. So a as we're recording this, Sam Altman was testifying in front of the Senate Judiciary Committee on, uh, like, Tuesday of this week. Um, I think more of those conversations are a good thing. I think, as fast as things are moving, like, you really will need the technology community to come together and to agree on some sensible things that we can do before the regulation even is in place.

And I think that's all important and not a thing like. The the thing that none of us should be doing at this point is sort of like looking at the prospect of regulation and saying, Oh my god, this is a, you know, this is like a pain. Uh, like, I don't wanna deal with this. Like, the fact that there is a desire for it is like a very good signal that the things that we're working on actually matter.

Because, like, nobody's trying to regulate frivolous things. And like, the purpose of regulation is to make sure that you can build a solid, trusted foundation for things that you know, maybe be become ubiquitous in society. Like, if you think of this, like electricity, for instance, you wanna strike the right balance between allowing the technology to develop and make progress and flourish. But like, you also need to make sure that your electric power generators are built safely.

And you don't allow people to wander in and like, stick their finger on the electrode and disintegrate themselves. And like, you wanna make sure that, you know, the distribution of electricity is coordinated and that, you know, when it comes into your house, uh, like, it doesn't burn your house down. And when you, uh, you know, like you plug your appliances into the wall, that, you know, they function as as expected. And so, like, I I think that is, you know, a similar way, like it, there's not gonna be one size fits all.

Like, I think the most of the stuff that people ought to be thinking about is deployments, uh, like making sure that, like, as you deploy the technology, getting, you know, the requirements and the expectations right. There is the most important thing. Um, and then, you know, these big engines that we're building that are the like, the largest of the foundation models. Like, you know, making sure that, you know, you, you have a set of safeguards around those.

But like, also the way that we're building these things, they don't, they don't get distributed to the world, uh, like, by themselves, uh, like, there's a whole layer of things, uh, on top of them to, like, render them safe, uh, and then, yeah, a whole set of things per application, per deployment that we do to, like, make the deployment safe. And so, like, you know, I th- I think everybody, like all the startups, uh, like everyone in the open source community, everybody ought to be thinking about these things.

Like, how am I doing my part to make sure that we are creating as much space as possible for these optimistic, uh, uses? And like, we are deterring as many of the harmful ones as possible.

Yeah, I've been impressed by the degree to which the community has self acted from very early days in terms of AI safety and approaches to that. And so I know OpenAI has done stuff really early, Anthropic has, Google has, Microsoft has. You know, I feel like a lot of the main players have actually been, you know, remarkably thoughtful about this area and, you know, keen to make sure that it's done properly.

Yeah, I mean, it, it, the the thing that I will say is we fiercely compete with a whole bunch of these folks, uh. But like, one of the things that I don't do is like, look at any of those companies that you just named and like, worry that they're going to do something that, uh, like, is like, I take myself out of my role as CTO of Microsoft and like, just think about Kevin Citizen of the world, like I, Kevin Citizen of the world is not worried about, like, what my competitors are gonna do to, like, do something unsafe.

Like, I'm just not.

Thanks so much for being with us, Kevin. We really appreciate it.

Yeah, thanks for inviting me. This is awesome.

Yeah, thanks so much for the time, that was great.