Jensen Huang says he delivered the first Nvidia DGX deep learning system outside Nvidia to OpenAI.
“I delivered the very 1st one outside of our own company to OpenAI.”

In Good Company with Nicolai Tangen
12 Apr 2023 · 47m · Nicolai Tangen, Jensen Huang · 25 quoted facts
Jensen Huang argues that AI is the start of a new industrial revolution in which companies produce intelligence, not physical goods. Because people can program these systems in plain human language, he says computing has been democratized, with large productivity gains, from chip design to software engineering to climate and drug research. He also says governments must regulate the technology.
The tension runs between the speed of AI progress and society's readiness. Huang is enthusiastic about productivity, new jobs, and breakthroughs in biology and climate science. He also admits there are no real guardrails yet, that some jobs will be displaced, and that fake information is a real danger. The later personal section ties this together: his success rests on long-term vision, resilience and first-principles thinking.
Huang says deep learning models are too large for any PC or phone, so Nvidia rebuilt chips, networking and systems, buying Mellanox to link computers. Training still takes weeks, so shaving time off matters.
Huang argues ChatGPT is the first computer anyone can program just by describing a task, with no coding language needed. He claims this grows the pool of programmers from tens of millions to billions and narrows the technology gap.
Huang says Nvidia's chips are now too complex to design without AI, weather simulation runs about 10,000 times faster, and AI writes a large share of code on GitHub. He expects engineer productivity to improve tenfold.
He says AI can do harm as well as good and compares it to regulated industries like food, drugs and broadcasting, saying 'we regulate cereal'. He admits no real guardrails exist today and sees government as the only body able to set them.
Huang says Nvidia lacked basic skills at founding and faced several existential crises in its first 15 years. He credits the resilience of its people in overcoming that adversity, together with a correct long-term direction.
Huang says AI software is too big and data-hungry for ordinary computers, so every part, from chips to networking, had to be redesigned. The result was the DGX, and the first one outside Nvidia went to OpenAI.
Huang says technologies grow slowly and then shift abruptly once they become easy to use, as with Windows 95, the web browser and the iPhone. For AI, that moment is that anyone can now instruct a computer in everyday language.
He says it is less likely in the near term that AI itself displaces jobs than that someone using AI does. He expects some jobs to change, some to be created and some to be displaced.
He names drug discovery, where AI can learn the language of proteins and chemicals to improve the odds of finding drugs, and climate prediction, including regional impacts like extreme weather. He also mentions faster weather simulation using less energy.
He says he works every day, wakes at five, is asleep by about 9:30, and finds work, family time and reading relaxing. He says his attitude is 'how hard can it be', even though things turn out very hard.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Jensen Huang says he delivered the first Nvidia DGX deep learning system outside Nvidia to OpenAI.
“I delivered the very 1st one outside of our own company to OpenAI.”
Huang says AlexNet, designed for computer vision, was the first widely seen deep learning network and required Nvidia GPUs to produce.
“the 1st deep learning network that we, uh, that the industry, a lot of people saw, was AlexNet, and AlexNet was, uh, designed for computer vision.”
Nvidia acquired Mellanox to change how computers are connected and work together for AI workloads.
“we bought a company called Mellanox so that we can change the way computers are connected and the way can computers can work together.”
Huang says GPT-3 has 175 billion parameters, each of which could be a floating point number.
“Chatgpt three, uh, is 175 billion parameters, and each one of the parameters could be a floating point number.”
Huang says one breakthrough of GPT-4 is that it can learn from both language and images at the same time.
“One of the one of the ones is, uh, the fact that it can learn from both language and images at the same time.”
Huang says Windows 95 marked a sudden shift in the personal computer, about ten years after the PC was introduced.
“All of a sudden, Windows 95 came and everything about the personal computer changed, and yet the personal computer was introduced some ten years before that.”
Huang says there are over 5 million iPhone applications, far more than mainframe applications.
“the number of iPhone applications is over 5 million.”
Huang claims AI programmable by human language raised the number of potential programmers from tens of millions to several billion.
“the number of programmers in the world just increased from from tens of millions to several billion.”
Huang says AI-based weather simulation at Nvidia runs thousands of times faster than numerical methods, citing 10,000 times elsewhere in the discussion.
“we're already simulating weather now. Ten zero, 50,000 times faster than you using numerics”
Huang says Microsoft estimated that roughly 40 to 50 percent of software written on GitHub is produced by AI through Copilot.
“they've estimated that some 40, 50 of the software that's now written in GitHub is produced by AI.”
Huang says Nvidia expects to improve its engineers' productivity by a factor of ten using AI.
“we're gonna improve the productivity of our engineers by a factor of ten.”
Huang estimates the R&D budget for one of Nvidia's large chip generations at about $5 billion, more than building a rocket.
“The R D budget for it is probably something like $5 billion”
Huang says AI has learned the language of proteins, allowing synthesis of proteins with desired functions for uses such as drugs and breaking down plastics.
“Just as we've learned the language of humans, we've now learned the language of proteins”
Huang says he had downloaded but not yet read Microsoft's roughly 150-page paper on sparks of artificial general intelligence.
“Well, I'm anxious to, I'm anxious to see the paper, and, uh, it's 150 pages”
Asked whether guardrails for AI are in place, Huang says there are not really any.
“Uh, no, not really, not really.”
Huang says people using AI are more likely than AI itself to displace jobs in the near term.
“it's less likely that in the near term that AI is going to displace our jobs, it's more likely that someone uses AI is going to displace our jobs.”
Huang says over 100 million people had used ChatGPT, which he called the best software product ever made.
“you know, over 100 million people have used it, and there's no instruction manual.”
Huang says he wakes at around five in the morning and starts working immediately, working every day.
“I wake up at five, zero, and and uh, the moment I wake up, I start working”
Huang says he is usually asleep by 9:30 p.m.
“I'm asleep probably by 09:30. And I, I like my sleep”
Huang says he had recently read the book Chip War.
“Uh, let's see, I just, I just read chip wars.”
Nicolai Tangen states that Jensen Huang started Nvidia in 1993 at age 30.
“You started Nvidia in 93, you were 30 years old.”
Huang says he did not know how to be a CEO and nobody at Nvidia knew how to build a company when it began.
“I didn't know how to be a CEO, and n no, nobody in the company knew how to build a company.”
Huang says Nvidia faced existential crises maybe five, six or seven times in its first 15 years.
“The 1st 15 years, the adversity were incredible. Um, maybe five, six, seven times, it was existential.”
Tangen attributes to Huang a 2003 Stanford remark that his will to survive exceeds almost everybody else's will to kill him.
“In 2003 at Stanford, you said, Um, my will to survive exceeds almost everybody else's will to kill me.”
Huang says he held jobs as a dishwasher, busboy, paper deliverer and waiter and loved each of them.
“I loved it when I was a dishwasher, I loved it when I was a busboy, I loved it when I was delivering papers, I loved it when I was waiting tables.”
Huang explains how deep learning, starting with AlexNet, forced Nvidia to redesign computers and deliver the first DGX to OpenAI.
“I delivered the very 1st one outside of our own company to OpenAI.”
The scale of models like GPT-3 means no single computer can run them and training takes weeks, so performance gains matter greatly.
“Chatgpt three, uh, is 175 billion parameters”
Huang argues AI is a phase shift like the PC and internet, because anyone can now program in natural language.
“the number of programmers in the world just increased from from tens of millions to several billion.”
Examples from chip design, weather simulation and software engineering illustrate how AI boosts output and why intelligence becomes the product.
“the production of intelligence is going to be what what all companies do in the future.”
Huang lists protein engineering, drug discovery and regional climate prediction as top problems, plus assistants for people and companies.
“Just as we've learned the language of humans, we've now learned the language of proteins”
Huang calls for government guardrails, defends OpenAI's choices, and warns of AI-generated misinformation.
“we regulate cereal for God sakes, you know, we should regulate AI”
Huang describes his routine and says Nvidia's resilience, formed through early existential crises, is the core of its success.
“Maybe five, six, seven times, it was existential.”
Huang urges young people to ignore most advice, think independently and learn to love what they do.
“it's easier to fall in love with what you're doing.”
Yes, and a big thank you for, uh, taking the time today. I have to say I'm normally very excited when I record these podcasts, but, uh, wow, when I did the preparation for this incredible. You really are in the middle of all the important things which are happening in society just now. How does it feel
It is, uh, gratifying and uh, rewarding. And I'm happy for our company to be able to contribute to so many different areas of science and society. And so, uh, it is a thrilling time.
now? You, um, delivered, uh, well, you hand delivered, actually, the 1st supercomputer to OpenAI some years ago. Tell us about that.
We were just getting into, um, artificial intelligence ourselves, and we were working on autonomous vehicles. And so we were imagining what kind of computer, uh, has to be built for this new way of doing um software. You know, as as the audience probably know by now, that artificial intelligence is a computer that works with a works with software. Programmers to write software that is refined from data, and that software is impossible for humans to write. It's a gigantic body of code and and it requires a special type of computer.
And so, uh, when deep learning 1st came along, uh, we reasoned about how it would fundamentally change computer science. Because the early effectiveness of of, um, the 1st deep learning network that we, uh, that the industry, a lot of people saw, was AlexNet, and AlexNet was, uh, designed for computer vision. And its effectiveness was so incredible, it shattered records and shattered the effectiveness of computer scientists of several decades. And and it was a piece of software that required Nvidia's GPUs to produce, and we were so, so inspired by that.
Um, we reasoned about what kind of computer, what kind of software is this, where can it go? What kind of problems can it solve, and what is the implications to, um, everything about computer science? From chips to systems, to operating systems, to data centers, to networking, to the algorithms, all the way to applications. And we came to the conclusion that that, um, a new type of computer had to be created, and we created this new computer.
We call DGX. It's a deep learning um system, and uh, it's, it's a AI supercomputer, if you will. And I delivered the world's, uh, very 1st one. When I announced it that we were building it for ourselves, uh, I, I was. I thought that some people would be interested, but it turned out a lot of people were interested. And so I delivered the very 1st one outside of our own company to OpenAI.
What were the um big breakthroughs that you had to go through to get there?
Well, the the the way that the software is written, uh, it processes a giant amount of data to find patterns and relationships, patterns and relationships in the data, and the data is ones and zeros. Um, and somehow, using these architectures of deep learning models and the way that the deep learning um neural network is constructed. It was possible to scale into very large models, very large networks, and process a gigantic amount of data, looking for patterns and relationships.
And so the question is, is what kind of computer, uh, would be good at that? And so we reasoned through the computer architecture and came to the conclusion that that every aspect of the computer has to change from the way the the processor is designed. Um, our GPUs, our graphics processors, which simulates the world, uh, was a almost as a perfect starting point for understanding the world. You know, a chip, a a graphics chip that was designed to simulate the virtual world, as it turns out, is fairly good at understanding the world.
The mathematics is similar, but yet the size of the amount, the size of the data and the amount of computation necessary to do it, to find it, to go through all of that data is enormous. And so we broke it down chip by chip, and the CPU was no longer the the ideal processor, and so we evolved our GPU to become a deep learning, if you will, a artificial intelligence processor. The PC architecture is suited for personal computing, but it's not suited for giant AI systems.
To learn from data, the IO has to change, the networking has to change. Um, we bought a company called Mellanox so that we can change the way computers are connected and the way can computers can work together. Instead of instead of, maybe, uh, a handful of CPU cores working together, we have millions of GPU cores working together. To process the data, to find patterns, and and to learn what is called a representation, to learn the language of the subject that you're you're trying to learn.
You could be learning the language of music, you'd be learning the language of language, human language, or the language of the world, the physical world. To learn computer vision, we can learn proteins, we can learn chemicals, we can learn all kinds of things.
And of all these things, what were the biggest challenges?
Well, the biggest challenge is, um, it, and it remains now, is that the the problem space is gigantic. Whereas, um, an application for a phone can fit in a few gigabytes, uh, you know, or a PC, a few gigabytes. The the the software that we're talking about here, you know, Chatgpt is a hundred, and well, Chatgpt three, uh, is 175 billion parameters, and each one of the parameters could be a floating point number. And so us 175 billion parameters just for the neural network math.
Uh, not to mention the applications that sit around it. And so this is a giant application that doesn't fit in one PC, doesn't fit on any phone, doesn't fit on one PC. And it takes many computers working together just to run it, it takes a giant data center to learn from it, and it ch. It takes an entire, uh, you know, it takes a a large number of computers just to run it. And so this, this type of application has never existed before, and so every aspect was was changed.
We, we, uh, literally rearchitected everything of computer, uh, computers that we know of, from the ground all the way to the top. And so now now, these, uh, these AI computers are un are unlike anything that we've ever built before. And and because the performance, the performance needed is so great, uh, it takes, uh, it takes weeks and weeks to, um, process the data. So that we can learn from, learn from the data and learn its representation, learn the model, if you will, the the model of the the subjects that you're, the world model that you're, you're trying to learn from the data.
Um, it takes so much time to, uh, train these models that if you could even shave, you know, half of the time off, it's measured in weeks.
You mentioned GPT three, what's the the progress from three to four, just in terms of complexity?
Well, the complexity is is um hard to hard to estimate because, uh, open AI hasn't really described it, but there's there's a a great number of new breakthroughs. Uh, one of the one of the ones is, uh, the fact that it can learn from both language and images at the same time. Just as just as with humans, we learn more about anything if we could see, if we could, you know, read the words and see the images.
And and um, you know, one one good example, one good example is is, uh, if you've only, if you only s. If you've only seen horses, uh, and you've never seen a zebra, but I told you that a zebra is like a horse, um, but, uh, with black and white stripes. The fact that you have knowledge of both modalities, images and language, allows you to connect the two and learn something in your brain. Even though, you know, you imagine a zebra in your brain, even though maybe you've never seen one actually.
And so the ability for us to learn from multimodality is very important, and and um, GPT four has learned that capability.
You mentioned recently that we have now reached the iPhone moment for artificial intelligence. What do you mean by that?
Over the last 40 years or so that I've been in the computer industry, we transitioned, we phase shifted, you know, at 1st slowly, and all of a sudden, abruptly, just like just like water turning to ice or water turning to vapor. At 1st, the temperature, kind of, you know, increases linearly, and then all of a sudden, at some phase shift, the structure of the molecule changes all of a sudden, and instantly something happens. It happened when we went from mini computers and workstations, client servers to personal computers.
The very 1st four or five years, it grew linearly. All of a sudden, Windows 95 came and everything about the personal computer changed, and yet the personal computer was introduced some ten years before that. And the same thing happened with the internet. The 1st five, six, eight years, uh. Scientists and researchers were already using the internet. And then all of a sudden, one day Mosaic came along, Yahoo came along, and bam, there's a phase shift in the way that that, um, the internet was perceived and used.
In every single one of these transitions, then cloud, then the iPhone, the mobile cloud. In each one of these transitions, the computer itself is programmed differently, it's easier to program. Let me give you one example. The number of mainframe applications in the world is not that many, but the number of iPhone applications is over 5 million. And so the fact that there are so many applications must suggest that it's easier to create amazing applications, and it's absolutely true.
The applications we have on iPhones and and the mobile devices is is surely, uh, beyond expectation, beyond imagination, just 30 years ago. And yet, you know, people are creating these applications, obviously, um, very, very quickly, uh, the. And so the programming models changed, the application capability has changed, and the reach of the computing has changed. And so let's apply that to, uh, GPT, let's apply that to artificial intelligence as we know it. The way that you program this is just with human language.
This is the only computer, the 1st computer in the history of humanity that everyone can program the computer to do something. And there's no programming language. You you don't have to use basic, you don't have to fortran, pascal, c, c, java, you don't have to use any of those programming lang, python, you don't have to learn anything.
Which is really good, cause I tried to learn Python last year and it didn't quite work So out. To the extent that I now have to sing at the so Uh, at the UH summer party.
the number of programmers in the world just increased from from tens of millions to several billion. And so we've, we've narrowed, we've narrowed, um, we've w i, if you will. We've democratized computers, we've democratized computer programming, and we've closed the gap between the have and the have not. Access to technology, the technology divide has now been, has now been closed.
And what are the implications of this, uh, democratization?
When you democratize technology and you put it in the hands of of almost everybody, you empower everyone. Uh, look at the number of applications that are coming out now that are based on Chatgpt, uh, where people are are connecting it to applications, making the applications better. Um, people are are using it to, uh, write stories, create music, um, write programs, uh. So instead of writing a program, you tell Chatgpt to help you write a program. You tell it the problem you're trying to solve, and it writes you a Python program, or you write it, writes you a, you know, a SQL query, and and um, uh, it might even, well, it even creates a website for you.
And so if you want to go into business and you don't know how to create a website, you can now just tell Chatgpt to help you create a web website. You describe what you want, and it connects it all up for you, and it's operating. And so here's a computer that can help you write programs, solve problems, empower you. And so that's one of the greatest things of of democratization of technology. We've now put this amazing tool in the hands of everyone.
What do you think it would do to society?
Well, the 1st thing that's gonna happen is our productivity will go up. You know, any profession that relies on knowledge and the access of knowledge, uh, the application of knowledge, uh, will now be boosted. And and so if you if you have domain knowledge, and most most companies in the world has very, very deep domain knowledge, is the reason why they're, they're a company. Their domain knowledge can now be, um, uh, put in the hands of their employees and applied and accessed, and applied, uh, in a much more, uh, rapid way.
Uh, of course, there's a lot of mundane information, um, tasks that are now, if you will, commoditized, and it's automated, you know, we're, uh, one of the things that that's really incredible about artificial intelligence. And the reason why this is definitely going to be the next industrial revolution is instead of producing, um, uh uh, steam to electricity, instead of mass producing physical things, we are now going to be producing the most valuable asset, the most valuable commodity that we know as society.
Information, knowledge, and so the the. The production of intelligence is going to be what what all companies do in the future. Uh, we have Nvidia has AI factories, we put data into it and improved software, and the software is artificial intelligence, improved intelligence. Software comes out every single day. I go to sleep and it keeps producing it, and we keep refining more data, we keep improving. The the the software, the software helps us design chips, it helps us operate robots like self driving cars, it helps us, uh, do computer vision for quality inspection.
It helps us develop software that helps us, you know, design and manufacture chips better. And so every company will be able to do that for their own particular domain. So I think the the next industrial revolution is going to be about the the production of intelligence. And and for for industries that relies on intelligence, our productivity will be insanely boosted. And of course, there will be some jobs that will be changed, there'll some jobs that will be created.
Right now, we're creating a whole lot of jobs for artificial intelligence, data scientists and and people who who, uh, understand this field. Um, of course, some jobs will be, uh, displaced. And so we have to make sure that that, as a society, that we understand what this technology is and take advantage of it, um, as fast as we can, so that we understand it. And, um, uh, apply it to social benefits.
How, how much do you think? um, I think, how much do you think productivity could increase? On the back of this?
Well, there's, there's a few ways we could measure it, so let me give you a couple of examples. Um, uh, one of the hardest things that we do I in in our company is designing chips. The chips that we build are the largest chips, the most complex chips the world builds today. No singular entity builds such large, enormous semiconductor chips, and these chips are simply impossible to build anymore without artificial intelligence. And the reason for that is because the the number of transistors and the way that we can connect up those transistors, the combinations is just is so insanely great.
And because so many people work on it. Um, the optimization of the mathematics, the optimization of how to place, you know, it's. It's kind of like, imagine, um, uh, a New York City, but it's a thousand times bigger than New York City. And you're trying to figure out, um, uh, how to organize, uh, uh, New York City from the ground up, uh, such that it is the most optimal placement of every single building. And and then you have to understand which, you know where the traffic goes from building to building.
You have to understand which building is associated with other building, and you know what building, cer, or what buildings are necessary to support certain buildings and certain infrastructure. Uh, where do you put the parks, where do you put the restaurants? Uh, it's insanely complicated, as you can imagine, you know, it's. The number of combinations is off the charts, and we can't solve these problems anymore without artificial intelligence. So, on the one hand, it lets us do things that we can't otherwise do.
On the other hand, let me give you another example. We use artificial intelligence right now to try to better understand climate change. And um, in order to understand climate change, you have to simulate the weather, uh, a lot more f, you know, a lot more quickly. Because you're trying to further extrapolate the implications of of climate, uh, out in the distance. Not not just tomorrow, um, but ideally next month, next year, next ten years, uh, next 30 years.
So in order to do that, we have to do weather simulation a lot, lot faster. And so we've created an artificial intelligence that helps us simulate the multiphysics of of weather, and we're already simulating weather now. Ten zero, 50,000 times faster than you using numerics, and so that's another way of thinking about productivity. When you can do something 10,000 times faster, you're doing it, you know, you're doing it 10,000 times faster. One last example, um, uh, the single greatest expense in our company is software engineers.
And now with, uh, Microsoft's copilot, uh, you could, you could, um, uh. And they and they've estimated that some 40, 50 of the software that's now written in GitHub is produced by AI. It's a little bit like, um, text completion, it's a little bit like grammar correction on in our in our word editing documents. Um, except this is for program completion, and so the AI can suggest, based on what you've already written and what you intend to write, um, it can write the program for you.
And so, so, if you could imagine the single most expensive um population at at Nvidia is now amplified by a factor of two, that's incredible. And so our our, our estimates of, you know, we're gonna improve the productivity of our engineers by a factor of ten.
When you talked about um complexity, being a thousand times the complexity of New York City, that you put in on, what kind of area, what's the how big is one of the chips?
Rich ships are probably, um, you know it, it comparing it to a stamp is, it's. It's a couple of inches per side, and um, uh, what is what is a couple of inches per side? It's kind of like a. It's smaller than a coff, you know, coffee cup, a coaster, probably, you know, probably a a two thirds the area of a coaster, if you will, just to get put in perspective. The R D budget for it is probably something like $5 billion, and then, you know, it's.
It costs more to build one of these generations than, for example, to build a rocket, and, you know, it's the R D budget's very high.
When you put together everything you said about productivity gains, um, when you look at the whole society, how do you think this could drive productivity gains in the whole society? If you were to put a number on it?
Um, I, I don't know how to do that, um, but one thing for sure, the the countries that don't have act, don't have, uh, the the richness of of computer scientists. And and haven't benefited as greatly from the enormous capabilities of computers. Um, this should be a reckoning moment for them, this should be just an extraordinary opportunity for them. Uh, the up and coming economies, the up and coming industries. Um, uh, I think India, Southeast Asia, um, uh, Africa, th this.
These are. These are regions and economies that I I think has a real benefit from artificial intelligence. Uh, enhancing the capabilities of the entire industry and their and their economy, and then f, you know, driving productivity to the limits. And so I I think, um, for the rest of the world, for, uh, the developed countries, the ability to reduce cost is incredible, not to mention, um, accelerating everything that we do.
So, when you look at the most important problems that AI will solve over the next five to ten years, w What are they?
One of the most important ones is, uh, digital biology, drug discovery. Just as we've learned the language of humans, we've now learned the language of proteins, and we've learned how to understand proteins, and we've learned how to. Uh, from the desired function of a protein, as a machine, the biological machine, the way that it's connected, the amino acids, the chain of amino acids, and the way that it comes together. The 3D shape of that protein determines its mechanical functionality.
If you will, it's kind of like, you know, the difference between the shape of a of a motorcycle and the shape of a car, and the shape of a, you know, unicycle. The fact that they're different shapes, their functionality is different, a plier and hammer, the func, the functionality is different because of their shape, and so proteins have different shapes and different functions. We can now, from the desired function of a protein, synthesize other proteins, other potential proteins that have properties that are maybe better for temperature, or b better ability.
Uh, so that we could, it goes into our bloodstream better, or maybe, um, we can use it to, uh, synthesize energy from light, uh, maybe we can, uh uh, break down plastics, uh, maybe it could break down, um, oil leaks in the ocean, or, um, you know, whatever the the interesting problem is, we can now use, uh, protein machinery and protein engineering, uh, to go help solve that problem. I think that that's tremendous, in incredible potentials.
Uh, we can understand the language of chemicals, uh, so that, and now that you can understand chemicals and proteins, you can understand their interactions. And, uh, do a better job discovering drugs, you know, drug discovery still costs enormous amounts of money and takes a very long period of time, and our success rate is very low. And so now we can improve the odds of that, and so drug discovery is one. The other one is climate change, both in understanding the impact of human factors, uh, to climate change, um, predicting, uh, climate change, uh, and the climate effects in regional, uh, uh, uh, regional climate, um, impact, uh, whether whether it's, um, uh, extreme weather in the in the uh, Gulf
of Mexico, or, um, the number of fires that that, uh. Because we just have so many dry days in, uh, Northern California. Climate change has a different impact in different parts of the world, and, you know, people are interested in in average climate change, but not really. You know, people want to know what climate change, uh, has an impact on them, and their local economy, and their agriculture, and their water supply, and the the quality of life, and the impact of extreme weather, and so on, and so forth.
And so so we want, we want to have a better understanding of the future of climate. And by doing that, the side effect is that the the algorithms, the mathematics that we, uh, and the the the way that we do computation of physics, uh, can have a tremendous impact in just about all other physical fields. Uh, in reducing the amount of computation necessary to do it. And so, on the one hand, we use, um, we have the ability to predict climate, on the other hand, uh, we use less energy to predict climate.
And so that artificial intelligence makes that possible.
We had Bill Gates on the podcast recently and he talked about, um, you know, a personal agent, in a way, like a digital personal assistant. How do you see that?
We'll have personal digital assistants, we'll have group digital assistants. Maybe maybe we have a study group and we have a digital assistant to help us. We'll have a company digital assistants of all kinds. Somebody who is a digital assistant for HR, somebody who's a digital assistant for IT, somebody a digital assistant for, you know, for programming. Somebody who just wants to. You want to understand, you want to, you want to model our company's business, or model the the potential effectiveness of a new product or a new service.
Or, you know, and so there'll be digital assistants of all kinds.
How close are we now to artificial general intelligence? I think Microsoft said that they have seen some sparks of it. What, um, what's going to happen there?
Well, I'm anxious to, I'm anxious to see the paper, and, uh, it's 150 pages, and so I'm looking for it. I'm, I'm getting, I'm, I've downloaded it. And and one of these weekends, I'm I'm gonna go through it. Um, intelligence is about perception, reasoning and planning. Um, we have done an extraordinary job with perception, and, uh, but we still have a long ways to go. Uh, in order to, in order to, uh, really have a, you know?
The the goal of perception is to create a a model of the world around you, a a model of the world around you. In a static form, but also in its dynamic form. You know, what are, uh, if I did this, what would happen to that? You know, you know, we do this all the time, and today the conversation is, you're, you're imagining. If you ask this question, it might lead to this answer, which leads to another question, which leads to another answer.
And we do this in human interaction, we do this in company and industrial interaction, we do this all the time. And we have a we have a mental model, uh, some of it is supported by simulation, a mental model of how the world behaves. Uh, we have to go create that model of the world, and there's so many different worlds. There's the world that is the the the human scale world, um, but there's the world that is molecular scale, there's the world that's atomic scale, and then there's, of course, the there's the world that's galactic scale.
Each one of these worlds are, uh, described, uh, sometimes by different physics, right? At some, at some level, you have to go to quantum physics, and all of these understanding these different worlds, uh, all matter. And so so the 1st thing is just understanding the world. The 2nd part is how do you reason through problems, uh, in a way that, uh, achieves the objectives, but are supportive. And and um, within the within the realms of your core values, uh, your principles, um, uh, keeping other people safe, uh, that's explainable under, you know, interpretable, and um, uh, that's in a transparent way.
Uh, you know, how do you reason? how do you reason through all of this? uh, with those things in mind? And then, and then, how do you come up with a plan that that, um, is uh, efficient and cost effective? And you know, all of the things that we do, we do as as humans and and as industries, and so those three, those steps, if you will. Um, uh, AI is making tremendous progress, uh, along that entire arc.
uh, uh, robotics is making great progress, and and that's, uh, understanding the world and being able to to plan your your motion. Um, we're making great progress in autonomous vehicles. uh, we're making great progress. Uh, uh, chatgpt, uh. Obviously, the fact that it can, it can, um, uh, take a problem that you described and be able to break it down into a computer program, uh, obviously suggests it has the ability to reason through several steps. Uh, it might not be able to reason, uh, through, uh, as many conceptual steps as we can, um, but it's.
It's surely demonstrating demonstrating the the ability to, um, uh, do some early level reasoning, and so so the progress is is quite fine, quite fast.
Changing, um, tack a bit and zooming in on the ethical side of this. Now, we just had a a letter, um, recently, from a thousand really well respected people who said, it's time to slow down, think and reassess. What do you think about it?
AI is a is a, um, very powerful technology, and it's a very powerful technology because it can perform tasks and do things that that are of great value. And and uh, technology that has that has this level of capability, obviously can also be, um, applied to do harm. Um, and so regulation is necessary. uh, we regulate cereal for God sakes, you know, we should regulate AI, and um, uh, uh. This technology, of course, is moving very quickly.
And so it's sensible that, uh, regulators really have to get engaged and, um, uh, understand the technology, uh, uh, to the best of their ability. Uh, put. But put some guardrails in, put some regulation in, so that, so that, um, uh, so that the the technology can advance in a way that's helpful to society and not hurtful.
Have we got any guardrails in place now?
Uh, no, not really, not really. Um,
Who should put them, who should put them in place?
well, the same people that that, uh, uh, governments, uh, there's really no choice but for for governments to step, step in and um, regulate this. Uh, we regulate food, we regulate drugs, we regulate transportation, we re regulate, um, uh, industries, the the creation of of of uh, chemicals, the creation of, um, you know, uh, materials that could be toxic. Uh, we regulate just about everything, we regulate electricity, we regulate communications, uh, we regulate, uh, um, you know, the broadcast of television.
There are certain things that you can't broadcast. Um, there should be certain things you can't generate, you know, generative AI generates information. There's, uh, there's certain things you can't generate. On the one hand, it's hard to, uh, regulate, um, people expressing themselves, uh, because of open speech, however, uh, it is possible to regulate what information you produce. Um, and so the regulation of of the production of things, and now we're producing information using computers. Uh, you can regulate that, and there's, uh, there are many things that you can regulate.
When you look at this arms race we are having now in in AI, and, you know, uh, powered by your technology, are you afraid?
Whatever power of technology there is, uh, we should try to democratize it as best we can. Um, if it were to land in the the hands of one company, it's it's, uh, obviously less good, um, than um, being available to everybody. Uh, it's less likely that in the near term that AI is going to displace our jobs, it's more likely that someone uses AI is going to displace our jobs. And and the same thing could be taken to all, all kinds of extremes.
And so so when a new technology that comes along, that that provi produces so much productivity gains, you know, whether it's the steam engine or heavy machinery, it gives it. Gave us superpower, a tractor gave us superpower, forklift gave us superpower, human strength. And now we have this capability to give us, um, amplify our intelligence, and, and um, uh, help us solve problems a lot more quickly. Uh, we, we've got to find a way to use that technology as soon as we can, and but make sure that that technology is is, uh, available to everybody who would like to use it and regulate it.
Um, you know, regulate it, um, as soon as we can.
You said it should be, um, democratized, but um, and and of course, open AI was meant to be open, right now it's turning into a commercial product. Uh, you were more guarded when you talked about the specifications, you know, there is a lot less disclosure about, uh, the underpinnings of it. Uh, how do you read that?
Well, that's, that's a company choice of theirs, and they have the right to do that. And in the meantime, there's a there's a great deal of, uh, AI research that's still done in the open. The number of large language models that are available in the open is is, uh, quite abundant. And so it's. It's not about access to the technology that is keeping anybody back, it's it's simply the willpower to go. And and um, the insight that this, the technology is at a very close to useful, useful state.
That insight was terrific. The insight that that between GPT two and GPT three is a very useful product, the difference between a marginally useful product to an incredible useful product, that was a great insightThose are, those are the same insights that that, um, you know, led to the iPhone, or that led to the PC, that led to the internet. Um, before each one of those, that led to Google search, uh, the, the, the insight that led to each one of those innovations is real.
Really about timing, you know, technology was invented early on, and it was even cultivating and brewing in certain circles for quite some time. And and yet the innovators are the ones that realized the timing is now. And to jump on it. And, um, industrialized it and turned it into a really great product. Uh, ChatgPt is unquestionably the single best software product the world's ever made. And and by that definition, uh, let me, let me, let me defend that.
You know, a great software product is something that does amazing things and surprisingly amazing things, and a a great software product is also easy to use. This is the easiest product to use on the planet. Anybody's, anybody could use it, you know, over 100 million people have used it, and there's no instruction manual. You don't read anything, you just start typing into it. And um, uh, if it's not sure what you, what you meant, it asks you questions back and tells you that it's not sure.
You just keep, you know, talking to it with your whatever language you use, and um, uh, and and it produces amazing things, surprising things. It is the single most useful, best application the world's ever written.
How is this going to change geopolitics? How is it going to change the relationship between the US. and China?
Well, hard to say, um, hard to say. I think the there, there's a, there's a genuine, there's a genuine um harm that can come from, uh, uh, fake news that's being generated, fake information that's being generated, and um, that could cause real harm. The same harm that's currently happening in in social media and and uh, fake news, and um, uh, some of it is generated by human, uh, um, well, most of it is generated by human today.
And so so, um, uh, you could imagine that that this AI has a better ability to detect human generated fake news, um, but this technology also has the ability to generate fake fake news. And so both, both of those, both of those, um, uh, possibilities exist in in in abundance.
Changing tack here, let's talk about the young Jensen. Who were you when you were young?
Uh, let's see, I was, I was, uh,
I'm not saying you're, I'm not saying you're not young. Uh, uh, still, but you know, you're, you're really young. So
uh. Statistically, I'm, I'm on the other side of that hill. Uh, let's see. I. I was focused, I was curious. um, I was, uh, I was a a perfectionist. Uh, I wanted to do everything well, I worked hard. I mean, those, I would say that those things characterize me.
do you think it's understood how much hard work that goes into great achievements?
Oh yeah, I mean, the amount of hard work, the there's hard work, and then there's insanely hard work. In order to be, in order to be, And where are in order to.
you? where are you? and where are you on that scale?
I, I'm on on the insanely hard work, you know, I'm, I'm the
What does that mean? What does a day look like?
I. I work every day, there's not a day that goes by, I don't work, and if I'm not working, I'm thinking about working.
And when do you kick off in the morning?
Well, you know, I, I wake up at five, zero, and and uh, the moment I wake up, I start working, and so I, I work, I work. Every single day, there's not a day that goes by, I don't work
When do you go to bed?
as early as possible. I, I, um, I'm in bed probably by, uh, I'm asleep probably by 09:30. And I, I like my sleep, and and uh, sleep is really important to me.
What do you do to relax? What do you relax?
Uh, I relax all the time. I I, um, uh, I enjoy relaxing at at work. Just, yes, working is relaxing for me, solving problems is relaxing for me. uh, achieving something is relaxing for me. um, and the most relaxing, just hanging out with my family. uh, doing anything is relaxing for me. Yeah, I, I relax in a whole bunch of ways. uh, um, reading about things that that, uh, uh, that's important to me is relaxing to me.
Um, you know, so just hanging out with my family is relaxing. I relax in a lot of different ways. I'm, I'm pretty relaxed.
What do you read?
Uh, let's see, I just, I just read chip wars. Um, uh. I skim through a lot of AI papers. I don't understand all of them, but I try to understand all of them. Um, I, you know, I try to. I try to read, uh, everything that that, um, that that's of curiosity to me.
You started Nvidia in 93, you were 30 years old. If you were to boil down the essence of the success, what type of characteristics is it that the company has that makes it so successful?
Your Your perspective about the future has to be, you know, on a on a fairly long arc. Um, pretty important, and and it has to be, it has to be somewhat directionally right. And we were, we were, um, uh, I would say, absolutely directionally right now. The question is, along that direction, there are a lot of different paths. And and um, uh, those some of those paths, uh, would've been easier if you had better s.
If we had better skills, you know, I didn't know how to be a CEO, and n no, nobody in the company knew how to build a company. And, uh, we didn't even know what a PC looked like at the time. Uh, I'd never even used one before. And so there were a lot of things about about the company that, uh, the skills that that we didn't have, uh, we that we had to. We had to develop those skills, uh, how to raise money, how to organize the company, uh, how to, how to recruit people.
Um, you know, tho, those were all all skills that I, that that we had to develop along the way. I I think that those skills are probably, uh, skills are learnable. I think the, you know, the the attitude of an entrepreneur and attitude of somebody who does something new, uh, is how hard can it be, you know? And my attitude has always been, you know how, how hard could it be to learn learn? PC? How hard could it be to build a company?
how hard could it be to hire people? how hard could it be to to create an organization? It turns out all of those things were super hard, all of those things were super. it turned out it was super, super hard. Um, but I think you want to go into it with the attitude. How hard can it be, you know? And so when we, when we got into the journey of artificial intelligence, we got into the journey of scientific computing, we got into the journey of autonomous vehicles.
You know, we, we started with the attitude. How hard can it be? And and so if it's a solvable problem, how hard can it be? And we reasoned about everything from 1st principles. And and um, uh, and if anybody could do it, I'm sure we could, we could, and you know, we'll just learn as fast as we can. And and so I would say, I would say, I would say, we didn't have any of those skills.
But but if I had to boil down, um, uh, what led the company to be successful? uh, our vision was right. Um, but. But the character of the company is probably the most important thing. A char, the character of a company, um, I is what makes it ultimately successful. And you know, how resilient is it? um, how does it deal with adversity? uh, how does it deal with learning? uh, you know, when it's presented with new assumptions, if the conditions change, how agile is the company?
You know, when the world changed around us continuously,
Those, um, those values, um, the learning, the agility, the ability to change. How do you install those values into the company?
you talk about it, you teach it, you, you live it. Um, and Nvidia's really fortunate, uh, as a as a long term successful company. Um, we, we have, we have excellent chance. And the reason for that is because we suffered so greatly in the beginning. For the 1st 15 years of our company. Um, it was one adversity after another, and then every, you know, and after that, uh, there were adversity after another, um, but the company was able to deal with it.
The 1st 15 years, the adversity were incredible. Um, maybe five, six, seven times, it was existential.
What's the key to coping with adversity?
I I think the the um I. In the beginning of a company, forming those forming the corporate character, the the corporate culture, it's people, it's the people's, uh, resilience, it's the character of the people there. Uh, u. Unfortunately, there's this, you know, a. A company is made of people, and, um, it's not made of the document that describes the culture. It's not made of the document that, you know, the the. The inscription of the core values, um, on the building.
Uh, that's not what makes the company's culture, it's the people and how the company overcame existential crises, how the company overcame, um, uh, the incredible adversity that it was presented at the time. Um, both in the agility of the people, the cleverness and the creativity of the people, the ingenuity of the people, and then also the the the will of the people. The countless times that our company has been presented with challenges and the willpower, the utter incredible ability to suffer, you know, willpower to to be able to to do something even in just extraordinary pain.
That is corporate.
In 2003 at Stanford, you said, Um, my will to survive exceeds almost everybody else's will to kill me. Where does that come from?
Well, you know, I I think everybody's, everybody's upbringing is is, um, unique to them. Uh, I, I've just always had that. Uh, you know, it's. It's pretty hard to discourage me, and if I believe in something, um, I'm just gonna. If I believe in it, I'm just gonna keep on doing it until it's done, until until we're great at it. Uh, it's hard to deter me, it's hard to distract me, it's hard to, you know, discourage me.
And uh, uh, I in my mind, it's always how, how hard can this be? And it turns out every time I say, how hard can this be? It turns out it's incredibly hard. And and uh, and I'm surrounded by amazing people helping me, uh, and it remains incredibly hard.
And last question, what is your advice to young people?
Well, there there are a lot of things to to learn. Um, I, I would, just I would advise, uh, be be a, be a learner, um, but um, uh, probably the best advice that that I can imagine is is, um, think from 1st principles. Uh, don't worry about anybody else's advice. You know, I, I, I've been given a lot of advice over the years. Uh, some of it have been very good, um, most of it has been irrelevant.
And the reason for that is because it was either the advice was either an opinion, um, it was perspective of the time, it was based on on wrong assumptions. Um, and and my advice would be, you know, think for yourself, think from 1st principles. And and and a lot of people say, you know, find something you love. Uh, I, I don't know about that, I guess, I guess I've fallen, fallen in love, um, in many things that I do.
I loved it when I was a dishwasher, I loved it when I was a busboy, I loved it when I was delivering papers, I loved it when I was waiting tables. Um, I've loved every single job that I've ever had, and I loved every single day at Nvidia that I've ever had. And um, uh, I just learned to love what I'm doing. And so so, I, I guess it's probably harder to to find something that, uh, that you love.
But it's easier to fall in love with what you're doing. And once you fall in love with what you're doing, because you just desperately want to do a good job at it. Um, it's easier to do a hard, it's easier to, uh, to do it well and do it hard.
Well, I think that's, um, that's a beautiful place to end. Um, I have to say this has been one of the most intriguing and interesting conversations I ever had in my whole life. And I think, you know, when we look back at this time, 20 years from now, 30 years from now, you could potentially have been the person who's changed the world the most.
I enjoyed our conversation. Nicolai Tangen,
Thank you so much. Keep Thanks for your time, it up,
thanks for the opportunity.
keep it up, thanks.
All right, take care.
Hm.