Nvidia’s CEO On What It Takes To Run An A.I.-Led Company Now

Southern New Hampshire University is an accredited institution with over 80 online graduate degree programs. That means you get the flexibility you need, the credibility you expect, and career focused courses you can count on. Visit snhu. Edu idea. Turn breakthrough ideas into real results at the virtual HBR Innovation Summit on November 12th, with top CEOs and master classes on AI and growth. HBR executive subscribers attend free with exclusive programming November 13th. Learn more at hbr. org slash innovation summit.

Welcome to the HBR IdeaCast from Harvard Business Review. I'm Alison Beard. There's no question that the biggest business story of 2023 is artificial intelligence. Not just all of us using generative AI tools like ChatGPT on our own, but what those advances mean for how companies can collect and use data to improve decision making, R D, and more. These developments are driving a new, exponentially faster era of computing, and our guest today is at the forefront of this warp speed transition.

Jensen Huang is the founder and CEO of NVIDIA, a world leader in AI driven computing that makes both software and hardware for a host of industries. An electrical engineer by training, he launched the company with two co founders at a Denny's restaurant. It now earns 26 billion in annual revenue and employs 26, zero people around the world. Its technology is quite literally propelling us into the future. Jensen spoke with HBR editor in chief Adi Ignatius at our Future of Business virtual conference last week about what's new in AI and computing and the kind of corporate culture and leadership that keeps NVIDIA agile and innovative.

Here's their conversation. Jensen, thank you for joining us. And what I know is an early morning for you, probably a busy day for you. We really appreciate you making the time.

So Nvidia is producing technology that is quite literally driving the future, and I'm talking about the chips and processors that are driving innovation in AI and generative AI. I I'd love your view as a as a sort of hardware guy. What's your what's your thought on AI, on gen AI? Do you think it will revolutionize how we work and play, or is it simply too soon to tell?

Well, at the core of generative AI is the ability for software to understand the meaning of data. It can understand the meaning of words, of course, letters and words and sentences and paragraphs, and find the relationships and patterns. And from a lot of examples, it figures out, learns the representation of of the data, and even understand its meaning. So it understands the structure and understand is how it's constructed, and and um, the meaning of the of the information.

This applies to English, it applies to pixels, 3D objects, proteins, chemicals. We've now used this deep learning method to, uh, learn the representation of a whole lot of different types of data, any data with structure. And so there are a lot of things in in in the physical world and in the world that we live in, uh, that has structure. Once we can understand the the meaning of the data, and uh, we can associate, uh, one modality to another modality, for example, uh, we can associate the pixels of a cat in an image with the word cat.

Once you can do this for one modality to another modality, then you can translate things. And so you could translate from English to Chinese, Chinese to French. Um, you could also, of course, translate from English to pixels, which is generative AI, or from pixels to language, which is captioning. Or you could go from text to text, uh, summarization, uh, so on and so forth. This basic model, uh, the model of, uh, that has learned representation of data, is incredibly powerful.

A And so now what the AI industry, what what the computer industry is doing, is mixing all of these type of generative methods and translation methods into all kinds of interesting applications. So I I think at at the core of this excitement i is the fundamental ability to to learn representation, their meaning, um, and to go from one modality to another, and that's just incredibly powerful.

I'm guessing that our viewers, by and large, are familiar with gen ai, maybe are playing around with it, probably relatively few are using it or applying it other than through experimentation. Uh, you know, I'd love do you have examples, maybe maybe using Nvidia technology of of applications that you're amazed by, that you're you're seeing out there already?

Well, we're using AI to design our chips, to find optimal placements to, uh, help us, um, connect transistors into SCA into, uh, circuits and help us optimize, uh, designs at a scale that that isn't really possible before. Uh, we're starting to use AI to help us determine from a failure where the bug likely, uh, started from w which line of code was responsible for the bug. And so we're using, uh, generative AI to help us, uh, from everything from designing the chip, optimizing the chip, to understanding where the failures are.

One of the most powerful things that's happening, of course, is that, uh, generative AI predicts the next word, or putting the next letter, next word, next sentence, next paragraph, predicts the next pixel, it predicts the next, um, frame of a video. Uh, these predictions could could, uh, hallucinate, and one of the the really effective ways is to one of them, of course, is is providing a lot of context, uh, a lot of prompts that people do to to, um, uh, set the background, precondition the model before you prompt it, before you ask it a question, or before you ask it to do something.

Uh, one of the things that's that's really useful is, uh, retrieval augmentation. What that means is a database that, whether it's structured or unstructured, tabular or, you know, otherwise, that has been vectorized, uh, vectorizing or embedding a database requires you to learn, uh, the relationship between all the data among each each other, just like you're learning, uh, the representation of of, uh, the struct, the data that I was talking about before. And by learning the representation and learning the, you know, vectorizing the database, you could understand its meaning.

And so, uh, what's really exciting now is that you can vectorize a database. You can connect it to a large language model, and, uh, that large language model essentially allows you to talk to your data. Every company has a ton of data in their company. Most of that data is now dormant. It's hard to use. Sometimes you have to, you have to do a lot of queries on it. But now, uh, you could have a a database that actually understands meaning, and so you could, you could do a semantic search into the database.

It brings back out the information that you meant to, uh, query, and you take that embedding and you augment that with, uh, the prompt that you're prompting the model with. And so now the context, the background information with the prompt, is much richer. So the thing that that is really quite generalized now is, um, the ease of being able to interact with applications because of large language models that understand your meaning. Um, and then secondarily, all of these databases that you can now vectorize, uh, which could then be used to augment the prompts and the queries.

And so the applications across your company is is really quite amazing. One of the one of the fun things, of course, if you're a company with a with a large customer service department and customer service agents are inputting, uh, their interactions with customers, whether it's problems or complaints or whatever it is, help that you're routing to customer through. All of that is somehow captured in a very large database. But wouldn't it be amazing if now that database was vectorized and you can just talk to the database?

Uh, you know what are the things that people are most upset about? Uh, what are some trends that we're starting to see? You know, if we want to, uh, improve the customer service of our company, what are the top two or three things that you can do? Uh, you could talk to your database like you're talking to a person, and that's really powerful.

So I want to shift here a little bit and talk about some of the challenges of of running a tech company, uh, in 2023. And, you know, for starters, how do you keep up with constantly shifting demand for for new products, for new services? And how do you? How do you build a management team that can imagine the future and keep pace with so much change?

well, um, you got to learn, you know, first of all, you're surrounded by people who are expert in the art, in the fundamental and the foundational parts of computer science. And they could be, you know, compiler experts, or they could be chip design experts, or they could be um, processor design experts, or interconnect, uh, experts, or uh, so on and so forth. And and and so you've got experts all over your company, and they're the best in the field of what they do.

Meanwhile, you want your company to be as alert as possible. And so the ability for information to be moving freely in your organization without being trapped in silos is is, uh, something that's really important. F uh, as the company gets larger and larger, especially when the when the industry's moving so fast, you want to be alert of, um, very weak signals that could be coming from somewhere. And so it could be a. It could be a research paper that's coming out of left field.

It could be related to, uh, uh, digital biology, it could be related to something in in robotics, and and somehow that robotics information. It could be generalized in such a way that it could be useful for for how we train a chatbot. You know, maybe, uh. Reinforcement learning was something that was really invented for, uh, articulating and animating stick figures and articulating fingers and and things like that, and playing a game. But it turns out that reinforcement learning, human feedback, could be quite useful in fine tuning a large language model.

And so so you wanna be able to see developments in adjacent fields and or even unrelated fields, and be able to somehow distill it back down to first principles, uh, and then extrapolate to, uh, useful applications in in, uh, other domains. That fundamental ability of starting from really good foundation, uh, expertise in the company, the ability to, uh, stay alert of weak signals, and um, and then being able to, uh, go back to first principles. You know, when, when you make some observation, uh, go back to first principles on on why is that important?

And what is the reason? And what are the governing dynamics associated with that, that phenomenon, and then be able to generalize them.

Um, and then you have the political issues that affect your business. I'd I'd love to hear you talk about how Nvidia is coping with the loss of, you know, I think it's billions of dollars in sales to China, partly as a result of US. restrictions on on sales of sophisticated chips to China.

Well, the restriction is a capability restriction, it's not a absolute restriction. The limitations are related to the performance levels of our GPUs. This is like restrictions of of, uh, CPUs and and computers, uh, that has gone back a very, very long time. The restrictions limit the level of capability or, um, the amount of computation that a a processor can do. So the first thing that we have to do is comply with the regulation, understand what, uh, the limits are, and to the best of our ability, offer products in the marketplace to serve customers that can still be competitive while, um, uh, complying with the restrictions.

And so the challenge there is that the local markets, especially in China, has a lot of competitors, and so we have to compete in the marketplace, you know, while, of course, uh, complying with the regulations that are that are inspired by by national security. It's not easy and it's a great challenge. And and, uh, the competitors are are, uh, moving quickly, and so, uh, it's it's like anything else you you've got to, you know, stay alert and and, um, uh, do the best you can.

So it sounds like you're saying you're not giving up on that market, you're finding ways, you know, within what's allowable, finding ways to, if I have you right, create new products for, uh, or new approaches to that market. Is that fair?

Only if the new products, uh, continues to be attractive to customers. Um, the first thing that we have to do is is understand what the regulation, uh, limits, and the second thing we do is go propose products to the marketplace that customers might still find attractive. Um, but there's no guarantees. I mean, competitors are coming up with all kinds of, all kinds of ways to attract the customer, and so, uh, we'll. We'll float the best possible products that we can that are, uh, fully compliant with, uh, regulations.

And and, uh, take it from there, see if customers are still attracted, uh, to the products that we we offer.

Okay, I want to go to an audience question now. This is from Steven. I don't know where Steven is, but the question is, Do you think that AI will always need human supervision?

Uh, human in the loop is pretty important. Uh, as you know, in robotics, uh, you could have, uh, different levels of of autonomy, most of the levels of autonomy, uh, until you. You have a great deal of safety, safety in all kinds of different ways. So, for example, in the case of a self driving car, if you would, if you want the self driving car to be able to take your family while they're sleeping in the back seat, uh, fully autonomously.

You. You want to be fairly assured that the sensor system, the computing system, the uh, vehicle control, all of it has diversity and redundancy. So, for example, you might insist on making sure that you have cameras as as well as radars, as well as lidars, you might even have um redundant steering control and uh redundant braking dynamics and so on and so forth. And these things give you assurance that, irrespective of the conditions, the critical components, uh, that affects the functionality, has diversity and redundancy.

Uh, diversity and redundancy is is an important part of autonomous intelligence in a lot of ways. Even in companies, we have, uh, diversity and redundancy, you know, notice we talk a lot about diversity in companies, uh, because it cr allows the company to be more resilient. We have, uh, a redundant ways of doing the same thing, or or trying to analyze the same problem from different perspectives. Maybe the the sales organization, uh, is augmented by the marketing organization, which is augmented by, you know, so on and so forth, and different different types of of ways to to ensure the vibrancy and the resilience of the company.

And, uh, most autonomous systems, uh, whether it's a as a robotic car or it's a a large organization, uh, wants to be designed in such a way that you have, uh, diversity and redundancy. And then above that, you want, uh, human in the loop. Sometimes the you get to a level where, uh, you don't believe you need human in the loop inside the car, uh, but maybe maybe you have human in the loop outside the car.

And you're monitoring, monitoring the effect of this. Um, this is no different than, uh, air traffic control, for example. Uh, you have, uh, you have two pilots, uh, you also have, uh, uh, redundant autopilot systems, but you also have air traffic control and pilots keep an eye out, uh, out for each other. And so there's a. There's a lot of different ways that autonomous systems are designed for maximum safety and reliability. The same line of thinking is, uh, will have to be done for, uh, generative AI.

We have to invent the technologies for, uh, for safety, and we have to apply engineering methods, uh, that allows us to to, uh, design systems that are safe, you know? Lastly, for, uh, quite a long time, uh, we will likely, uh, we will not likely, we'll, we will surely have human in the loop, uh, for example, while you're learning, learning the next batch from the next batch of data, um, before you, uh, release it out into the world, uh, you have humans validate it and test it, and, uh, red team it, uh, you know.

In. Instead of just have it be continuously learning and continuously updating all by itself. And so there are a lot of different methods that that, uh, we en we ensure, we ensure safety, and, uh, human in the loop is one of the best ways.

Um, I'm interested in hearing more about Nvidia's culture. Uh, one of your vice presidents was quoted as saying, Working at Nvidia feels existential every day. We feel like the company is going to go bankrupt tomorrow. So I wonder, is that deliberate? Do you try to instill a sense of paranoia to try to spur innovation?

Uh, you don't, you don't have to instill it. Uh, if you don't think you're in peril, uh, it's probably because you have your head in the sand. Uh, there are no, no companies that are that are assured survival. We have the benefit of having built the company from the ground up, and so I a. And having not exaggerated circumstances of nearly going out of business, actual experiences of nearly going out of business a handful of times.

We don't have to pretend that, uh, the company's always in peril, the company's always in peril, and we feel it. I don't need, and and the company doesn't need assurance that we will, we will do well, uh, in order to do our our best work, you know? And so I I think the company, uh, living somewhere between aspiration and desperation, and living somewhere within that, you know, within that spectrum, is a lot better than than either always optimistic or always pessimistic.

And so I I think a a, um, a guarded level of optimism and realistic understanding that that the company could be in peril at any given time. And we have to stay focused and alert and, uh, do our best work, um, and constantly earn the right to be in business. Those feelings and and that that sensibility, I think, is really good and healthy for companies to have.

Um, from what I've read, your your management style is somewhat unusual, that you, uh, you know, maintain a very flat organizational structure, that you have very few one on one meetings. Talk about your your, your theory of management.

Well, an architecture, an architecture, is designed, uh, for the company to do its work, to operate as efficiently as possible. And one of the things that that is really important for a company in technology that's moving as quickly as it is. Uh, we invented this technology called accelerated computing, and, uh, whereas Moore's law in the past, it, it's no longer nearly this fast. But Moore's law in in the older days would increase performance by a factor of ten every five years, and a factor of a hundred every ten years.

Uh, but in the case of accelerated computing, we're moving somewhere between a thousand to 100,000 times, uh, every ten years. And so when you're moving that fast, you want to make sure that that information is is flowing through the company as quickly as possible. You want the company to be as aligned as possible. A And also our company builds really complicated things. You know, people think our GPUs are like the like the add in cards that we design for gamers, and and they are we, you know, we, we build those with great pride and great joy.

Um, but the, uh, GPUs that we build for AI, uh, weighs 70 pounds. Uh, they have 35, zero parts. Uh, out of those 35, zero parts, eight of them come from TSMC. They, uh, consume 10,000 amps, uh, obviously, they're so heavy. You ha, you use robots to to manufacture them. The computers are so advanced that it takes a supercomputer to test another supercomputer, and so these GPUs that we're building are insanely complex, uh, and it requires a lot of software to run it.

And, uh, for a company like like us, that that designs the fundamental chips, creates the systems, creates the software, and creates all the networking and all the silicon photonics. And connect it into a large data center and help companies, uh, stand it up and operate it. Uh, our, our company, uh. Information has to really move quickly through the company and and really, really have no barriers and and no boundaries. And so we, uh, we architected a company that allows us to be able to work across this, this really complicated stack of computing technology.

On the one hand, uh, build incredibly complicated things on the other hand, and be very comfortable moving at light speed. If you want a company like that, then the last thing you want is for it to be, uh, information to to pass along hierarchically. And if you look at most organizations, uh, there are only two or three people at the top. And if that starts that way, then the number of layers of management, uh, layers of managers, uh, before you get to somebody who is actually, uh, dealing with ground truth, uh, could be seven, eight, nine, you know, ten layers.

And I didn't want that to happen, and so, uh, it. The best way to do that, uh, is to put the most at the highest level, to be as flat as possible up there. And so we probably reduced, you know, three, three or four layers of management, uh, out of our company just by doing so.

Yeah, so so we have time for one more quick question. I wanted to bring in one more audience question. This is from Leon, and the question is, you know, what can you say about quantum computing? to what extent, uh, are you investing in it? and you know, what can you say about about that?

Well, in order to build the world's next fast computer, uh, you need the world's fastest computer to help you design it, simulate it, uh, create new algorithms for it, and that the world's fastest computer, uh, is, uh, is Nvidia's, uh, accelerated computing systems. And so we don't build quantum computers, but we work with just about everybody in the quantum computing ecosystem, from people who are doing, uh, the basic quantum computing, uh, design to, uh, algorithm developers to architects who are who are working on classical quantum, uh, um, computers architectures.

And so we work with just about everybody out there. The quantum computer won't come along for, uh, in in a real industrial way, for probably another decade, and maybe two. And when it gets here, uh, it'll likely be an accelerator that's connected to the Nvidia accelerated computing systems. And, uh, some of the computation, uh, will be done on quantum computers. A lot of, a lot of the computation will be done, uh, on classical computers, doing pre processing, post processing, and so on, so forth.

And so so I'm looking forward to, uh, when quantum computers are are going to be, uh, uh, useful, effective, and and when it, when it comes classical computers, and particularly accelerated computing, and, uh, AI will be a very big part of it.

That's Jensen Huang, the CEO of NVIDIA, speaking with HBR's editor in chief, Adi Ignatius, at our Future of Business conference last week. And we have more episodes and more podcasts to help you manage your team, your organization, and your career. Find them at hbr.org slash podcasts, or search HBR on Apple Podcasts, Spotify, or wherever you listen. Thanks to our team, senior producer Mary Dew, associate producer Hannah Bates, audio product manager Ian Fox, and senior production specialist Rob Eckart.

And thanks for listening to the HBR IdeaCast. We'll be back with a new episode on Tuesday. I'm Alison Beard. If you listen to IdeaCast for conversations and strategy, you'll want to check out Founder's Mentality, the CEO sessions, in depth conversations with top executives about the judgment calls that define their careers. Follow wherever you get your podcasts.