Nvidia was already selling millions of GeForce GPUs a year when it decided to put CUDA on them, according to Jensen Huang.
“We were already selling millions and millions of GeForce GPUs a year.”
Person
Jensen Huang
3 episodes · 65 quoted claims
Jensen Huang says Nvidia's market value fell to about $1.5 billion after CUDA's cost hit the company.
“our market cap went down to like $1.5 billion, and so we were down”
Jensen Huang says CUDA raised the cost of GeForce GPUs by about 50% when Nvidia had roughly 35% gross margins.
“increased our cost by 50 and that con consumed, and we were a 35 gross margin company”
Jensen Huang says NVLink 72 let Nvidia place a model of 4 to 10 trillion parameters in a single computing domain, driven by the arrival of mixture-of-experts models.
“We could now take an entire 4 trillion, 10 trillion parameter model and put it in one computing domain, as if it's running on one GPU.”
Jensen Huang describes Nvidia's safety rule that an agent should have at most two of three capabilities: access to sensitive information, code execution and external communication.
“Agentic systems can s can access sensitive information, it can execute code, and it can communicate externally.”
Jensen Huang says Moore's law would have improved computing about 100 times in ten years, versus a million-fold gain Nvidia achieved.
“And so in the last ten years, Moore's law would have progressed computing about a hundred times in the last ten years.”
Jensen Huang says he persuaded several memory-maker CEOs about three years earlier to invest in HBM, then used mainly in supercomputers, as mainstream data-center memory.
“About three years ago, I was able to convince several of the CEOs that even though at the time HBM memory was used quite scarcely, you know, and and barely by supercomputers.”
Jensen Huang says about 200 suppliers contribute to Nvidia's Vera Rubin rack, which has roughly 1.3 million components.
“we have a couple of hundred suppliers that contribute the technology that goes into our 1.3 million component rack”
Jensen Huang says power grids are built for rare worst-case days and typically run around 60% of peak.
“Most of the time, we're nowhere near the worst case condition, and we're probably running around, call it 60 of peak.”
Jensen Huang says he started a management method called the speed of light about 30 years ago, which tests every design against physical limits.
“a method, uh, that I started, uh, 30 years ago, and it's called the speed of light”
Jensen Huang says Nvidia and its partners can produce about 200 Vera Rubin pods per week.
“probably kind of crank out about 200 of these pods a week, just to put in perspective”
Jensen Huang says roughly half of the world's AI researchers are Chinese and mostly based in China.
“50 of the world's AI researchers are Chinese, plus or minus, and they're mostly in China.”
Jensen Huang says Nvidia has done hundreds of billions of dollars of business with TSMC over three decades without a contract.
“Three decades, I don't know how many tens, hundreds of billions of dollars of business we've done through them, and we don't have a contract.”
Jensen Huang credits about 43,000 Nvidia employees and several million developers for CUDA's success.
“it wasn't three people that that made CUDA successful, it was 43,000 people that made CUDA successful”
Jensen Huang says Nvidia GPUs are already operating in space, doing imaging work on satellites.
“we're already there. Uh, NVIDIA GPUs are the 1st GPUs in space”
Jensen Huang predicts that customers will soon pay $1,000 per million tokens for the most capable AI output.
“the idea that somebody's willing to pay $1,000 per million tokens is just around the corner”
Jensen Huang says it is possible for Nvidia to reach $3 trillion in annual revenue in the near future.
“is it possible for NVIDIA to be a, you know, 3 trillion dollar revenues company in the near future? The answer is, of course, yes.”
Jensen Huang recalls a CEO telling him that a fabless semiconductor company could not exceed $1 billion in revenue, a prediction Nvidia later passed.
“it's theoretically impossible for a fabless semiconductor company to exceed $1 billion”
Jensen Huang names Doom as the most influential game, saying it turned the PC from an office tool into a gaming device.
“I would say Doom from a from a our the intersection of the cultural implication as well as the industry, turning a PC into a gaming device.”
Jensen Huang says radiology software is now driven by AI, yet the number of radiologists has grown despite predictions they would disappear.
“Every radiology platform and package today is driven by AI, and yet the number of radiologists grew, and so the question is why”
Jensen Huang estimates the number of people able to direct computers to write software rose from about 30 million to about 1 billion.
“I think we just went from 30 million to probably 1 billion.”
Jensen Huang says Moore's law in its earlier days increased performance tenfold every five years and a hundredfold every ten years.
“Moore's law in in the older days would increase performance by a factor of ten every five years, and a factor of a hundred every ten years.”
Jensen Huang says accelerated computing improves performance by roughly 1,000 to 100,000 times every ten years.
“in the case of accelerated computing, we're moving somewhere between a thousand to 100,000 times, uh, every ten years.”
Jensen Huang says Nvidia's AI GPUs weigh about 70 pounds and contain about 35,000 parts.
“GPUs that we build for AI, uh, weighs 70 pounds. Uh, they have 35, zero parts.”
Jensen Huang says eight of the roughly 35,000 parts in Nvidia's AI GPUs come from TSMC.
“Uh, out of those 35, zero parts, eight of them come from TSMC.”
Jensen Huang says Nvidia's AI GPUs consume 10,000 amps and are so heavy that robots are used to manufacture them.
“They, uh, consume 10,000 amps, uh, obviously, they're so heavy.”
Jensen Huang says Nvidia's computers are so advanced that a supercomputer is needed to test another supercomputer.
“The computers are so advanced that it takes a supercomputer to test another supercomputer”
Jensen Huang says Nvidia uses AI to design its chips, including finding optimal placements and connecting transistors into circuits.
“we're using AI to design our chips, to find optimal placements to, uh, help us, um, connect transistors”
Jensen Huang says Nvidia is starting to use AI to trace a failure back to the line of code that likely caused a bug.
“we're starting to use AI to help us determine from a failure where the bug likely, uh, started from”
Jensen Huang describes generative AI as predicting the next word, pixel or video frame, which can produce hallucinations.
“generative AI predicts the next word, or putting the next letter, next word, next sentence, next paragraph, predicts the next pixel”
Jensen Huang describes retrieval augmentation as connecting a vectorized database to a language model so retrieved information enriches the prompt.
“retrieval augmentation. What that means is a database that, whether it's structured or unstructured, tabular or, you know, otherwise, that has been vectorized”
Jensen Huang says most of the data companies hold is dormant and hard to use.
“Every company has a ton of data in their company. Most of that data is now dormant.”
Jensen Huang says reinforcement learning with human feedback turned out to be useful for fine-tuning large language models.
“it turns out that reinforcement learning, human feedback, could be quite useful in fine tuning a large language model.”
Jensen Huang says reinforcement learning was originally invented for animating stick figures and articulating fingers.
“Reinforcement learning was something that was really invented for, uh, articulating and animating stick figures and articulating fingers”
Jensen Huang says US restrictions on chip sales to China limit the performance level of Nvidia's GPUs rather than banning sales outright.
“The limitations are related to the performance levels of our GPUs. This is like restrictions of of, uh, CPUs and and computers”
Jensen Huang says Nvidia faces many local competitors in China while it complies with the export restrictions.
“the challenge there is that the local markets, especially in China, has a lot of competitors, and so we have to compete in the marketplace”
Jensen Huang says a fully autonomous car should combine cameras, radar and lidar and could add redundant steering and braking.
“you might insist on making sure that you have cameras as as well as radars, as well as lidars, you might even have um redundant steering control”
Jensen Huang says humans should validate, test and red-team each new batch of AI learning before it is released.
“before you, uh, release it out into the world, uh, you have humans validate it and test it, and, uh, red team it”
Jensen Huang says Nvidia has nearly gone out of business a handful of times in its history.
“actual experiences of nearly going out of business a handful of times.”
Jensen Huang says Nvidia cut roughly three or four layers of management by flattening its organization.
“we probably reduced, you know, three, three or four layers of management, uh, out of our company just by doing so.”