OpenNashPodcasts

Person

Jeff Dean

This page collects what Jeff Dean said on the Latent Space: The AI Engineer Podcast, in an episode called "Owning the AI Pareto Frontier". He covered Google's early search systems, the history of neural networks at Google, the Gemini AI models, and the computer chips and techniques used to run them.

Dean says Google put its entire search index into memory in 2001. He also says Flash models from a new Gemini generation have matched or beaten the previous generation's Pro model, and that he believes general models will win out over specialized ones in most cases. Each fact below comes with his exact words and a link to the moment.

1 episodes · 26 quoted claims

Jeff Dean says Google's index update rate went from about once a month to updating any page in under a minute.

“So it used to be once a month. And then we went to a system that could update any particular page in like sub one minute.”

Latent Space: The AI Engineer Podcast · 0:28:53

Jeff Dean gave a talk at a web search and data mining conference in 2009 about the origins of Google search.

“I actually gave a talk in, I guess, a web search and data mining conference in 2009.”

Latent Space: The AI Engineer Podcast · 0:24:09

Google redesigned its search and retrieval system through four to six generations between about 1999 and 2004 or 2005, according to Jeff Dean.

“But we went through sort of four or five or six generations of redesigning of the search and retrieval system from about 1999 through 2004 or 2005.”

Latent Space: The AI Engineer Podcast · 0:24:09

Jeff Dean says distillation was motivated by image specialists: treating about 50 specialist models as an ensemble performed much better but was impractical to serve.

“you get much better performance if you then treat that whole set of maybe 50 models you've trained as a large ensemble.”

Latent Space: The AI Engineer Podcast · 0:04:20

Jeff Dean says that across multiple Gemini generations, the Flash model of a new generation has matched or beaten the previous generation's Pro model.

“for multiple Gemini generations now, we've been able to make the sort of Flash version of the next generation as good or even substantially better than the previous generation's Pro.”

Latent Space: The AI Engineer Podcast · 0:06:27

Jeff Dean says Google's ranking systems start with a huge number of pages and progressively filter them with more sophisticated algorithms.

“our ranking systems would be built to start with a giant number of web pages in our index.”

Latent Space: The AI Engineer Podcast · 0:20:47

Jeff Dean's 'Latency numbers every programmer should know' lists roughly eight to ten metrics such as the time for a cache miss.

“I mean, this has like sort of eight or ten different kinds of metrics that are like how long does a cache miss take?”

Latent Space: The AI Engineer Podcast · 0:30:06

Jeff Dean says moving data across the same chip can cost about a thousand picojoules, far more than a multiply.

“moving data from the sram on the other side of the chip not even off the off chip but on the other side of the same chip can be you know a thousand picajoules”

Latent Space: The AI Engineer Podcast · 0:32:37

Jeff Dean describes speculative decoding as predicting several tokens ahead, which raises the effective batch size and amortizes the cost of loading weights.

“You predict eight tokens out, and that enables you to sort of increase the effective batch size of what you're doing by a factor of eight”

Latent Space: The AI Engineer Podcast · 0:39:50

Jeff Dean says he named Gemini for the twin teams merging, and also for NASA's Gemini program preceding Apollo.

“there's also the NASA interpretation of the early Gemini project being an important thing on your way to the Apollo project”

Latent Space: The AI Engineer Podcast · 1:08:59

Jeff Dean says that before Gemini, Google Research and the Brain team had separate large language model efforts, alongside multimodal work and DeepMind's own models.

“at the time, we had efforts within Google Research and in the brain team in particular on large language models.”

Latent Space: The AI Engineer Podcast · 1:07:42

Jeff Dean says he met Andrew Ng in a Google micro kitchen when Ng had just started consulting, which led to scaling up neural nets at Google.

“I actually bumped into Andrew Ng, who's a Stanford faculty member, and I knew him from, I'd given talks at Stanford a couple years before”

Latent Space: The AI Engineer Podcast · 1:04:46

Jeff Dean wrote an undergraduate thesis on parallel neural network training in 1990.

“I did an undergrad thesis on neural network training, parallel neural network training, back in 1990 when I got exposed to neural nets.”

Latent Space: The AI Engineer Podcast · 0:59:19

Jeff Dean says an early Google paper on sparsely activated models was titled 'Outrageously Large Neural Networks'.

“we did an early paper on this where we really scaled up, you know, outrageously large neural networks.”

Latent Space: The AI Engineer Podcast · 1:01:29

Jeff Dean says Transformers gave a 10x to 100x improvement in compute cost to reach a given quality level compared with LSTMs.

“Transformers similarly gave you a 10x to 100x improvement in compute cost to a given quality level versus, say, LSTMs at the time.”

Latent Space: The AI Engineer Podcast · 1:02:01

Jeff Dean says Gemini turned a YouTube highlight video of 18 sports moments into a table of events, dates and descriptions.

“it was like a YouTube highlight video of 18 memorable sports moments across the last 20 years or something.”

Latent Space: The AI Engineer Podcast · 0:19:15

Jeff Dean says Google's math Olympiad result in the latest year came from a single unified model close to the production model rather than specialized systems.

“switching to a single unified model that is roughly the production model with a little bit more inference budget”

Latent Space: The AI Engineer Podcast · 0:48:06

Jeff Dean says models were struggling with simple GSM8K word problems about two years earlier, before reaching Olympiad-level math.

“two years ago we were struggling with gsm 8k problems right like you know fred has two rabbits he gets three more rabbits”

Latent Space: The AI Engineer Podcast · 0:45:29

Jeff Dean describes DeViSE as an early model that fused a language model with an image model so it could label unseen categories such as microscope.

“I did some work on an early model that fused together a language-based model where you have nice word-based representations and then an image model”

Latent Space: The AI Engineer Podcast · 0:57:58

Jeff Dean says Kalamong, an example low-resource language, is spoken by about 120 people and has no written text.

“which is truly low resource because it's only spoken by, I think, 120 people in the world and there's no written text.”

Latent Space: The AI Engineer Podcast · 0:56:10