2020s
The transformer technology enables AI to generate new text, language, video and audio content.
“This is where the new technology called the transformer enables us to create new content, text, language, video, audio”
Topic
Transformer
6 facts from 4 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.
2026
Yudkowsky described putting up a prediction market on whether by 2026 researchers will have understood anything inside a large transformer network that was not known in 2006.
“putting up a prediction market on whether, um, in 2026, we will have understood anything that goes on inside a giant transformer net that was not known to us in 2006.”
Undated
Yudkowsky cited the discovery of induction heads inside transformer networks as an example of interpretability progress.
“we have now, uh, understood induction heads in these systems by dint of much research and great sweat and triumph.”
Undated
Aidan Gomez said it took about a year after the Transformer paper for the research community to take notice of it.
“So I think it took about a year for the community to take notice.”
Undated
Aidan Gomez said the Transformer's creators did not expect the technology to advance as far as it has in seven years.
“There's no way seven years ago when we created the transformer, any of us thought we'd be here in seven years.”
Undated
Rana El Kaliouby says the Transformers paper was written by Google researchers and kickstarted the rise of generative AI.
“The Transformers paper Brett is talking about was written by researchers at Google, and it kickstarted the rise of generative AI.”
Episodes
- 30 Mar 2023#368 – Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization3h 22m
- 19 Mar 2026Strategy Summit 2026: Why AI Means Radical Change30m
- 10 Jun 2024AI will make money sooner than you think, says Cohere CEO Aidan Gomez1h 15m
- 20 Nov 2024How agentic AI will save customer service, with Bret Taylor38m