Topic
Attention Is All You Need
5 facts from 5 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.
2017
Kapireddy says the 2017 paper 'Attention Is All You Need', which introduced transformer architectures, was when the field changed.
“as a matter of fact, 2017, I think, is when it all changed with attention is all you need and the transformer architectures.”
Chandra Kapireddy · Overcoming AI Hallucinations: Truist’s Chandra Kapireddy, 0:05:29
2017
The 2017 transformer paper "Attention Is All You Need" had been cited over 173,000 times by 2025.
“As of today, in 2025, this paper has been cited over 173,000 times in other academic papers, making it currently the seventh most cited paper of the 21st century.”
Acquired · Google Part III: The AI Company, 2:01:01
2017
Arvind Krishna dates the foundational transformer-based approach behind LLMs to a 2017 paper.
“this stuff is eight years old at this point. That's the first paper I think was 2017 when intention and this approach came to be.”
Arvind Krishna · IBM CEO Arvind Krishna says there is no AI bubble after all, 1:05:48
2017
Churchman credits the 2017 transformer paper 'Attention Is All You Need' with letting every word in a context window attend to every other word.
“it's really the transformer paper in 2017, attention is all you need, which basically said that actually you can have every word pay attention to every other word in a context window”
Chris Churchman · Building AI Systems for Capital Markets, 0:21:26
Undated
Taylor names the Transformer architecture, introduced in the Google paper 'Attention Is All You Need', as the most meaningful AI breakthrough of the past decade.
“Probably the most meaningful breakthrough over the past decade was the Transformers model, which was a paper out of Google called Attention Is All You Need”
Bret Taylor · Bret Taylor - The Agent Era - [Invest Like the Best, EP.386], 0:53:42