OpenNashPodcasts

Topic

Attention Is All You Need

5 facts from 5 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.

2017

Kapireddy says the 2017 paper 'Attention Is All You Need', which introduced transformer architectures, was when the field changed.

“as a matter of fact, 2017, I think, is when it all changed with attention is all you need and the transformer architectures.”

Chandra Kapireddy · Overcoming AI Hallucinations: Truist’s Chandra Kapireddy, 0:05:29

2017

The 2017 transformer paper "Attention Is All You Need" had been cited over 173,000 times by 2025.

“As of today, in 2025, this paper has been cited over 173,000 times in other academic papers, making it currently the seventh most cited paper of the 21st century.”

Acquired · Google Part III: The AI Company, 2:01:01

2017

Churchman credits the 2017 transformer paper 'Attention Is All You Need' with letting every word in a context window attend to every other word.

“it's really the transformer paper in 2017, attention is all you need, which basically said that actually you can have every word pay attention to every other word in a context window”

Chris Churchman · Building AI Systems for Capital Markets, 0:21:26

Undated

Taylor names the Transformer architecture, introduced in the Google paper 'Attention Is All You Need', as the most meaningful AI breakthrough of the past decade.

“Probably the most meaningful breakthrough over the past decade was the Transformers model, which was a paper out of Google called Attention Is All You Need”

Bret Taylor · Bret Taylor - The Agent Era - [Invest Like the Best, EP.386], 0:53:42

Episodes