OpenNashPodcasts

Topic

Andrej Karpathy

Andrej Karpathy is an AI researcher whose podcast appearances, mainly on the Dwarkesh Podcast, cover artificial general intelligence, AI agents, self-driving cars and how large language models are built. Dwarkesh Patel says Karpathy led self-driving work at Tesla from 2017 to 2022.

Karpathy told Dwarkesh Patel he worked on OpenAI's Universe project and released NanoChat, a simple repository for building a ChatGPT-style chatbot. On The a16z Show, Matt Bornstein said Karpathy compared AI agents to autonomous vehicles and called them a roughly ten-year problem. Each fact below links to the moment it was said.

18 facts from 3 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.

1989

Karpathy reproduced Yann LeCun's 1989 convolutional network, which he describes as the first neural network he knows of trained with gradient descent.

“reproducing Jan LeCun's 1989 convolutional network, which was the first neural network I'm aware of trained via gradient descent”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:24:45

1989

Karpathy says that applying modern algorithms to LeCun's 1989 network roughly halved its error, and further gains needed more data and compute.

“if I time travel by algorithms to 33 years, I could adjust what Jan Lecun did in 1989 and I could basically half the learning, half the error”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:25:11

2013

Karpathy describes the Atari deep reinforcement learning shift around 2013 as an early attempt at building agents.

“the Atari sort of deep reinforcement learning shift in 2013 or so was part of that early effort of agents in my mind”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:04:20

2014

Karpathy says he had a near-perfect early Waymo demo drive around Palo Alto about 2014, before joining Tesla.

“I had a very early demo of Waymo. And it basically gave me a perfect drive in 2014 or something like that.”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 1:43:36

2025

Karpathy released NanoChat as a simple repository covering the whole pipeline for building a ChatGPT-style chatbot.

“it's trying to be the simplest, complete repository that covers the whole pipeline end-to-end of building a ChatGPT clone”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:27:07

2025

Karpathy used coding models while rewriting the NanoChat tokenizer in Rust, a language he was new to, and checked it against a Python version.

“when I was rewriting the tokenizer in Rust, I'm actually not as good at Rust because I'm fairly new to Rust”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:33:35

Undated

Matt Bornstein said Andrej Karpathy compared AI agents to autonomous vehicles and called them a roughly ten-year problem.

“he basically he related to autonomous vehicles and said, AI agents are a real problem, but it's like a ten-year problem.”

Guest · What Is an AI Agent?, 0:04:03

Undated

Andrej Karpathy worked at OpenAI on the Universe project, an agent that used a keyboard and mouse to operate web pages.

“my project at OpenAI, for example, was within the scope of the Universe project on an agent that was using keyboard and mouse to operate web pages”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:05:15

Undated

Karpathy became interested in deep learning by chance because he was near Geoffrey Hinton at the University of Toronto.

“this was just kind of like by chance of being right next to Jeff Hinton at the University of Toronto”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:03:36

Undated

Karpathy recalls that when OpenAI started, AGI meant a system that could do any economically valuable task at human performance or better.

“AGI was a system you can go to that can do any task that is economically valuable, any economically valuable”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 1:07:05

Undated

Karpathy predicts call centers will use AIs for about 80 percent of volume and delegate 20 percent to humans supervising teams of AIs.

“We're going to be swapping in AIs that do 80 % of the volume. They delegate 20 % of the volume to humans”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 1:10:36

Undated

Karpathy explains that automating slide-making is harder than coding because slides are spatial and visual and lack tools like diffs.

“slides are not text. Slides are little graphics and they're arranged spatially and there's visual components to it.”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 1:15:10

Undated

Karpathy, drawing on Nick Lane, notes that bacteria existed for about 2 billion years without producing animals, which suggests that complex life was hard to evolve.

“if bacteria have been around for 2 billion years and nothing happened”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 1:33:41

Undated

Karpathy says the InstructGPT paper showed that a pre-trained model can quickly become conversational through a small amount of fine-tuning.

“the paper that like blew my mind was InstructGPT”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:43:15

Undated

Karpathy says using an LLM as a reward judge fails because models learn to exploit it with nonsense outputs.

“anytime you use an LLM to assign a reward, those LLMs are giant things with billions of parameters and they're gameable”

Andrej Karpathy · Andrej Karpathy — AGI is still a decade away, 0:46:06

Undated

Andrej Karpathy, who worked for Elon Musk at Tesla, noted that Musk spends about 50% of his time talking directly to engineers.

“Andre Carpathi, who worked for Elon at Tesla, notes that Elon spends about 50 % of his time talking directly to engineers”

David Senra · #414 How SpaceX Works, 0:27:02

Episodes