2015
Chris Olah says he spent years studying Inception v1, a vision model that was state-of-the-art in 2015.
“Inception v1, um, which is this one vision model, it was, um, state-of-the-art in 2015”
Topic
Chris Olah
5 facts from 2 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.
2023
In Anthropic's Towards Monosemanticity work, Chris Olah says dictionary learning on a one-layer model produced interpretable features.
“So, we took a 1 layer model, um, and it turns out if you go and you, you know, do dictionary learning on it, you find all these really nice interpretable features.”
2024
Chris Olah says Anthropic's Scaling Monosemanticity work applied sparse autoencoders to Claude 3 Sonnet, then a production model.
“and we did it on Claude 3 Sonnet, which at at that point was, uh, one of our production models.”
Undated
Yudkowsky cited the discovery of induction heads inside transformer networks as an example of interpretability progress.
“we have now, uh, understood induction heads in these systems by dint of much research and great sweat and triumph.”
Undated
Chris Olah says that while at OpenAI he found a dedicated Donald Trump neuron in the CLIP vision model.
“to give a concrete example, there we found that there was a Donald Trump neuron.”
Episodes