Dario Amodei CEO of Anthropic: Claude, New models, AI safety and Economic impact
In Good Company with Nicolai Tangen · 26 Jun 2024
Source: https://shows.acast.com/in-good-company-with-nicolai-tangen/episodes/dario-amodei-ceo-of-anthropic-claude-new-models-ai-safety-an
Transcript: https://podcasts.opennash.com/shows/in-good-company-with-nicolai-tangen/dario-amodei-ceo-of-anthropic-claude-new-models-ai-safety-and-economic-impact-3a2bf6/ (via OpenNash Podcasts, machine-generated)

Hiya everybody, welcome to In Good Company. Today, really exciting, we have, uh, Dario Amodei, the CEO and co-founder of Anthropic. Visiting. Now, Dario, he's a superstar in the AI world, and together with his team, has developed a Claude language model, one of the best out there, and they are backed by Amazon and Google. Now, you are a leading, uh, figure, Dario, on AI safety and ethics, and you even interrupted your holiday to come here and, uh, and to talk to us.

So, big thanks for coming.

Thank you for having me.

Now, what are the latest breakthrough in AI?

Yes, um, so, a few things I could talk about, um, 1 is, you know, I think the scaling trends of AI are continuing, so I think we're gonna see over the over the next year, you know, much bigger and more powerful models that are able to do greater tasks. In fact, by the time this podcast airs, a new model will be out from Anthropic that will probably be the most intelligent and powerful model in the world.

Um, but one area I'm particularly excited about that we're developing in parallel with that is interpretability of models, the ability to see inside our AI models and see why they make the decisions they make. That area has been mainly a research area for the last few years, and it's just at the beginning of starting to have practical applications. So, that's one area I'm very excited about.

Why is that so important?

Um, so, uh, you know, uh, if you look at what what AI models do today, often you won't understand why an AI model does what it does. Um, you know, I was just, uh, I was just talking to someone, uh, at at lunch, uh, let's say you want, uh, you know, consider your industry. Let's say you want an AI model to be trained on some data, to be able to predict, you know, what what happened, did a, you know, what, you know, uh, in in with particular set of financial data.

Um, 1 problem you have with training a model to work on that is that if you train it on data in the past, the model might have memorized it because it was trained on, uh, you know, it it it basically knows what happens. It it knows the future in that case. Interpretability might allow you to tell the difference. Is the model deducing the answer to the question, or is it memorizing the answer to the question? Similarly, if a model acts in a way that, say, shows prejudice against a particular group, or or appears to do so, can we look at the reasoning of the model, you know, is it really being driven by prejudice?

There are also a number of legal requirements, right, in the EU, you know, there's a right to explanation. And so, interpretability, being able to see inside the model, could help us to understand why the models do and say the things that they do and say, and even to intervene in them and s and change what they do and say.

So, uh, so a while back, you stated that we still don't know how the advanced AI models work. Does this mean that this will solve this problem?

Um, you know, I wouldn't say solve, I would say we're at the beginning. Maybe we now, like, understand 3% of how they work.

Really?

Um, uh, you know, we're we're at the level where we can look inside the model and we can find features inside it that correspond to very complex concepts, like, uh, one one feature might represent the concept of hedging or hesitating. The the, uh, particular genre of music, a particular type of metaphorical situation that a character could be in, or the idea of, you know, again, prejudice for for or against various groups. Um, so, we have all of these features, but we think we've only found a small fraction of what there is, and what we still don't understand is we don't understand how all of these things interact to give us the behaviors we see from models every day.

So, you know, it's a little like the brain, right? We can do brain scans, we can say a little about the human brain, but, you know, we we don't we we don't have a spec sheet for it. We can't go and say, well, this is this is why that that person did exactly what they did, so we're

But will we ever, uh, understand fully how they work?

Um, I don't know about, you know, fully down to the last detail, but I think progress is happening fast, and I'm I'm optimistic about getting

But is pro is progress happening faster than than complexity of the new models?

That that is that is that is a great question, and that is the thing we're contending with. So, we are putting a lot of resources behind interpretability of language models to try and keep pace with the rate at which the complexity of the models is increasing. I think this is one of the biggest challenges in the field. The field is moving so fast, including by our own efforts, that we wanna make sure that our understanding keeps pace with our our abilities, our capabilities to produce powerful models.

What's so good about your model? So, this is a Claude model, right?

this is the Claude models,

By Anthropic?

Yeah, so to give some context, we recently released, um, a set of Claude 3 models, um, uh, they're called Opus, Sonnet, and Haiku. Um, they're different tradeoffs between power and intelligence and, uh, uh, you know, speed and low cost while still being intelligent. Um, uh, at the time that Opus was released, it was actually the best all-around model in the world. Um, but I think one thing that particularly made it good is that we put a lot of engineering into its character.

And we recently put out a post about, you know, how do we design Claude's character? People have generally found the Claude models are warmer, more human, they enjoy interacting with them more. Some of the other models sound more, uh, robotic, more uninspired. Um, we're continuing to innovate quickly, and as I said, by the time this podcast comes out, we'll probably have at least part of a new generation of models out. Um,

Tell me about the new one.

Um, uh, so, uh, I can't say I can't say too much about it, but, But if you if uh

you had to say a bit?

Um, but if I if I had to say a bit, you know, I would say that, uh, we're pushing the frontier. Right now, there's a tradeoff between speed of model speed and, um, uh, low cost of models and quality. So, you can imagine that as a trade-off curve, right, a frontier. There's gonna be a new generation of models that pushes that frontier outward. Um, and and so, you know, you're gonna see by by the time this podcast is out, you know, we'll have we'll have we'll have a name for it, um, uh, for at least at least some of those models, and we'll see that things that you needed the most powerful model to be able to to be able

to do, you'll be able to do with some of the some of the mid-tier or low-tier models that are faster, cheaper, and even more capable than the than the past than the the So Last generation, the previous model.

So, Dario, what's going to be the wow factor here? So, when I get this model, w- w- you know, what is what what is it going to do to me?

Yeah, um, you know, you're gonna see, uh, models that are better at, say, things like code, math, uh, better at reasoning. One of my favorite is, uh, you know, uh, biology and medicine, that's one of the sets of applications I'm most excited about for for the new models. Um, so, uh, you know, the models we have today, they're kind of like early undergrads in their knowledge of many things, or, like, interns, and I think we're starting to push that boundary towards advanced undergrads, or even graduate level knowledge.

And so, when we think of use of models for drug development or, you know, in your own industry, use of the models for thinking about investing or or even trading, um, uh, I think the models are just gonna get a good deal more sophisticated at those tasks. Um, and, you know, we're hoping that every few months, we can release a new model that pushes those boundaries further and further.

Mm-mm. Now, one of the things that which has accelerated lately is just how we, um w- how we kinda weave AI into everything we do. And, uh, you know, with recently the announcement from from Apple, uh, and OpenAI, um, what is h- how do you look at this?

Yeah. Yeah. So, um, you know, Anthropic thinks of itself, uh, you know, more as providing services to enterprises than it does on the consumer side. Um, and so, we're thinking a lot about how to integrate AI in work settings. Um, so, if you think about, you know, today's models, today's chatbots, you know, it's it's a bit like if I use them in an enterprise setting, it's like if I took some, you know, random person on the street but who is very smart, but who knew nothing about your company, and I brought them in and I asked them for advice.

What I'd really like is someone that acts more like an AI model that acts more like, uh, someone that's been, you know, trained with knowledge of your company for many years. And so, we're working on connecting our AI models to knowledge databases, having them cite work, having them be able to use internal enterprise tools, and really integrate with the enterprise as, you know, sort of a virtual assistant to an employee. Uh, so, that's that's one way I think about, you know, dr- really driving the integration.

So, if you look at the long-term goal of Anthropic, what what is the long-term goal?

Yeah, you know, if I think about our long Cause you're you're term goal

Only, like, 3 years old, right?

Yeah, we're we're only 3-1/2 years old. Yeah, we're by far the, you know, the newest player in the space that's been able to build models on the frontier. You know, we're a public benefit corporation, and I think our long-term goal is to make sure all of this goes well. Um, and, you know, that's being done, obviously, through the vehicle of a company. But if you think about our long-term strategy, what we're really trying to do is create what we call a race to the top.

So, you know, race to the bottom is this well-known thing where everyone, you know, fights to cut corners cause the market competition is so intense. We we think that there's a way to have the reverse effect, which is that if you're able to produce higher standards, innovate in ways that make the technology more ethical, um, then others will follow suit. They'll either be inspired by it, or they'll be kind of, you know, bullied into it by their own employees or public sentiment, or ultimately, the law will go in that direction.

And so, we're hoping to kind of provide an example of how to do AI right and pull the rest of the industry along along with us. Um, that's a lot of the work behind our interpretability work, behind our safety work, behind how we think about responsible scaling. We have something called a responsible scaling policy. So, I think our our overall goal is is kind of to try and try and help the whole industry be better.

So, you kinda pitch yourself as a good guys?

I mean, you know, I wouldn't I wouldn't say I wouldn't say anything that grandiose, right? I it's more like, I wanna you know, I think more in terms of incentives and structures more than I think of good guys and bad guys. I wanna help change the incentives So that everyone can be the good guys.

Yeah. Do you think we will care who which model we interact with, or are we going to have just, like, 1 agent who picks the model, which is the best for, you know, for that purpose? That that was kind of what Bill Gates said when he was on the, uh Yeah. I On the podcast.

You know, I think I think it really depends on the setting. Um, a few points on this. 1 is I think, um, we are increasingly going in the direction where models are good at different things. So, for example, I was just talking about Claude's character, right? Claude is more warm and friendly to interact with. For a lot of applications and use cases, that's very desirable. For other applications and use cases, a model which focuses on different things might be helpful.

Some people are going in the direction of agents. Some people are going in the direction of models that are good as code. Claude, for example, is, uh, uh, another thing it's good at is creative writing. And so, I think we're gonna have an ecosystem where people use different models for different purposes. Now, d in practice, does that mean you're kind of there's something that's choosing models for you? I think in some consumer contexts, that will be the case.

I think in other contexts, someone will say, Oh, yeah. No, you know, the the job I'm doing, or the kind of person that I am, I wanna use this particular model all the time.

Well, what makes a warm model? Is I mean, how can you make a model friendly? Is it more humanistic or more polite, or Just, like, you know, Putting some red hearts if In between, or

And we actually try and avoid too many emojis because it gets annoying. But, um, uh, uh, y- you know, if you I don't know, if you go on Twitter and see some of the comments when people when people interact with Claude, um, it it, you know, it it just kind of I don't know how to describe it. It just kind of sounds more like a human, right? I think a lot of these bots have certain ticks, right?

Like, you know, models will you know, there are certain phrases. I apologize, but as an AI language model, I can't do X, Y, and Z, right? That's kind of like a common phrase. And we've helped the model to to vary their thinking more, to sound more like a human, that that kind of thing.

Now, um, y- when you launch new models, you got a pretty pretty good, uh, predictions on how accurate it will be, right? It's a function of number of parameters and so on. Um now, to get to AGI, what how far out are we there? I mean, that this is the general intelligence, so Are we more he more intelligent than So, I've said this Humans?

I've said this a few times, but, you know, back in 10 years ago, when all of this was kind of science fiction, I used to talk about AGI a lot. I I now have a different perspective where I don't think of it as 1 point in time. I just think we're on this smooth exponential, the models are getting better and better over time. Um, there's no one point where it's like, Oh, the models weren't generally intelligent, and now and now they are.

I just think, you know, like like a human child learning and developing, they're getting better and better, smarter and smarter, more and more knowledgeable. And I don't think there'll be any single point of note. But I think there's a phenomenon happening where, over time, these models are getting, you know better, better, and better than even the best humans. Um, I do think that if we continue to increase the scale, the amount of funding for the models, if it goes to, say, 10, So, now so billion or

Now, a model would cost what? $100,000, 000,000? You know? And then

now, 100,000, 000, there are models in training today that are more like a billion. Um, I think if we go to 10 or 100 billion, and I think that will happen in 2025, 2026, maybe 2027, um, and the algorithmic improvements continue apace, and the chip improvements continue apace, then I think there there is, in my mind, a good chance that by that time, we'll be able to get models that are better than most humans at most things.

So 10 billion, you think, a model will be next year?

Um, I think that the training of of order 10,000, 000,000, 000 dollar models, yeah, could start sometime in 2025. I'm not

Not many people can participate in that race.

No. No, no, no. And, you know, I I you know, of course, I think there's gonna be a vibrant downstream ecosystem, and there's gonna be an ecosystem for small models.

But you don't have money. You don't have that much money. Uh, I mean, Yet.

We have of order of order that, um, we've raised, uh, I believe it's, uh, a little over eight 8 billion to date. Um, so, generally generally of order that, and of, uh, you know, of course, we're we're, you know, we're we're we're always, uh, we're always interested in getting to the next level of scale.

Yeah. Um, now, this is, of course, also a function of of the chips, um, and we just learned that NVIDIA is halving the time now between launches, right? So, in the past, every other year, now it's more like every year. So, what what are the implications of this?

Yeah, um, you know, I think that is a you know, I can't speak for NVIDIA, but I think that is a natural consequence of the recognition that, uh, chips are going to be super important, right? And also facing competition. Um, Google is building their own, you know, chips, as we know. Amazon is building their own chips, uh, you know, Anthropic is collaborating with with both to, uh, to, uh, you know, to work with those chips, and, you know, without getting specific, what I can say is that the, you know, the chip industry is getting very competitive, and there are some very strong offerings from How how A large Far behind number of players.

How far behind would Google and Amazon be in the chip development?

I you know, that's not something I could say, and it's not it's not

But I mean, if you One-dimensional. So but just some kind of indication.

Yeah, I mean, you know, what what again, you know, I would just I would just repeat that I think there are now strong offerings from from multiple players that have been useful to us and will be useful to us in different ways.

Okay, so it's not only about NVIDIA anymore, is that what you're I I Saying?

Don't think it's only about NVIDIA anymore, um, but, you know, of course, you look at their stock val- you look at their stock price, which, you know, you might certainly

For sure.

Certainly aware of, and, you know, it's it's it's an indicator, I think, both about them and the industry.

Yeah. You mentioned that you were more on the enterprise side and not necessarily on the consumer side, but, um, just lately, there has been more talk about, uh, having chips in in phones, and we talk about, uh, AI, PC, and so on. What, um, how do you look at this?

Yeah. No, I think that's gonna be an important development, and again, if we go back to the curve I talked about, right, the trade-off curve between powerful, smart, but, you know, relatively expensive and slow models, and models that are super cheap, super fast, but very smart for how fast fast and cheap they are, as that curve shifts outward, we are gonna have models that are very fast and cheap that are smarter than the best models of today, even though the the best models then will be even smarter than that.

Um, and and I think, uh, we'll be able to put those models on phones and on mobile chips, and, uh, you know, they'll they'll pass some threshold where the things that you need to call to a cloud or the server for today, you can do there, and so, I'm I'm very excited about the implications of that. Um, you know, I'm of course, even more excited about pushing the frontier of where things will go, but as this curve shifts outward, an implication is that both things will happen.

Hm. We hear from Mistral, the French, uh, competitor that they have developed some really efficient, uh, kind of low-costs or lower-cost models. What what do you how do you view that?

Yeah, I mean, I, you know, I can't I can't comment on, uh, you know, what's going on at other companies, but I think we are seeing this this kind of general moving of the curve, and so, it is definitely true we're seeing efficient, low-cost models, but I think of it less as, like, things are leveling out, you know, costs are going down, and more as the curve is shifting, right? We can do more with less, but we can also do even more with more resources.

So, I think both both trends coexist.

Dario, changing tack a bit here, uh, your background. You kicked off in physics.

Yes, I was an undergrad in physics, and then, uh, uh, did grad school in neuroscience.

Yeah. So, how come you ended up in AI?

Yeah. So, you know, when I finished my physics degree, I you know, I wanted to do something that, you know, would would have an impact on, you know, humanity in the world, um, and I felt that, you know, an important component of that would be understanding intelligence, you know? That that that's one of the things that's obviously shaped our world. And that was back in the mid 2000s, and in those days, I wasn't particularly, to be honest, that excited about the AI of the day, uh, and so I felt like the best way to study intelligence in those days was to study the human brain.

So, I went into neuroscience for grad school, computational neuroscience that used some of my physics background, and and, you know, studied kind of collective properties of neurons, but by the end of that, by, you know, by the end of grad school, after that, I did a short postdoc, um, AI was really starting to work. We Really saw, you know, the deep learning revolution, I saw the work of, you know, Ilya Sutskever back back then, uh, and so I decided, based on that, to go into the AI field, uh, and I worked, you know, different places.

I was at Baidu for a bit, I was at Google for a year, I worked at OpenAI for 5 years, and and and, you Well, you Know

You were instrumental in developing ChatGPT 2 and 3, right?

Uh, yes, yes, I led the development of both of those.

Why did you leave?

Um, you know, we had reached around the end of 2020, um, we had kind of reached a point, the set of us who worked in this these on these projects in these areas, where we kinda had our own vision for how to do things. Uh, so, uh, you know, again, you know, we had this picture that I I think I've already kind of implicitly laid out of 1, you know, real belief in this scaling hypothesis, and 2, in the importance of safety and interpretability.

So, it was the safety side which made you leave?

I mean, I think, you know, we just we just had our own vision of things. There were a set of us who were co-founders who really felt like we were on the same page, really felt like we trusted each other, really felt like, you know, w- we just wanted to do something together.

Right, but you were a bit more AI doomsday before than you are now.

Um, I you know, I wouldn't say that. My view has always been that, uh, there there are important risks, and there are benefits, and that as the technology goes on its exponential, the risks become greater and the benefits become greater. Um, and so we are, you know, including at Anthropic, very interested in these questions of catastrophic risk, right? We have this thing called responsible scaling policy, and that's basically about measuring models at each step for catastrophic risk.

What what is catastrophic risk?

So, this would be, um, I would put it in 2 categories, um, 1 is misuse of the models, which could include things in the realm of biology, or cyber, or kind of, um, you know, election operations at scale, um, things that are really disruptive to society, um, so that misuse would be one bucket, and then the other bucket would be autonomous unintended behavior of the model. So, you know, today, it might be just, you know, the model doing something unexpected, but increasingly, as models act in the world, we have to worry about them behaving in ways that you wouldn't expect.

And what was it that you saw exactly with ChatGPT, um, well, 3, I guess, then, which made you particularly concerned about this?

Yeah, it wasn't about any particular model, um, you know, if we go all the way back to 2016, you know, before I even worked at OpenAI, when I was at Google, I wrote a paper called, With some colleagues, some of whom are now Anthropic co-founders, Concrete problems in AI safety, um, and Concrete problems in AI safety laid out this concern that, you know, we have these powerful AI models, neural nets, but they're fundamentally statistical systems. And so that's gonna create all these problems about predictability and uncertainty, and if you combine that with the scaling hypothesis, and I really came to believe in the scaling hypothesis as I worked on GPT 2 and GPT 3, those 2 things together told me,

Okay, we're gonna have something powerful, and it's not gonna be trivial to control it. And so we put those 2 things together, and and that makes me think, Oh, this is an important problem that we have to solve.

Hm. How do you solve, um, the 2 catastrophic risk problems In Anthropic?

Yes, um, so, uh, one of the biggest tools for this is our RSP, our responsible scaling policy, and so the way that works is every time we have a new model that you, you know, that that represents a significant leap, a certain amount of compute above an old model, we measure it for both the misuse risks and the autonomous self-replication risks.

And how do you do that?

Um, so, we have a set of evaluations that we run, um, we've, in fact, worked with for the misuse risks, folks in the national security community. So, for example, we've worked with this company called Griffin Biosciences that, um, contracts with the US government that does biosecurity work, and they're they're the experts on responding to biological risk, and so they say, What is the stuff that's not on the Internet that if the model knew, it would be concerning?

And they run their tests, and, you know, we give them access to the new model, they run their tests, and every time so far, they've said, Well, it's better at the test than it was before, but, you know, it's not yet at the level where it's a serious concern.

So, a misuse test would be, if, for instance, if I put in, Hey, can you come up with a virus which is just going to wipe out the earth, the people? Is The That that's that's an example, I I mean, conceptually, Right?

Yes, although it's m less about answering 1 question, it's more about can the model go through a whole workflow, like, could some could some bad actor, over the period of weeks, use this model to to do s a as they were doing something nefarious in the real world, could the model give them hints on how to help them? Could the model help them through the task over a long period of time?

Okay, so what you're saying is that, uh, the AI models so far cannot, uh, do this.

They they they know individual isolated things which are concerning, um, And they get better at it every time we release a new model, but they haven't reached this point yet. And and my guess is And what about That

the other one, the autonomous? Yeah, so Uh, how far are we away from that?

We we test the models there for things like ability to train their own models, ability to pro provision cloud compute accounts, and, you know, take actions on those accounts, ability to, like, you know, simultane- you know, sign up for accounts and engage in financial transactions, just some of the measures of things that would kind of unbind the model and enable them to take actions.

How far are we away from that, do you think?

Um, I I think it's kind of the same story as with misuse, they're getting better and better at individual pieces of the task. There's a clear trend towards ability to do that, but we're not there yet. Um, I I again point to the 2025, 2026, maybe 2027 window, just as I think a lot of the the extreme positive economic im- applications of AI are are gonna arrive sometime around then. Um, I think some of the negative concerns may may start to arise then as well, but, you know, I I'm not a crystal I'm sorry, 25, Ball.

Around that, you know, I mean

So, what do you so, what do you what do you do then? Do you build in, like, a kill switch or what do you

Yeah, well, I mean, there's a number of things. Um, I think on the autonomous behavior, a lot of our work on interpretability, a lot of our work on, um, you know, we haven't discussed constitutional AI, but that's another way we provide kind of values and principles for the AI system. On the autonomous risk, what we really wanna do is understand what's going on inside the model and make sure that we design it and can iterate on it so that it doesn't do these dangerous things we don't want it to do.

Um, on misuse risk, again, it's it's m there, it's more about putting safeguards into the model so that people can't ask it to do dangerous things, and we can monitor when people try to use it to do dangerous things.

So how can so so gen so generally speaking, I mean, there's been a lot of talk about this, but how can one regulate AI? Can companies self-regulate?

Yeah. So, um, you know, one way I think about it is, uh, the RSP, the responsible scaling policy that I was describing is maybe the beginning of a process, right? That represents voluntary self-regulation, and, you know, I mentioned this concept of race to the top. Last September, we put in place our RSP. Since then, other companies like Google, OpenAI, have put in place similar frameworks. They've given them different names, but they operate in roughly the same way, and now we've heard, uh, e- you know, Amazon, Microsoft, even Meta, reportedly, is public reporting are are at least considering similar frameworks, and so I would like it if that process continues, right, where we have some time for companies to experiment with

different ways of voluntarily self-regulating. Some kind of consensus emerges from some mixture of public pressure, experimentation with what what is unnecessary versus what is really needed, um, and then I I would imagine the real way for things to go is once there's some consensus, once there's industry best practices, probably the role for legislation is to is to look in and say, Hey, there's this thing that 80% of the companies are already doing, that's a consensus for how to make it safe.

The job of legislation is just to enforce force those 20% who aren't doing it, force that companies are telling the truth about what they're doing. I don't think regulation is good at coming up with a bunch of new concepts that people Should follow.

So so where so how do you view the EU AI Act?

Yeah. So, the you know, the EU AI Act, I should say, first of all

And and the California Safety Bill as well, right?

Yeah, yeah, you know, I I should say I should say the EU AI Act, you know, kind of it's still being it's still being, um, you know, the the kind of details of it are still be even though the act was passed, the, you know, many details are still being worked out. So, you know, I think a lot of this depe- a lot of this depends on the details. Um, you know, the the California bill, you know, I would say that, um, it is very you know, i it has some structures in it that are very much like kind of the RSP, uh, and and so, you know, I think something that resembles that structure at

at some point could be a good thing, um, if I if I Do you think Have a concern, Compe- do you think Though, I think it's that we're very early in the process, right? I described this process that's like, you know, first first 1 company has an RSP, then many have RSPs, then these these kind of industry consensus comes into place. My only question would be, are we are we too early in that process?

Too early in regulation?

2- Yeah, uh uh, or or you know, that that maybe regulation should be the last step of a series of steps Yeah, what's the And so

What's the danger of regulating too early?

Um, I don't know, 1 1, um, 1 thing I could say is that I'll look at our own experience with RSPs. Um, so, if I look at what we've done with RSP, you know, we wrote an RSP in September, um, and, you know, since then, we've deployed 1 model, we're soon gonna deploy another. You you see so many things that not that it was too strict or not strict enough, but you just didn't anticipate them in the RSP, right?

Like, you know, there are various Kinds of, like, AB tests you can run on your models that are even informative about safety, and our RSP didn't speak one way or another about, like, when those are okay and when there's not, and so we're updating our RSP to say, Hey, how should we handle this issue we've never we've never even thought of, and and so, I think in the early days, that flexibility is easy. If you don't have that flexibility, if your RSP was written by a third party, um, and you didn't have the ability to change it, and the process for changing it was very complicated, I think it it could cre- it could create a version of the RSP

that doesn't protect against the risks, but also is very onerous, and then people could say, Oh, man, all this regulation stuff, all this catastrophic, it's all nonsense, it's all a pain, so, eh I I'm not I'm not against it, you just have to do it delicately and in the right order.

But we build AI into, you know, into the race between the superpowers, right? We're building it Into the weapons, the cars, the medical research, into everything. How can you how can you regulate when it's part of the the power balance in the world?

Yeah, yeah, so, I think there's different there's different questions, right? 1 question is, uh, you know, how do you regulate the use domestically? And, you know, there, I think there's there's a history of it, right? Um, you know, I I I think an analogy I would make is, like, you know, I don't know, the way cars and airplanes are regulated, right? I think that's been a reasonable story. I don't know that much about Europe, but, like, in the US, I think that's been a reasonable story.

Everyone understands there's huge economic value. Everyone understands that these things are dangerous, and they can kill people, and, you know, everyone understands, yes, you have to do this kind of basic safety testing. Um, and, you know, that's that's evolved over years. I think that's generally, um, you know, gone reasonably well, it hasn't been perfect. Um, so so, I think for domestic regulation, you know, that's that's what we should aim to, things are moving fast, but, you know, we should try to go through all the steps to get there, uh, from from an international point of view, I mean, you know, I think that's a completely different question.

That's less about regulation, and more about there's an international race to the to the bottom, and how do you how do you handle how do you handle that race to the bottom? I mean, I think it's an inherently difficult question, because, on one hand, we don't wanna, you know you know, we we we don't wanna just recklessly build as fast as we can, particularly on the weapons side, uh, you know, on on the other side, I think looking, you know, as as a citizen of the US, here I here I am in Norway, another democracy, um, I'm I'm very worried about if autocratic regimes were to lead lead in this technology, I think that's very, very dangerous, um, and so

How far behind are they now? Or are they behind?

Uh, I mean, it's hard to say. I would say that with some of the restrictions that have put been put in place, uh, on, for example, you know, shipment of chips and equipment to Russia and China, um, I think if the US government plays its plays its cards right, then, uh, you know, those countries could be kept behind, I don't know, maybe 2 or 3 years. Right? Will Um, that doesn't give us much margin. T talking Um

About democracies, will AI impact the US election?

Yes, uh, you know, I I am concerned about that, you know, Anthropic actually just put out a post about what we're doing to, you know, to counter election interference, How c- how Um

Could it interfere?

Um, so, uh, you know, if we look back at, say, the 2016 election, something that happened in that election was that there were large numbers of people who were being cre- you know, who were being, uh, paid to provide content. I don't know how effective that was in the end. It's very hard to measure, um, but a lot of the things that, you know, you had you had, you know, farms of of people being paid to do could could now be done by AI.

I think it's less that, like, you know, you could make content that people necessarily believe, it's it's more that you could kind of flood the information ecosystem with a bunch of very low-quality content that would make it hard for people to believe things that really are true.

Did that happen, for instance, in India, in the European election? I mean, is it ha- is it really happening Yeah, we don't This year?

We don't have particular evidence of the use of our models. We've banned their use for electioneering, and we monitor use of the models. Um, you know, occasionally, we shut things down, but I don't think we've ever seen a super large-scale operation. I can only speak for use of our models, but I don't I don't think we've ever seen a super large-scale operation there.

Changing, um, topic slightly, you mentioned that you thought we were going to see some extreme positive effects of AI in 25, 26. What are these extremely positive things?

Yeah, so, again, if we go back to the analogy of, like, today's models are like undergraduates, um, if we get to the point where the models are

I I I suspect you were a better undergrad than me, though. I I can kind of feel it across I couldn't I The table.

Couldn't speak to it, but, um, uh, but if, uh, you know, let's say those models get to the point where, you know, they're kind of, you know, graduate level or strong professional level, think of biology and drug discovery. Think of, um, a model that is as strong as, you know, a Nobel Prize-winning scientist or, you know, the head of the you know, the head of head of drug discovery at a major pharmaceutical company, um, uh, I look at all the things that have been invented, you know, if I look back at biology, you know, CRISPR, the ability to like edit genes, if I look at, um, you know, CAR T therapies, which have cured certain kinds of cancers, there's probably

dozens of discoveries like that lying around, and if we had a million copies of an AI system that are as knowledgeable a and as creative about the field as all of those scientists that invented those things, then I think the rate of of those discoveries could really proliferate, and, you know, some of our really, really long-standing diseases, uh, you know, could be could be addressed or even cured. Now, I don't think all of that will come to fruition in 2025, 2026, at most, I think that the caliber of AI that's that's capable of starting the process of addressing all those things could be ready then.

It's another question of, like, applying it all, putting it through the regulatory

Sure, but what could it do to productivity in society?

Yeah, um, uh, you know, I think of, again, virtual assistants, like, uh, you know, uh, like a chief of staff for everyone, right? I have a chief of staff, but not everyone has a chief of staff. Uh, you know, could could everyone have a chief of staff who helps them, uh, you know, just deal with every deal with everything that lands on their desk, everything So if That lands We so if On their plate?

Everybody had that kind of thing, what would it do could you put a number on productivity gain?

Uh, you know, I'm not an economist, I couldn't tell you, you know, X, I couldn't tell you X percent, um, but if we look at kind of the exponential, right? If we look at, like, you know, revenues for AI companies, like, th- it seems like they've been growing roughly 10x a year, and so, you could imagine getting you can imagine getting to the hundreds of billions in, you know, 2 to 3 years, and getting even to the trillions, which per year, which no company has reached.

But I'm saying that's revenue for the company.

Revenue for the comp- right, But what And then and

About productivity? There's What about productivity in Productivity Society?

In society, right? So, that depends on how much is this replacing something that was already being done versus doing new things. I think with things like biology, we're probably gonna be doing new things, so I don't know if let's say you extend people's, you know let's say, you know, you extend people's productive ability to work by 10 years, right? That could be, you know, 1/6 of the whole economy, um,

Do you think that's a realistic target?

I mean, again, like, I know some biology, I know something about how the AI models are gonna happen. I wouldn't be able to tell you exactly what would happen, but, like, I can tell a story where it's So so Possible, right?

Fif- so 15%, and when will we so, when could we have added the equivalent of 10 years to our life? I mean, how long what is what's the timeframe?

Again, like, you know, this involves so many unknowns, right? If I if I try and give an exact number, it's just gonna sound like hype, but, like, a thing I could a thing I could imagine is, like, I don't know, like, 2 to 3 years from now, we have, like, AI systems that are, like, capable of making that kind of discovery, like, 5 years from now, like, you know, those b- those discoveries are actually being made, and 5 years after that, it's all gone through the regulatory apparatus and and really ha- so, you know, we're talking about more we're talking about, you know, a little over a decade, but, really, I'm just pulling things out of my hat here, like,

I don't know that much about drug discovery, I don't know that much about biology, and, frankly, although Well, that's a Although I've Bit to be Invented AI scaling, I don't know that much about that either, I can't predict Well, It.

I think you know know more about this thing than than most of us. Um, And How do you And and How do you how Yet, and Do you

Yet, and yet, it is also hard to predict.

Absolutely. Have you thought about what it could do to inflation?

Yeah, um, so, again, I'm not I'm not an economist, um, if we look at, uh if we look at in inflation, I mean again, using my limited economic reasoning, I think if we had very large real productivity gains, um, that would tend to be deflationary rather than inflationary, right? Like, you would be able to do you'd be able to do more with less, the dollar would go further, um, so, directionally, at least, that suggests disinflation, but, Again

Totally, but what what kind of magnitude how What Big of a

Kind what kind of mag- I mean, I I that you are more the expert on than I am, maybe I should ask you to predict that.

How do you work with the hyperscalers, like, you know, s you know, some of your shareholders, like Google and and and Amazon?

Yeah, yeah, so, you know, I think, um, you I'm I'm Know

Sorry, I'm just getting straight. These are called hyperscalers because why?

Um, I actually don't know the reason for the name, but, you know, they're they're they're hyper-cap companies in terms of their valuation, but but also they make very large, you know, they m very large AI data centers. I seem to So how Refer to the second one, but, uh

Absolutely. How do you work with them?

Um, so, uh, you know, I would say that the relationship with these companies makes sense in the sense that we have complementary inputs, right? That they provide the chips in the cloud, and then we provide the model, and and then that model is something that, again, can be sold to customers on the cloud, so, there's kind of a layered cake where we provide some layers and they provide the other layers, so this partnership these partnerships make sense on on multiple grounds.

You know, at the same time, we've always been very careful, right? We have our own kind of value as values as a company, our own way of doing things, and so we try to stay as independent as possible, and one of the things we've done is, of course, we have relationships with multiple of these cloud providers, right? We work with both Google and Amazon, and and that has allowed us some flexibility in in in our ability to make sure that there isn't too much, um, uh, exclusivity, and that we're kind of, uh, you know, free to deploy our models on multiple surfaces.

The fact that these companies are becoming so incredibly powerful, what kind of systemic risk does that pose?

Yeah, I mean, you know, I would say that and this is maybe broader than AI, it maybe relates to just, kind of, the era that we're that we're living in. There are certain eras in history where, you know, there's a powerful there's a powerful technology or there's an economic force that kind of tends to concentrate resources, um, uh, you know, I I think, you know, probably the same thing happened in the 19th century, and so I think it I think it actually is important to make sure that the benefits are shared by all, um, so one thing that's often on my mind is there's been, for example, very little penetration of AI and language models in some parts of the

developing world, right? In, like, sub-Saharan Africa, and so how do we how do we bring these models to those areas? How do we even help with challenges in those areas, like health or education? Um, so, I I I definitely agree, we're we're, you know, living in an era of more cons- concentrated wealth, and that's an area of concern, an area that, you know, we should we should we should, you know, we should do what we can to find countervailing forces.

But is it uh, w- what's the risk in in that these companies are now becoming more powerful than than countries and governments?

Yeah, I mean, you know, this is this is kind of, you know, what I said on in terms of in terms of regulation, like, you know, I think that, uh, AI is a very powerful technology, um, and, you know, our governments, our democratic governments, do need to step in and set some basic rules of the road, right? It needs to be done in the right order, it can't be stifling, but I think it I think it does it does need to be done, uh, Because, you know, because because, as you said, like, we're getting to a point where, um, the amount of concentration of power can be, you know, can be greater than that of national economy, you know, g-

na- national governments, and, you know, we don't want that to happen at the end of the day, you know, the you know, all the people of the country and all entities, including companies that that work in it, you know, they ultimately have to be accountable to democratic processes, right? There's there's no other there's no other way.

Would will AI increase or decrease the difference between rich and poor countries?

Um, that, I think, depends on what we choose to do with it, Yeah, the way Right?

You the way you look at, um, the way you look at the path forward just now.

Yeah, so, you know, I would say that we are looking for ways, um, for it not to make you know

Sure, but is that happening?

I would s I mean, it's too early to say with, like, how you know, with how the technology is being deployed, um, I would definitely say, I I do see something related to it that's a little worrying to me and that we're trying to counter, which is that, uh, if you look at the natural applications of the technology, the you know, the the things that customers are are, you know, the the most eager customers that come to us often, I think because we're a Silicon Valley company, often the most eager c- customers are other kind of technologically forward Silicon Valley companies that kind of also use the technology, and so I think there's this danger of the what you might

call like a a kind of closed loop where it's like, an AI company, you know, supplies a, like, AI legal company, which supplies an AI productivity company, which supplies a, you know, some other company in Silicon Valley, and, you know, is it all a closed ecosystem where

And it's all being used It's venture By the mo- and it's all being used by the most highly educated E- e- People.

E- exactly, and and so how do we break out of that loop? And so, we thought about a number of ways to break out of that loop, one of the reasons I talk about biology and health is that I think biology and health can be used to help us to break out of that loop, um, you know, innovations in health, assuming we distribute them well, can apply to everyone, um, I think things like education, uh, can help here, um, another area that I'm very excited about is use of AI for provision of everyday government services, um, you know, I don't know what the names of these services are in Norway, you know, in the US, every time you interact with,

like, the DMV, the IRS, various social services, people almost always have a bad experience, And it it drives cynicism about the role of government, um, and I would I would love it if we can modernize government services that everyone used so that they can actually deliver what people across the world need.

Mm. I have to say that I think in this country, we are we are fortunate in that we are not so many people, and we are Y- y- Uh, we got a you know, it's heavily digitalized You you And and it's pretty Are probably Good.

Much better than we are at this. I think we I'm reacting Could To my experience in the United States, which which, you know, I think could be better.

Yeah. So, net net, what do you think? Will, in 10 years' time, will the gap between rich and poor be bigger or smaller?

I I just have to say, like, if we handle this the right way,

Yeah, no, I hear what you're If we handle Saying, but I mean, It the right What is it Way, we can I hear what

Narrow the gap.

I hear what you say, but The the default What do you think? What do you think will happen?

I I don't know what I think will happen. I know that if we are not ex- extremely thoughtful about this, If we're not extremely deliberate about it, then, yes, it will increase the gap.

Okay, who will make the most money on on AI? Will it be the chip manufacturers, or will it be, uh, you guys, or the scalers, or, uh, or the consumers, or Yeah, I The companies? I my boring answer Or

Is that I think it's gonna be distributed among all of them, and that The pie is gonna be so large, um, that in some ways it may not even matter, like, certainly right now, the, you know, the the chip companies are making the most money, I think that's because training of models comes before deployment of models comes before revenue, So, I think the way I think about it is the valuation of the chip companies is a leading indicator, um, the, um, valuation of the AI companies is maybe a present indicator, and the valuation of lots of things downstream is a lagging indicator, but but the wave is gonna reach everyone.

So, when you look at the the market cap of, uh, NVIDIA, for instance, you th- that's an indicator, I mean, wh- what do you multiply that by to find, uh, the potential impact of AI?

Yeah, I mean, what I think that and, you know, obviously, I can't give stock advice on on on podcasts about chips, So 3 trillion But, um So that's 3 But

$3 1 trillion, right?

Yeah, but but 3 trillion, so so why Which is Is that?

Which is nearly twice the size of, uh, this fund, which is the largest sovereign wealth fund in the world.

Yes, um, you know, if I think about that again, like, speaking very abstractly and conceptually, um, what's that driven by? Probably that's driven by, like, anticipated demand, like, people are building very large AI clusters, those clusters involve lots of revenue for NVIDIA, presumably, companies like us are paying for those clusters because they think that the models they build with them will generate lots of revenue, but that revenue is not present yet, and so, what we're seeing so far is just, man, people wanna buy a lot of chips, and of course, it's possible it's consistent with the whole picture that all of this will be a bust, the models don't turn out to be, you know, that powerful, like companies like

Anthropic and the other companies in the space don't do as well as we expected because the models don't keep getting better, that always could happen. That's not my bet, that's not what I think is gonna happen. What I think is gonna happen is that these models are gonna produce a great deal of revenue, and then there's gonna be even more demand for chips, NVIDIA's value will go up, the AI companies value will go up, all these downstream company, you know, that's the op- that's the bullish scenario that I'm that I'm I'm betting on by by leading this company, um, but I'm I'm not sure, it could go the other way, I don't think anyone knows

Where is the, uh, biggest constraint just now? I mean, is it in is it in chips, uh, talent, uh, algorithms, Yeah, I I would Electricity?

Say, um, you know, a big bottleneck we're dealing with is data, um, but as I've said elsewhere, uh, we we and other companies are working very hard on synthetic data, um, and I think that bottleneck's gonna be lifted.

So, data, just to get it straight, that's just information you feed into your models to Yeah, base information Train them, right?

That's that's fed into the models, but we're getting increasingly good at synthesizing the data.

Tell me what is synthetic data?

Uh, so synthetic data, um, the example I like to give is, uh, 7 years ago, uh, you know, DeepMind, uh, as part of Google, uh, produced the AlphaGo model, which was able to beat the world champion in Go, and there was no that version or there was a version of it called AlphaGo Zero that was not trained on any humans playing Go, all it did was the model played Go against itself for for for a long time, basically, for forever, um, uh, and and so, basically, with just the little tiny rules of Go and the models playing against each other, pushing against each other using that rule, they were able to get better and better to the level where they

were better than any human, and so you can think of those models as having been trained on synthetic data that are created by other models with the help of these this kind of logical structure of the rules of Go, and so I think there are things analogous to that that can be done for language models.

Mm, hm. How do you think AI will affect geopolitics?

Yeah, I you know, I think that's a big one, um, uh, my my view is that, again, if we get to the level of AI systems that are better than the best professionals at a wide range of tasks, um, you know, then tasks like military and intelligence are gonna be among those tasks, uh, and, uh, you know, we shouldn't be naive, everyone is going to try to deploy those, um, I think we should try to create cooperation and restraints where we can, um, but that, uh, you know, in many cases, that won't be possible, and when it isn't possible, you know, I'm I'm on the side of democracies in the free world, um, I I want to make sure that

the future is democratic, that as much as possible of the world is democratic, and that democracies have, um, a lead and an advantage on the world stage, um, the idea of powerful AI plus autocracies terrifies me, and I don't want it to happen.

Should each country have its own language model? Should Norway build a language model?

Uh, you know, i i

5,000, 000 people.

It it it i it kind of really depends on what you're aiming to do, right? Um, uh, you know, it it may make sense from a national security perspective for every country to have language models, I think an idea that might work, uh, that, you know, like, another direction we could go in is imagining some kind of democratic coalition or cooperation in which democratic countries, you know, work together to provide for their mutual security, to protect each other, to protect the integrity of their democratic processes, maybe it makes sense for them all to pool their resources and make a very small number of very large language models, but then there, you know, there may also be value in decentralization, I

don't have a strong opinion on which of those is better.

Is it a national security issue that US controls AI? Um, you know, uh Should we sh- should Europe be worried about this?

Yeah, I mean, again, you know, I I wou- I would go to, you know, each each country has to kind of worry about its own security, even separately from its allies, um, you know, I think that's that's more of a question for for kind of for kind of for kind of individual governments, I mean, you know, I would think of it probably this is a provocative analogy, but a little like nuclear weapons, right? Some countries, even though they're allies, feel the need to have their own nuclear weapons, for example, France, um, other countries say, no, we trust that we're being, you know, protected by the US and the UK and France, um, I think it may be somewhat similar with

these more powerful models, and I think it's less important how many of them exist within the democratic world, as that the democratic world is in a strong position relative to relative to autocracies.

Mm, mm. You talk about, um, cooperation and partnerships, and so on. Do you guys in in AI actually like each other?

Do do we do we in AI actually like each other? I mean, we've done a number of collaborations. Um, so, you know, uh, I think, uh, very early on when I was at OpenAI, you know, I drove the original RL from Human Feedback paper was considered safety work, and this ended up being a collaboration between DeepMind and OpenAI, and we've worked together, um, you know, in organizations like the Frontier Model Forum, uh, to collaborate with each other.

That said, I mean, you know, I'll be honest, I don't think every company in this space takes issues of safety and responsibility e- equally seriously from from from all the other companies. But, you know, instead of pointing fingers and But is that's Saying

Why is that the kind of things that make you not being so keen on other companies? Is it their view on safety and security? No, let me I mean, it's Let me It's one of the few industries Where you ev- where you even consider, you know, having a cage fight between, you know

Yeah, so, I'm I'm I'm not a fan I'm not a fan of the cage fights, I'm not a So, I mean, Fan of the You're you're The feuds

In good faith, so I'm

Between people.

Sure you'd do well, but, I I You know, I know, eh, well, Even even though I suspect it won't be your strength.

I eh, no, I fighting in cage fights is not not my not my forte, but the thing I was gonna say is, look, instead of instead of pointing fingers, instead of having feuds and saying, This guy's the bad guy, this guy's the good guy, let's think systemically, right? Um going back to, like, the race to the top idea, right? The idea that it's like, let's set standards, instead of pointing fingers at people doing something bad, let's do something good, and and then a a lot of the time, people just follow along.

We invent an interpretability idea, you know, just a few weeks ago, we we put out, you know, I was was talking about it a few minutes ago, this innovation in interpretability, being able to see inside the model. A few weeks later, we got similar things from OpenAI. We've seen, internally, other companies increase their prioritization on it. So, a lot of the time, you can just do something good, and you can inspire others to do something good.

Now, if you've done a lot of that, if you've set these standards, if they're industry standards, and then there's someone who's not complying with them, there's something that's really wrong, then you can talk about pointing fingers.

Yeah. Let's spend a few minutes talking about culture. How many people are you in the firm?

We are, uh we are about, uh, 600, uh, as of as of a couple weeks ago. Uh, I've been on vacation, so it may be even higher now.

What's the culture like?

Yeah, um, I would describe a few elements of the culture. Um, uh, one one element of the culture is is what I describe as Do the stupid simple thing that works. Um, a number of folks at Anthropic are, uh, ex-physicists, uh, because, you know, I myself have that background, and a couple of my co-founders have that background, including 1 person who was actually a professor of physics before he, uh, before he co-founded Anthropic, and, you know, physicists look for simple explanations of things.

So, one of the e- elements of our culture is, you know, don't do something overcomplicated, right? A lot of academic ML research tends to overcomplicate things. We go for the simplest thing possible that works. We have the same view in engineering, and, again, we have the same view even on things like safety and ethics, On interpretability, on, you know, our constitutionally AI methods, they're all incredibly simple ideas that we just try and push as far as we can.

Even this race to the top thing, right? You can say it in a sentence or 2, right? It's not complicated, like, you don't need a 100 page paper to talk about it. It's a simple strategy: do good things and try and encourage others to follow.

When you hire 600 people in 3 years, how can you be confident that they are good?

Yeah, um, so, uh, you know, I think I think candidly, one challenge of the AI industry, right, is how fast everything moves. So, uh, you know, in a normal startup, things, you know, might grow 1.5x or 2x a year, um, we recognize that in this field, things move so fast that faster growth is required in order to meet the needs of the market, uh and that ends up entailing, um, faster growth than than usual. I I was actually worried about this at the beginning of the con- I said, Oh my god.

We have this dilemma, how do we deal with it? I have generally been positively surprised at how well we've been able to handle it so far, right? How how good we've been able to scale hiring processes, how much I feel everyone is both technically talented, knowledgeable, and just just generally kind and compassionate people, which I think are equally important as So Hiring technically talented

So, what do you look for? People. So, here I'm sitting, I'm you're interviewing me now for a position, what do you look for?

Yeah, I mean, again, you know, we look for willingness to do the simple thing that works, uh, you know, we look for talent, um, it generally, you know, we don't necessarily look at years of experience in the AI field, like, a number of folks we hire are physicists or other natural scientists who, you know, have have maybe only been doing doing AI for a month or so, right, have only have only been doing a project on their own, and so we look for we look for ability to learn, we look for, um, you know, we look for curiosity, ability to quickly get to the heart of the matter, and then in terms of values, you know, we we just look

for thinking in terms of the public benefit, Right? Like, it's it's less that we have particular opinions on what the right policies for Anthropic is or what the right things to do in the world, it's it's more we we want to carry a spirit as we as we scale the company, and it gets increasingly hard as the company gets bigger and bigger, um, because, you know, how do you how do you find how do you find all these people?

Uh, but we we want people who who c- carry some amount of public spirit, who understand, on one hand, that Anthropic needs to be a commercial entity to be To be close enough to the center of this to have an impact, But when you hire, But to to Do you Understand that that at that in the long run, we're we're aiming to, you know, we're aiming for this this public benefit, the societal impact.

When you hire, do you feel you have unlimited amount of money?

Um, you know, I think compute is almost all of our expenses, um, I won't give an exact number, but, you know I I think, you know, can be publicly backed out that it's more than 80%, and, so, So sal- You know

Salaries doesn't matter really.

Yeah, in terms of in terms of paying people, we think more about what is fair, right? We wanna do something that's, you know, that's fair, that meets the market, that treats people well. It's it's less of a consideration of, like, you know, how much how much money are we spending because compute is is the biggest expenditure, it's more How can we how can we create a place where everyone feels they're treated fairly and people who do equal work get equal pay?

Hm. Now, you work with all these brilliant minds and kind of geniuses and and perhaps even some some, uh, prima donnas. How what's the best way to manage them or lead them? Yeah, you know, I I Think I guess they can't be managed, One of the most So, you need to lead, One of the Right?

Most important principles is is just the thing you said, which is letting creativity happen, um, uh, you know, if things are too top-down, then it's h- it's hard for people to be fully creative, uh, it, uh, y- you know, if you look at a lot of the big innovations in the ML field over the last 10 years, like the invention of the transformer, you know, no one at Google kind of ordered, you know, Oh, you know, here's the project, here's what we're trying to produce.

It was it was just kind of, you know, it was it was a decentralized effort. At the same time, you have to make a product, and everyone has to work together to make a single thing, and I think that creative tension between we need new ideas, but we need everyone to kind of contribute to one thing, I think that creative tension is where is where the magic is, finding finding the right combination so that you can get the best of both worlds.

Hm. You run this company together with your sister, right? Yes, yes, we How how is that?

We we both worked at, uh, OpenAI, and then we both, uh, founded Anthropic together, um, it's really great, so, um, you know, the real division of labor is, you know, she she does most of the things you would describe as running the company day to day, managing people, figuring out the structure of the company, um, you know, making sure we have, you know, a CFO, a chief product officer, um, you know, making sure comp is set up in a reasonable way, making sure the culture is good, I I think more in terms of, uh, kind of ideas and strategy, um, every couple weeks, I'll give a talk to the company, basically a vision talk, where I say, Here's some things

we're thinking about strategically, these aren't decisions, this is kind of a a picture of what leadership is thinking about, What do we think is gonna be big in the next year? Where do we think things are going? Both on the commercial side, the research side, the public benefit side.

Is she younger or older than you?

Uh, she is 4 years younger than me.

Is she cleverer than you?

Uh, we we we are both extremely skilled in different ways.

What do your parents do?

Um, so, my my father is deceased, he was he was previously, uh, a craftsman, um, my mother's, uh my mother's retired, uh, she was, uh she was a project uh, project manager for public libraries.

How how were you raised?

Uh, how was I raised? Um, you know, it it there really was, I think, a big focus on social responsibility and helping the world, like, that was that was, I think, a big thing for my for my parents, you know, they really thought about, How do you how do you, you know, how do you make things better? How do you how do people who have been born in a fortunate position, um, you know, reflect their responsibilities and, you know, deliver their responsibilities to those who are who are less fortunate, and, you know, you can kind of see that in the public benefit orientation of the company.

So, like, the 14-year-old Dario, what was he up to?

Um, I mean, I was really into, you know, math and science, like, you know, I did, like, math competitions and all of that, but, you know, I was I was just also thinking about, like, uh, you know, what, ha- I you know, how could I how could I apply those skills to, you know, invent something that would that would help people?

Did you have any friends?

Did I have any friends? Um, uh, you know, less than less than I would have less than I would have liked, you know, I was I was I was a little bit, uh, I was a little bit introverted, but, um, you know, there there were there were people who who, you know, who I knew back then who I still who I still know now.

So is Anthropic, like, a revenge of the nerd?

Uh, you know, I wouldn't really put it I wouldn't really put And I It in those

Think that's a good thing. Terms Yeah, I love that kind of stuff, Yeah, I wouldn't Okay?

Really put it in those terms, if only because, you know, I'm I'm kind of reluctant to, like, set different groups against off each other, you know, like, people are different kinds of people are good at different things, you know? Like, we have a whole sales team, like, they're good at a whole different set of things than I am, like, you know, of course, I'm the CEO, so I have to learn how to do some sales as well, but, you know, they're just very different skills, and one of the things you think about, uh, you realize in a company is that different kinds of people with very different kind of skills just, you know, like, you you recognize the value of

a very wide range of skills, including ones that, you know, that that you're that that that you have no ability in yourself, Right?

So what but what's what drives you now?

You know, I think we're in a very special time in kind of, like, the AI world, like, you know, this is, you know, these these things I've said about, you know, how crazy things could be in 2025 or 2026, um, you know, I think it's important to get that right, and, uh, you know, running Anthropic, I'm, you know, that's only one small piece of that, right? There are there are other companies, you know, some of them are, uh, are bigger or better known than we are, and so, uh, on one hand, you know, we have only one small part to play, but, uh, you know, I think given the importance of what's happening for, you know, for the economy, for

for for humanity, I I think we have an important opportunity to, you know, make sure that this things go well, there's a lot of variance in how things could go, um, and I think we have The ability to affect that. You know, of course, day to day, we have to we have to grow the business, we have to hire people, we have to sell products, um, and, you know, I think that's important, and it's, you know, it's it's important to to do that well so that the company is relevant, but but I I think in the long run, the thing that that drives me, or that at least I hope drives me, is is, you know, is is

is the desire to to capture some of that variance and push things in a good direction.

How do you relax?

How do I relax? Um, so, you know, I'm in Norway now, Yeah, but this Um, uh, I

This is not relaxing.

Just, uh this is not relaxing, but I I came here from from my vacation from my vacation in, uh, from my vacation in Italy, um, uh, so, you know, every every year, I take, uh, I take a few weeks, uh, I take a few weeks off to kind of relax and think about the deeper concepts, um, I go swimming every day, um, uh, actually, uh, me and my sister, uh, still play video games, we used to do this Since high school, um, and, uh, you know, now now, uh, you know, I'm I'm over 40, and she's she's like, you know

What kind of what kind of games do you play?

Uh, well, we recently got the new, uh, the new Final Fantasy game, so we played Final Fantasy in high school, it was, like, you know, a game made in the 90s, um, and they recently made a remake of it, um, and and so so, you know, we recently started playing, like, you know, the new version with all the, like, fancy graphics from, like, you know, 20 years of 20 years of progress and, well, actually, GPUs, uh, and, you know, w- we were ourselves noticing it was like, Wow, you know, this is like we used to do this when we were in high school, now we're, like, running this company.

Well well, I'm glad to hear that some people never grow up.

Uh, I I don't think we've grown in certain Ways, I don't think we've grown up, hopefully, we have in others.

Uh, talking of which, we always finish off these podcasts with, um, with, um, uh, with a question: What kind of advice do you have to young people?

Yeah, um, you know, uh, I would I would say, um, you know, get to, you know, gain familiarity with these new AI technologies, um, you know, I'm not I'm not gonna offer some kind of bromide about, you know, I know exactly which jobs are gonna, you know, be be big and which which aren't, um, I think we don't know that, and and, also, um, you know, we don't we don't know that AI won't won't touch every area, um, but I think it's safe to say that there will al- you know, there there's gonna be a role for humans in kind of using these technologies and working alongside them at the very least at the very least, understanding them and the

public debate that's going to come from them. I guess the other thing I would say, and this is already important advice, but I think it's gonna get really more important is just the the faculties about skepticism, about information, as AI generates more and more information and content, um, being discerning about that information is gonna become more and more important and more and more necessary. I hope that we'll have AI systems that help us sift through everything that, you know, that help us understand the world so that we're kind of less vulnerable to these kind of, you know, to to these kind of attacks, um, but but at the end of the day, it has to come

from you, you have to have some basic desire, some basic curiosity, some basic discernment, and so I think developing those is important.

Well, that's a a really great, uh great advice. Well, big thanks, this has been a a true blast, and I, uh, uh, wish you, uh, all the best, uh, get back to Italy, get some more rest, and do some more, uh, deep conceptual thinking. Yes, thank Very good.

You so much for having me on the podcast.

Thank you.