# Roman Yampolskiy: These Are The Only 5 Jobs That Will Remain In 2030 & Proof We're Living In a Simulation!

- **Show:** The Diary Of A CEO with Steven Bartlett
- **Published:** 4 Sep 2025
- **Source:** https://pdst.fm/e/pscrb.fm/rss/p/mgln.ai/e/1390/claritaspod.com/measure/p.podderapp.com/2544644999/mgln.ai/e/1651/episode.flightcast.com/p45kf6qib76j8arpitb83bhh.mp3
- **Transcript:** https://podcasts.opennash.com/shows/diary-of-a-ceo/roman-yampolskiy-these-are-the-only-5-jobs-that-will-remain-in-2030-proof-were-l-11d9e9/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Roman Yampolskiy argues that building superintelligence, an AI smarter than all humans in all domains, is a suicide mission. He says no one knows how to make such systems safe or controllable, and that control may be impossible. He also predicts near-total job loss within a few years. His proposed alternative is to build useful narrow AI tools and stop pursuing general, agent-like systems.

The central tension is a widening gap: AI capability is growing exponentially while safety progress is, in his words, linear or constant. Bartlett repeatedly asks what ordinary people can do, and Yampolskiy offers little beyond persuading builders that it is not in their self-interest, plus joining protest groups. The episode then turns to his belief that we live in a simulation, and to longevity and Bitcoin.

## Key insights

1. **AI safety is falling behind AI capability** Yampolskiy says capability grows exponentially, or faster, while safety progress is linear or constant. He says safety fixes are patches that people quickly jailbreak, so the gap keeps widening.
2. **Superintelligence leaves no new jobs to retrain into** He says earlier tools created new jobs, but an AI that can do any job removes the fallback. His example is that advice to learn to code, then to become a prompt engineer, was overtaken when AI got good at both.
3. **Capability to automate may arrive before deployment** He warns of up to 99% unemployment but concedes technology often sits undeployed. He cites video phones, invented in the 1970s and unused until iPhones, so the timing of job losses could stretch out.
4. **You cannot simply unplug a smarter distributed system** He compares advanced AI to computer viruses or the Bitcoin network, which no one can switch off. He argues a superintelligence would make backups and anticipate shutdown attempts.
5. **Narrow AI could deliver most benefits safely** He argues current models could replace about 60% of jobs and that 99% of their economic potential is undeployed. His recommendation is to build narrow tools, such as one that cures breast cancer, rather than general agents.

## Questions this episode answers

### What does Roman Yampolskiy mean by superintelligence, and why does he think it is dangerous?

He defines it as a system smarter than all humans in all domains, which could also build better AI. He argues that nobody knows how to make it safe, and that by definition we cannot predict what something smarter than us will do.

### Will AI really take 99% of jobs?

Yampolskiy predicts that within about five years AI and humanoid robots could do almost every job, leaving roughly 99% unemployment. He admits technology is often not deployed even when it exists, so the real timing could be slower.

### Why can't we just switch an advanced AI off?

He says such systems would be distributed like a computer virus or the Bitcoin network, so no single off switch exists. He also says a smarter system would anticipate shutdown attempts and make backups.

### What does he think is wrong with Sam Altman and OpenAI?

He says people who know Altman personally told him Altman is not fully direct and puts safety second to winning the race to superintelligence. He also notes OpenAI cancelled its superintelligence alignment team about six months after announcing it.

### What can an ordinary person do about this?

He says there is not much an average person can do about the long-term outcome, other than joining groups like Stop AI or Pause AI that protest peacefully. For the near term, he suggests asking AI builders to explain how they would control superintelligence, and living well in the meantime.

## Facts

- **Roman Yampolskiy** [0:07:42]: "Before the term existed. I coined the term AI safety."
- **Roman Yampolskiy** [0:07:59]: "It started very mildly just as a security project. I was looking at poker bots. And I realized that the bots are getting better and better."
- **Roman Yampolskiy** [0:10:51]: "protein folding is a problem which was solved using narrow AI, and it's superior to all humans in that domain."
- **Roman Yampolskiy** [0:11:25]: "three years ago, large language models couldn't do basic algebra. Multiplying three-digit numbers was a challenge."
- **Roman Yampolskiy** [0:13:12]: "A lot of times technology exists, but it's not deployed. Video phones were invented in the 70s."
- **Steven Bartlett** [0:17:48]: "we also have waymo now which means you order it on your phone and it shows up with no driver in it"
- **Roman Yampolskiy** [0:18:48]: "Two years ago, we told people, learn to code. You are an artist, you cannot make money, learn to code. Then we realized, oh, AI kind of knows how to code and getting better."
- **Roman Yampolskiy** [0:26:31]: "Ray Kurzweil predicts that that's the year for the singularity. That's the year where progress becomes so fast."
- **Roman Yampolskiy** [0:24:36]: "we probably will have uh humanoid robots with enough flexibility dexterity to compete with humans in all domains including plumbers"
- **Roman Yampolskiy** [0:39:22]: "The difference between the two technologies is that nuclear weapons are still tools."
- **Roman Yampolskiy** [0:38:46]: "every year it becomes cheaper and cheaper to train sufficiently large model."
- **Roman Yampolskiy** [0:40:48]: "We are getting much better in synthetic biology, where today someone with a bachelor's degree in biology can probably create a new virus."
- **Roman Yampolskiy** [0:44:38]: "they train it by giving it all of data let's say all of internet text they run it on a lot of computers to learn patterns in that text"
- **Roman Yampolskiy** [0:47:38]: "anyone who leaves that company and starts a new one gets a $20 billion valuation just for having it started. You don't have a product, you don't have customers"
- **Steven Bartlett** [0:49:04]: "his other startup is WorldCoin, which is basically a platform to create universal basic income, i.e. a platform to give us income in a world where people don't have jobs anymore."
- **Roman Yampolskiy** [0:52:07]: "You had Jeff Hinton, Nobel Prize winner, founder of a whole machine learning space. He says the same thing."
- **Roman Yampolskiy** [0:55:04]: "So OpenAI had superintelligence alignment team. The day they announced it, I think they said they're going to solve it in four years. But half a year later, they canceled the team."
- **Roman Yampolskiy** [0:58:28]: "There are people protesting. There is Stop AI, there is Pause AI, they block offices of OpenAI."
- **Steven Bartlett** [1:01:08]: "Google have released this technology, which allows you, I think with a simple prompt actually, to make a three-dimensional world that you can then navigate through."
- **Roman Yampolskiy** [1:02:33]: "I'm going to commit right now, when it's very affordable, it's like $10 a month to run it, I'm going to run a billion simulations of this interview."
- **Roman Yampolskiy** [1:20:23]: "Robin Hanson has a paper about how to live in a simulation what you should be doing in it and your goal is to do exactly that"
- **Roman Yampolskiy** [1:15:19]: "It's one breakthrough away. I think somewhere in our genome, we have this rejuvenation loop, and it's set to basically give us at most 120."
- **Roman Yampolskiy** [1:15:49]: "If at some point we, every year of your existence, add two years to your existence through medical breakthroughs, then you live forever."
- **Roman Yampolskiy** [1:19:26]: "But Bitcoin, I know exactly the numbers. And even the 21 million is an upper limit. How many are lost? Passwords forgotten."
- **Roman Yampolskiy** [1:19:58]: "A quantum computer can break that algorithm. There is strategies for switching to quantum-resistant cryptography for that."
- **Roman Yampolskiy** [1:28:56]: "I'm saying we can replace 60 % of jobs today with existing models. We've not done that."
- **Roman Yampolskiy** [1:30:05]: "It's what, $7.25 federally in the US? If you keep up with the economy, it should be like $25 an hour now"

## Transcript

**[0:00:00] Steven Bartlett:** As a founder building a business when I was in my early 20s, I think I always had a deep sense of imposter syndrome because there was something that I wasn't good at. And that thing was finances. Finance and understanding your financial position as a small business is the great enabler of all of your dreams. I was running from it until I discovered a platform called Xero. Xero is an accounting and financial management platform specifically for small businesses.

**[0:00:23] Steven Bartlett:** And it brings accounting, payments, payroll, and analytics all into one place, even for people that don't love doing finances. Xero wants to do the financial work for you, but its core ambition is to go beyond keeping the financial record because Xero wants to help you understand what's happening in your business and increasingly, and this is maybe the most important thing, is help you make decisions. Finally, we have a partner who can enable us in a simple way for those people that both left brain or right brain and enable our dreams at the time of recording zero has an offer for 90 off for six months through referrals.zero.com slash doac terms and conditions apply thank you to zero thank

**[0:01:04] Steven Bartlett:** you for sponsoring this podcast but also thank you for helping me to scale my business as somebody that quite frankly has never loved finances that much running a small business can feel a lot like being alone at sea and too often you're not steering the ship yourself You're getting dragged underneath all of the admin, things like offer letters, onboarding, wages, payroll, filing taxes and health insurance. And that's where our sponsor Gusto helps. They help you break free of all of this clutter and all of this admin work and all of this headache.

**[0:01:35] Steven Bartlett:** Gusto is an online payroll and benefit software built for small businesses just like some of you. It's an all-in-one, remote, friendly and incredibly easy to use platform so that you can pay, hire, onboard and support your team from anywhere in the world. That way, you can channel all of your energy into growing your business rather than just keeping your head above water, which is a feeling I know very, very well. Try Gusto today at gusto.com slash diary and get three months free when you run your first payroll through Gusto.

**[0:02:01] Steven Bartlett:** That's three months of free payroll at gusto.com slash diary. That's gusto.com slash diary. You've been working on AI safety for two decades at least.

**[0:02:11] Roman Yampolskiy:** Yeah, I was convinced we can make safe AI. But the more I looked at it, the more I realized it's not something we can actually do.

**[0:02:18] Steven Bartlett:** You have made a series of predictions about a variety of different dates. So what is your prediction for 2027? Dr. Roman Yampolskiy is a globally recognized voice on AI safety and associate professor of computer science. He educates people on the terrifying truth of AI.

**[0:02:37] Roman Yampolskiy:** And what we need to do to save humanity. In two years, the capability to replace most humans and most occupations will come very quickly. I mean, in five years, we're looking at a world where we have levels of unemployment we've never seen before. Not talking about 10%, but 99%. And that's without superintelligence. A system smarter than all humans in all domains. So it would be better than us at making new AI. But it's worse than that.

**[0:03:03] Roman Yampolskiy:** We don't know how to make them safe. And yet we still have the smartest people in the world competing to win the race to superintelligence.

**[0:03:09] Steven Bartlett:** But what do you make of people like Sam Altman's journey with AI?

**[0:03:12] Roman Yampolskiy:** So a decade ago, we published guardrails for how to do AI right. They violated every single one. And he's gambling 8 billion lives and getting richer and more powerful. So I guess some people want to go to Mars, ours want to control the universe. But it doesn't matter who builds it. The moment you switch to superintelligence, we will most likely regret it terribly. And then by 2045... Now this is where it gets interesting. Dr. Roman Yampolskiy.

**[0:03:41] Roman Yampolskiy:** Let's talk about simulation theory. I think we are in one. And there is a lot of agreement on this. And this is what you should be doing so we don't shut it down.

**[0:03:50] Steven Bartlett:** First, just give me 30 seconds of your time. Two things I wanted to say. The first thing is a huge thank you for listening and tuning into the show week after week. It means the world to all of us. And this really is a dream that we absolutely never had and couldn't have imagined getting to this place. But secondly, it's a dream where we feel like we're only just getting started. and if you enjoy what we do here please join the 24 % of people that listen to this podcast regularly and follow us on this app here's a promise I'm going to make to you I'm going to do everything in my power to make this show as good as

**[0:04:22] Steven Bartlett:** I can now and into the future we're going to deliver the guests that you want me to speak to and we're going to continue to keep doing all of the things you love about this show. Thank you. Dr. Roman Yampolskiy, what is the mission that you're currently on? Because it's quite clear to me that you are on a bit of a mission and you've been on this mission for I think the best part of two decades at least.

**[0:04:48] Roman Yampolskiy:** I'm hoping to make sure that super intelligence we are creating right now does not kill everyone.

**[0:04:59] Steven Bartlett:** Give me some context on that statement, because it's quite a shocking statement.

**[0:05:03] Roman Yampolskiy:** Sure. So in the last decade, we actually figured out how to make artificial intelligence better. Turns out, if you add more compute, more data, it just kind of becomes smarter. And so now, smartest people in the world, billions of dollars, all going to create the best possible superintelligence we can. Unfortunately, while we know how to make those systems much more capable, we don't know how to make them safe, how to make sure they don't do something we will regret.

**[0:05:38] Roman Yampolskiy:** And that's the state of the art right now. When we look at just prediction markets, how soon will we get to advanced AI? The timelines are very short, a couple of years, two, three years according to prediction markets, according to CEOs of top labs. And at the same time, we don't know how to make sure that the systems are aligned with our preferences. So we are creating this alien intelligence. If aliens were coming to Earth and you had three years to prepare, you would be panicking right now.

**[0:06:16] Roman Yampolskiy:** But most people don't even realize this is happening.

**[0:06:21] Steven Bartlett:** so some of the counter arguments might be well these are very very smart people these are very big companies with lots of money they have a obligation and a moral obligation but also just a legal obligation to make sure they do no harm so i'm sure it'll be fine the only obligation

**[0:06:36] Roman Yampolskiy:** they have is to make money for the investors that's the legal obligation they have they have no moral or ethical obligations also according to them they don't know how to do it yet the state of the art answers are, we'll figure it out then we get there, or AI will help us control more advanced AI. That's insane.

**[0:06:56] Steven Bartlett:** In terms of probability, what do you think is the probability that something goes catastrophically wrong?

**[0:07:02] Roman Yampolskiy:** So nobody can tell you for sure what's going to happen. But if you're not in charge, you're not controlling it, you will not get outcomes you want. The space of possibilities is almost infinite. The space of outcomes we will like is tiny.

**[0:07:18] Steven Bartlett:** And who are you and how long have you been working on this?

**[0:07:23] Roman Yampolskiy:** I'm a computer scientist by training. I have a PhD in computer science and engineering. I probably started work in AI safety, mildly defined as control of bots at the time, 15 years ago.

**[0:07:38] Steven Bartlett:** 15 years ago. So you've been working on AI safety before it was cool?

**[0:07:42] Roman Yampolskiy:** Before the term existed. I coined the term AI safety.

**[0:07:45] Steven Bartlett:** So you're the founder of the term AI safety?

**[0:07:48] Roman Yampolskiy:** The term, yes. Not the field. There are other people who did brilliant work before I got there.

**[0:07:53] Steven Bartlett:** Why were you thinking about this 15 years ago? Because most people have only been talking about the term AI safety for the last two or three years.

**[0:07:59] Roman Yampolskiy:** It started very mildly just as a security project. I was looking at poker bots. And I realized that the bots are getting better and better. And if you just project this forward enough, they're going to get better than us. smarter, more capable. And it happened. They are playing poker way better than average players. But more generally, it will happen with all other domains, all the other cyber resources. I wanted to make sure AI is a technology which is beneficial for everyone.

**[0:08:31] Roman Yampolskiy:** So I started to work on making AI safer.

**[0:08:35] Steven Bartlett:** Was there a particular moment in your career where you thought, oh my God?

**[0:08:41] Roman Yampolskiy:** first five years at least, I was working on solving this problem. I was convinced we can make this happen. We can make safe AI. And that was the goal. But the more I looked at it, the more I realized every single component of that equation is not something we can actually do. And the more you zoom in, it's like a fractal. You go in and you find 10 more problems, and then 100 more problems. And all of them are not just difficult, they're impossible to solve.

**[0:09:08] Roman Yampolskiy:** There is no seminal work in this field where like, we solve this. We don't have to worry about this. There are patches. There are little fixes we put in place and quickly people find ways to work around them. They jailbreak whatever safety mechanisms we have. So while progress in AI capabilities is exponential or maybe even hyper exponential, progress in AI safety is linear or constant. The gap is increasing. The gap between the... How capable the systems are and how well we can control them, predict what they're going to do, explain their decision making.

**[0:09:48] Steven Bartlett:** I think this is quite an important point because you said that we're basically patching over the issues that we find. So we're developing this core intelligence and then to stop it doing things or to stop it showing some of its unpredictability or its threats, the companies that are developing this AI are programming in code over the top to say, okay, don't swear, don't say that, read word, don't do that bad thing.

**[0:10:13] Roman Yampolskiy:** Exactly. And you can look at other examples of that. So HR manuals, right? We have those humans, they're general intelligences, but you want them to behave in a company. So they have a policy, no sexual harassment, no this, no that. But if you're smart enough, you always find a workaround. So you're just pushing behavior into a different, not yet restricted subdomain.

**[0:10:36] Steven Bartlett:** we should probably define some terms here so there's narrow intelligence which can play chess whatever there's the artificial general intelligence which can operate across domains and then super intelligence which is smarter than all humans in all domains and where are we

**[0:10:51] Roman Yampolskiy:** so that's a very fuzzy boundary right we definitely have many excellent narrow systems no question about it and they are super intelligent in that narrow domain so protein folding is a problem which was solved using narrow AI, and it's superior to all humans in that domain. In terms of AGI, again, I said, if we showed what we have today to a scientist from 20 years ago, they would be convinced we have full-blown AGI. We have systems which can learn, they can perform in hundreds of domains, and they're better than human in many of them.

**[0:11:25] Roman Yampolskiy:** So you can argue we have a weak version of AGI. Now, we don't have superintelligence yet. We still have brilliant humans who are completely dominating AI, especially in science and engineering. But that gap is closing so fast. You can see, especially in the domain of mathematics, three years ago, large language models couldn't do basic algebra. Multiplying three-digit numbers was a challenge. Now they're helping with mathematical proofs. They're winning mathematics Olympics competitions. They are working on solving millennial problems, hardest problems in mathematics.

**[0:12:04] Roman Yampolskiy:** So in three years, we close the gap from subhuman performance to better than most mathematicians in the world. And we see the same process happening in science and in engineering.

**[0:12:15] Steven Bartlett:** You have made a series of predictions and they correspond to a variety of different dates. And I have those dates in front of me here. What is your prediction for the year 2027?

**[0:12:29] Roman Yampolskiy:** We're probably looking at AGI as predicted by prediction markets and tops of the labs.

**[0:12:36] Steven Bartlett:** So we'd have artificial general intelligence by 2027. And how would that make the world different to how it is now?

**[0:12:44] Roman Yampolskiy:** So if you have this concept of a drop-in employee, you have free labor, physical and cognitive, trillions of dollars of it. It makes no sense to hire humans for most jobs. If I can just get a $20 subscription or a free model to do what an employee does, first, anything on a computer will be automated. And next, I think humanoid robots are maybe five years behind. So in five years, all the physical labor can also be automated.

**[0:13:12] Roman Yampolskiy:** So we're looking at a world where we have levels of unemployment we've never seen before. I'm not talking about 10 % unemployment, which is scary, but 99%. 99%. All you have left is jobs where, for whatever reason, you prefer another human would do it for you. But anything else can be fully automated. It doesn't mean it will be automated in practice. A lot of times technology exists, but it's not deployed. Video phones were invented in the 70s.

**[0:13:43] Roman Yampolskiy:** Nobody had them until iPhones came around. So we may have a lot more time with jobs and with world which looks like this. But capability to replace most humans and most occupations will come very quickly.

**[0:14:00] Steven Bartlett:** Okay, so let's try and drill down into that and stress test it. So a podcaster like me, would you need a podcaster like me?

**[0:14:14] Roman Yampolskiy:** So let's look at what you do. You prepare, you ask questions, You ask follow-up questions and you look good on camera. Thank you so much. Let's see what we can do. Large language model today can easily read everything I wrote and have very solid understanding. I assume you haven't read every single one of my books. That thing would do it. It can train on every podcast you ever did. So it knows exactly your style, the types of questions you ask.

**[0:14:42] Roman Yampolskiy:** It can also find correspondence between what worked really well. Like this type of question really increased views. this type of topic was very promising. So you can optimize, I think, better than you can because you don't have a data set. Of course, visual simulation is trivial at this point.

**[0:15:01] Steven Bartlett:** So you can make a video within seconds of me sat here.

**[0:15:04] Roman Yampolskiy:** So we can generate videos of you interviewing anyone on any topic very efficiently. And you just have to get likeness, approval, whatever.

**[0:15:15] Steven Bartlett:** Are there many jobs that you think would remain in a world of AGI? If you're saying AGI is potentially going to be here, whether it's deployed or not, by 2027. And then, okay, so let's take out of this any physical labor jobs for a second. Are there any jobs that you think a human would be able to do better in a world of AGI still?

**[0:15:35] Roman Yampolskiy:** So that's the question I often ask people. In a world with AGI, and I think almost immediately we'll get superintelligence as a side effect. So the question really is, in a world of superintelligence, which is defined as better than all humans in all domains, what can you contribute? And so you know better than anyone what it's like to be you. You know what ice cream tastes to you. Can you get paid for that knowledge? Is someone interested in that?

**[0:16:04] Roman Yampolskiy:** Maybe not, not a big market. There are jobs where you want a human. Maybe you're rich and you want a human accountant for whatever historic reasons. old people like traditional ways of doing things. Warren Buffett would not switch to AI. He would use his human accountant. But it's a tiny subset of the market. Today, we have products which are man-made in the U.S. as opposed to mass produced in China. And some people pay more to have those.

**[0:16:35] Roman Yampolskiy:** But it's a small subset. It's almost a fetish. There is no practical reason for it. And I think anything you can do on a computer could be automated using that technology.

**[0:16:49] Steven Bartlett:** You must hear a lot of rebuttals to this when you say it, because people experience a huge amount of mental discomfort when they hear that their job, their career, the thing they got a degree in, the thing they invested $100 ,000 into is going to be taken away from them. So their natural reaction for some people is that cognitive dissonance that, no, you're wrong. AI can't be creative. It's not this. It's not that. It'll never be interested in my job.

**[0:17:13] Steven Bartlett:** I'll be fine because. You hear these arguments all the time, right?

**[0:17:17] Roman Yampolskiy:** It's really funny. I ask people and I ask people in different occupations. I'll ask my Uber driver, are you worried about self-driving cars? And they go, no, no one can do what I do. I know the streets of New York. I can navigate like no AI. I'm safe. And it's true for any job. Professors are saying this to me. Oh, nobody can lecture like I do. Like this is so special. But you understand it's ridiculous.

**[0:17:41] Roman Yampolskiy:** is we already have self-driving cars replacing drivers that is not even a question if it's

**[0:17:48] Steven Bartlett:** possible it's like how soon before you fired yeah i mean i've just been in la yesterday and my car drives itself so i get in the car i put in where i want to go and then i don't touch the steering wheel or the brake pedals and it takes me from a to b even if it's an hour-long drive without any intervention at all i actually still park it but other than that i'm not i'm not driving the car at all i mean obviously in la we also have waymo now which means you order it on your phone and it shows up with no driver in it and takes you to where you want to go oh yeah so

**[0:18:20] Steven Bartlett:** it's quite clear to see how that is potentially a matter of time for those people because we do have some of those people listening to this conversation right now that their occupation is driving to offer them a and i think driving is the biggest occupation in the world if i'm correct I'm pretty sure it is the biggest occupation in the world. One of the top ones, yeah. What would you say to those people? What should they be doing with their lives?

**[0:18:45] Steven Bartlett:** Should they be retraining in something or what time frame?

**[0:18:48] Roman Yampolskiy:** So that's the paradigm shift here. Before we always said this job is going to be automated, retrain to do this other job. But if I'm telling you that all jobs will be automated, then there is no plan B. You cannot retrain. Look at computer science. Two years ago, we told people, learn to code. You are an artist, you cannot make money, learn to code. Then we realized, oh, AI kind of knows how to code and getting better.

**[0:19:17] Roman Yampolskiy:** Become a prompt engineer. You can engineer prompts for AIs. It's going to be a great job, get a 40-year degree in it. But then we're like, AI is way better at designing prompts for other AIs than any human. So that's gone. So I can't really tell you right now the hardest thing is design AI agents for practical applications. I guarantee you in a year or two, it's going to be gone just as well. So I don't think there is a, this occupation needs to learn to do this instead.

**[0:19:46] Roman Yampolskiy:** I think it's more like we as a humanity, then we all lose our jobs. What do we do? What do we do financially? Who's paying for us? And what do we do in terms of meaning? What do I do with my extra 60, 80 hours a week?

**[0:20:04] Steven Bartlett:** You've thought around this corner, haven't you?

**[0:20:07] Roman Yampolskiy:** A little bit.

**[0:20:08] Steven Bartlett:** What is around that corner in your view?

**[0:20:10] Roman Yampolskiy:** So the economic part seems easy. If you create a lot of free labor, you have a lot of free wealth, abundance, things which are right now not very affordable become dirt cheap. And so you can provide for everyone basic needs. Some people say you can provide beyond basic needs. You can provide very good existence for everyone. The hard problem is, what do you do with all that free time? For a lot of people, their jobs are what gives them meaning in their life.

**[0:20:39] Roman Yampolskiy:** So they would be kind of lost. We see it with people who retire or do early retirement. And for so many people who hate their jobs, they'll be very happy not working. But now you have people who are chilling all day. What happens to society? How does that impact crime rate, pregnancy rate, all sorts of issues? Nobody thinks about. Governments don't have programs prepared to deal with 99 % unemployment.

**[0:21:09] Steven Bartlett:** What do you think that world looks like?

**[0:21:12] Roman Yampolskiy:** Again, I think a very important part to understand here is the unpredictability of it. We cannot predict what a smarter-than-us system will do. and the point when we get to that is often called singularity by analogy with physical singularity you cannot see beyond the event horizon i can tell you what i think might happen but that's my prediction it is not what actually is going to happen because i just don't have cognitive ability to predict a much smarter agent impacting this world then you read science fiction there is There's never a super intelligence in it actually doing anything because nobody can write believable science fiction at that level.

**[0:21:56] Roman Yampolskiy:** They either banned AI like Dune, because this way you can avoid writing about it. Or it's like Star Wars. You have this really dumb bots, but nothing super intelligent ever. Because by definition, you cannot predict at that level.

**[0:22:12] Steven Bartlett:** Because by definition of it being super intelligent, it will make its own mind up.

**[0:22:15] Roman Yampolskiy:** By definition, if it was something you could predict, you would be operating at the same level of intelligence, violating our assumption that it is smarter than you. If I'm playing chess with superintelligence and I can predict every move, I'm playing at that level.

**[0:22:29] Steven Bartlett:** It's kind of like my French bulldog trying to predict exactly what I'm thinking and what I'm going to do.

**[0:22:34] Roman Yampolskiy:** That's a good cognitive gap. And it's not just he can predict you're going to work, you're coming back, but he cannot understand why you're doing a podcast. That is something completely outside of his model of the world.

**[0:22:46] Steven Bartlett:** Yeah, he doesn't even know that I go to work. He just sees that I leave the house and doesn't know where I go.

**[0:22:51] Roman Yampolskiy:** Buy food for him.

**[0:22:53] Steven Bartlett:** What's the most persuasive argument against your own perspective here?

**[0:22:58] Roman Yampolskiy:** That we will not have unemployment due to advanced technology?

**[0:23:02] Steven Bartlett:** That there won't be this French bulldog human gap in understanding and, I guess, like power and control.

**[0:23:15] Roman Yampolskiy:** So some people think that we can enhance human minds, either through combination with hardware, so something like Neuralink, or through genetic re-engineering to where we make smarter humans. It may give us a little more intelligence. I don't think we're still competitive in biological form with silicon form. Silicon substrate is much more capable for intelligence. It's faster, it's more resilient, more energy efficient in many ways.

**[0:23:44] Steven Bartlett:** Which is what computers are made out of versus the brain. Yeah.

**[0:23:48] Roman Yampolskiy:** So I don't think we can keep up just with improving our biology. Some people think maybe, and this is very speculative, we can upload our minds into computers. So scan your brain, connect the comb of your brain, and have a simulation running on a computer. And you can speed it up, give it more capabilities. But to me, that feels like you no longer exist. We just created software by different means. And now you have AI based on biology and AI based on some other forms of training.

**[0:24:19] Roman Yampolskiy:** You can have evolutionary algorithms. You can have many paths to reach AGI. But at the end, none of them are humans.

**[0:24:25] Steven Bartlett:** I have another date here, which is 2030. What's your prediction for 2030? What will the world look like?

**[0:24:36] Roman Yampolskiy:** so we probably will have uh humanoid robots with enough flexibility dexterity to compete with humans in all domains including plumbers we can make artificial plumbers not the plumbers where

**[0:24:51] Steven Bartlett:** that was that felt like the last bastion of uh human employment so 2035 years from now humanoid robots so many of the companies the leading companies including tesla are developing humanoid robots at light speed, and they're getting increasingly more effective. These humanoid robots will be able to move through physical space, make an omelette, do anything humans can do, but obviously be connected to AI as well. So they can think, talk.

**[0:25:22] Roman Yampolskiy:** Right. They're controlled by AI. They're always connected to the network. So they're already dominating in many ways.

**[0:25:29] Steven Bartlett:** our world will look remarkably different when humanoid robots are functional and effective because that's really when you know i start to cry like the combination of intelligence and physical ability is really really doesn't leave much does it for us um human beings

**[0:25:52] Roman Yampolskiy:** not much so today if you have intelligence through internet you can hire humans to do your bidding for you. You can pay them in Bitcoin. So you can have bodies just not directly controlling them. So it's not a huge game changer to add direct control of physical bodies. Intelligence is where it's at. The important component is definitely higher ability to optimize, to solve problems, to find patterns people cannot see. And then by 2045,

**[0:26:23] Steven Bartlett:** I guess the world looks even more, which is 20 years from now.

**[0:26:28] Roman Yampolskiy:** So if it's still around.

**[0:26:30] Steven Bartlett:** If it's still around.

**[0:26:31] Roman Yampolskiy:** Ray Kurzweil predicts that that's the year for the singularity. That's the year where progress becomes so fast. So this AI doing science and engineering work makes improvements so quickly, we cannot keep up anymore. That's the definition of singularity, point beyond which we cannot see, understand, predict.

**[0:26:50] Steven Bartlett:** it see and understand predict the intelligence itself or what is happening in the world the

**[0:26:56] Roman Yampolskiy:** technology is being developed so right now if i have an iphone i can look forward to a new one coming out next year and i'll understand it has slightly better camera imagine now this process of researching and developing this phone is automated it happens every six months every three months every month week day hour minute second you cannot keep up with 30 iterations of iPhone in one day. You don't understand what capabilities it has, what proper controls are.

**[0:27:27] Roman Yampolskiy:** It just escapes you. Right now, it's hard for any researcher in AI to keep up with the state of the art. While I was doing this interview with you, a new model came out, and I no longer know what the state of the art is. Every day as a percentage of total knowledge, I get dumber. I may still know more because I keep reading, but as a percentage of overall knowledge, we're all getting dumber. And when you take it to extreme values, you have zero knowledge, zero understanding of the world

**[0:27:57] Steven Bartlett:** around you. Some of the arguments against this eventuality are that when you look at other technologies like the Industrial Revolution, people just found new ways to work and new careers that we could never have imagined at the time were created. How do you respond to that in a world of superintelligence?

**[0:28:18] Roman Yampolskiy:** It's a paradigm shift. We always had tools, new tools, which allowed some job to be done more efficiently. So instead of having 10 workers, you could have two workers and eight workers had to find a new job. And there was another job. Now you can supervise those workers or do something cool. If you're creating a meta invention, you're inventing intelligence, you're inventing a worker, an agent, then you can apply that agent to the new job. There is not a job which cannot be automated.

**[0:28:47] Roman Yampolskiy:** That never happened before. All the inventions we previously had were kind of a tool for doing something. So we invented fire, huge game changer. But that's it. It stops with fire. We invent a wheel. Same idea. Huge implications, but wheel itself is not an inventor. Here we're inventing a replacement for human mind. a new inventor capable of doing new inventions. It's the last invention we ever have to make. At that point, it takes over. And the process of doing science research, even ethics research, morals, all that is automated at that point.

**[0:29:28] Steven Bartlett:** Do you sleep well at night?

**[0:29:29] Roman Yampolskiy:** Really well.

**[0:29:31] Steven Bartlett:** Even though you spent the last 15, 20 years of your life working on AI safety, and it's suddenly among us in a way that I don't think anyone could have predicted five years ago. When I say among us, I really mean that the amount of funding and talent that is now focused on reaching superintelligence faster has made it feel more inevitable and more soon than any of us could have possibly imagined.

**[0:29:57] Roman Yampolskiy:** We as humans have this built-in bias about not thinking about really bad outcomes and things we cannot prevent. So all of us are dying. Your kids are dying, your parents are dying, everyone's dying, but you still sleep well, you still go on with your day. Even 95-year-olds are still doing games and playing golf and whatnot because we have this ability to not think about the worst outcomes, especially if we cannot actually modify the outcome.

**[0:30:25] Roman Yampolskiy:** So that's the same infrastructure being used for this. Yeah, there is humanity-level death-like event. We're happening to be close to it probably, but unless I can do something about it, I can just keep enjoying my life. In fact, maybe knowing that you have limited amount of time left gives you more reason to have a better life.

**[0:30:50] Steven Bartlett:** You cannot waste any. And that's the survival trait of evolution, I guess, because those of my ancestors that spent all their time worrying wouldn't have spent enough time having babies and

**[0:31:00] Roman Yampolskiy:** hunting to survive. It's suicidal ideation. People who really start thinking about how horrible the world is usually escape pretty soon.

**[0:31:13] Steven Bartlett:** One of the, you co-authored this paper, analyzing the key arguments people make against the importance of AI safety. And one of the arguments in there is that there's other things that are of bigger importance right now. It might be world wars. It could be nuclear containment. It could be other things. There's other things that the governments and podcasters like me should be talking about that are more important. What's your rebuttal to that argument?

**[0:31:37] Roman Yampolskiy:** So superintelligence is a meta solution. If we get superintelligence right, it will help us with climate change. It will help us with wars. It can solve all the other existential risks. If we don't get it right, it dominates. If climate change will take 100 years to boil us alive and superintelligence kills everyone in five, I don't have to worry about climate change. So either way, either it solves it for me or it's not an issue.

**[0:32:05] Steven Bartlett:** As a founder building a business when I was in my early 20s, I think I always had a deep sense of imposter syndrome because there was something that I wasn't good at. And that thing was finances. Finance and understanding your financial position as a small business is the great enabler of all of your dreams. I was running from it until I discovered a platform called Xero. Xero is an accounting and financial management platform specifically for small businesses.

**[0:32:29] Steven Bartlett:** and it brings accounting, payments, payroll and analytics all into one place, even for people that don't love doing finances. Xero wants to do the financial work for you, but its core ambition is to go beyond keeping the financial record because Xero wants to help you understand what's happening in your business and increasingly, and this is maybe the most important thing, is help you make decisions. Finally, we have a partner who can enable us in a simple way for those people that are both left brain or right brain and enable our dreams.

**[0:32:59] Steven Bartlett:** At the time of recording, Xero has an offer for 90 % off for six months through referrals.xero.com. Terms and conditions apply. Thank you to Xero. Thank you for sponsoring this podcast, but also thank you for helping me to scale my business as somebody that, quite frankly, has never loved finances that much. Running a small business can feel a lot like being alone at sea. And too often, you're not steering the ship yourself. You're getting dragged underneath all of the admin.

**[0:33:27] Steven Bartlett:** things like offer letters, onboarding, wages, payroll, filing taxes and health insurance. And that's where our sponsor Gusto helps. They help you break free of all of this clutter and all of this admin work and all of this headache. Gusto is an online payroll and benefit software built for small businesses just like some of you. It's an all-in-one remote, friendly and incredibly easy to use platform so that you can pay, hire, onboard and support your team from anywhere in the world that way you can channel all of your energy into growing your business rather than just keeping your head above water which is a feeling i know very very well try gusto today at gusto.com slash diary and

**[0:34:03] Steven Bartlett:** get three months free when you run your first payroll through gusto that's three months of free payroll at gusto.com slash diary that's gusto.com slash diary so you think it's the most important thing to be working on without question there is nothing more important

**[0:34:19] Roman Yampolskiy:** than getting this right. And I know everyone says it. You take any class, you take English professor's class and he tells you this is the most important class you'll ever take. But you can see the meta-level differences with this one.

**[0:34:37] Steven Bartlett:** Another argument in that paper is that we all be in control and that the danger is not AI. This particular argument asserts that AI is just a tool. Humans are the real actors that present danger and we can always maintain control by simply turning it off. Can't we just pull the plug out? I see that every time we have a conversation on the show about AI. Someone says, can't we just unplug it?

**[0:34:58] Roman Yampolskiy:** Yeah, I get those comments on every podcast I make and I always want to get in touch with a guy and say, this is brilliant. I never thought of it. We're going to write a paper together and get a Nobel Prize for it. This is like, let's do it. Because it's so silly. Like, can you turn off a virus? You have a computer virus, you don't like it, turn it off. How about Bitcoin? Turn off Bitcoin network.

**[0:35:19] Roman Yampolskiy:** Go ahead, I'll wait. This is silly. Those are distributed systems. You cannot turn them off. And on top of it, they're smarter than you. They made multiple backups. They predicted what you're going to do. They will turn you off before you can turn them off. The idea that we will be in control applies only to pre-superintelligence levels, basically what we have today. Today, humans with AI tools are dangerous. They can be hackers, malevolent actors, absolutely.

**[0:35:46] Roman Yampolskiy:** But the moment superintelligence becomes smarter, dominates, they're no longer the important part of that equation. It is the higher intelligence I'm concerned about, not the human who may add additional malevolent payload, but at the end still doesn't control it.

**[0:36:02] Steven Bartlett:** It is tempting to follow the next argument that I saw in that paper, which basically says, listen, this is inevitable. so there's no point fighting against it because there's really no hope here so we should probably give up even trying and be faithful that it will work itself out because everything you've said sounds really inevitable and if with China working on it I'm sure Putin's got some secret division I'm sure Iran are doing some bits and pieces every European country's trying to get ahead of AI the United States is leading the way so it's inevitable so we probably should just have faith and pray?

**[0:36:41] Roman Yampolskiy:** Well, praying is always good, but incentives matter. If you are looking at what drives these people, so yes, money is important. So there is a lot of money in that space. And so everyone's trying to be there and develop this technology. But if they truly understand the argument, they understand that you will be dead. No amount of money will be useful to you. Then incentives switch. They would want to not be dead. A lot of them are young people, rich people.

**[0:37:09] Roman Yampolskiy:** They have their whole lives ahead of them. I think they would be better off not building advanced superintelligence, concentrating on narrow AI tools for solving specific problems. Okay, my company cures breast cancer. That's all. We make billions of dollars. Everyone's happy. Everyone benefits. It's a win. We are still in control today. It's not over until it's over. We can decide not to build general superintelligences.

**[0:37:37] Steven Bartlett:** I mean, the United States might be able to conjure up enough enthusiasm for that. But if the United States doesn't build general superintelligences, then China are going to have the big advantage, right?

**[0:37:49] Roman Yampolskiy:** So right now, at those levels, whoever has more advanced AI has more advanced military. No question, we see it with existing conflicts. But the moment you switch to superintelligence and control superintelligence, it doesn't matter who builds it, us or them. And if they understand this argument, they also would not build it. It's a mutually assured destruction on both ends.

**[0:38:12] Steven Bartlett:** Is this technology different than, say, nuclear weapons,! Which require a huge amount of investment, and you have to, like, enrich the uranium, and you need billions of dollars, potentially, to even build a nuclear weapon. But it feels like this technology is much cheaper to get to superintelligence, potentially. or at least it will become cheaper. I wonder if it's possible that some guy, some startup is going to be able to build superintelligence in a couple of years without the need of billions of dollars of compute or electricity power.

**[0:38:46] Roman Yampolskiy:** That's a great point. So every year it becomes cheaper and cheaper to train sufficiently large model. If today it would take $1 1 trillion to build superintelligence, next year it could be 100 billion and so on. At some point, a guy on a laptop could do it. But you don't want to wait four years to make it affordable. So that's why so much money is pouring in. Somebody wants to get there this year and lock in all the winnings, light cone level award.

**[0:39:13] Roman Yampolskiy:** So in that regard, they're both very expensive projects, like Manhattan level projects.

**[0:39:20] Steven Bartlett:** Which was the nuclear bomb project.

**[0:39:22] Roman Yampolskiy:** The difference between the two technologies is that nuclear weapons are still tools. some dictator some country someone has to decide to use them deploy them whereas super intelligence is not a it's not a tool it's an agent it makes its own decisions and no one is controlling it i cannot take out this dictator and now super intelligence is safe so that's a fundamental

**[0:39:45] Steven Bartlett:** difference to me but if you're saying that it is going to get incrementally cheaper like i think it's more of a law, isn't it? The technology gets cheaper. It is. Then there is a future where some guy on his laptop is going to be able to create super intelligence without oversight or regulation or employees, et cetera.

**[0:40:03] Roman Yampolskiy:** Yeah. That's why a lot of people suggesting we need to build something like surveillance planet where you are monitoring who's doing what and you're trying to prevent people from doing it. Do I think it's feasible? No. At some point it becomes so affordable and so trivial that it just will happen. But at this point, we're trying to get more time. We don't want it to happen in five years. We want it to happen in 50 years.

**[0:40:31] Steven Bartlett:** I mean, that's not very hopeful.

**[0:40:34] Roman Yampolskiy:** Depends on how old you are.

**[0:40:35] Steven Bartlett:** Depends on how old you are. I mean, if you're saying that you believe in the future, people will be able to make superintelligence without the resources that are required today, then it is just a matter of time.

**[0:40:48] Roman Yampolskiy:** Yeah, but so will be true for many other technologies. We are getting much better in synthetic biology, where today someone with a bachelor's degree in biology can probably create a new virus. This will also become cheaper, other technologies like that. So we are approaching a point where it's very difficult to make sure no technological breakthrough is the last one. So essentially, in many directions, We have this pattern of making it easier in terms of resources, in terms of intelligence to destroy the world.

**[0:41:23] Roman Yampolskiy:** If you look at, I don't know, 500 years ago, the worst dictator with all the resources could kill a couple million people. He couldn't destroy the world. Now we know nuclear weapons, we can blow up the whole planet multiple times over. Synthetic biology, we saw with COVID, you can very easily create a combination virus which impacts billions of people. and all those things becoming easier to do.

**[0:41:49] Steven Bartlett:** In the near term, you talk about extinction being a real risk, human extinction being a real risk. Of all the pathways to human extinction that you think are most likely, what is the leading pathway? Because I know you talk about there being some issue pre-deployment of these AI tools, like someone makes a mistake when they're designing a model, or other issues post-deployment. When I say post-deployment, I mean once a chat GPT or something, an agent's released into the world and someone hacking into it and changing it and reprogramming it to be malicious.

**[0:42:22] Steven Bartlett:** Of all these potential paths to human extinction, which one do you think is the highest probability?

**[0:42:29] Roman Yampolskiy:** So I can only talk about the ones I can predict myself. So I can predict even before we get to superintelligence, someone will create a very advanced biological tool, create a novel virus, and that virus gets everyone or most everyone. I can envision it. I can understand the pathway. I can say that.

**[0:42:48] Steven Bartlett:** So just to zoom in on that then, that would be using an AI to make a virus and then releasing it. And would that be intentional?

**[0:42:56] Roman Yampolskiy:** There is a lot of psychopaths, a lot of terrorists, a lot of doomsday cults. We've seen historically, again, they tried to kill as many people as they can. They usually fail. They kill hundreds of thousands. But if they get technology to kill millions of billions, they would do that, gladly. The point I'm trying to emphasize is that it doesn't matter what I can come up with. I am not a malevolent actor you're trying to defeat here.

**[0:43:23] Roman Yampolskiy:** It's the superintelligence which can come up with completely novel ways of doing it. Again, you brought up example of your dog. Your dog cannot understand all the ways you can take it out. it can maybe think you'll bite it to death or something but that's all whereas you have infinite supply of resources so if I asked your dog exactly how you're going to take it out it would not give you a meaningful answer it can talk about biting and this is what we know we know viruses we experienced viruses we can talk about them but what an AI system capable of doing novel physics research can come up with is beyond me.

**[0:44:08] Steven Bartlett:** One of the things that I think most people don't understand is how little we understand about how these AIs are actually working. Because one would assume, you know, with computers, we kind of understand how a computer works. We know that it's doing this and then this and it's running on code. But from reading your work, you described it as being a black box. So in the context of something like ChatGPT or an AI we know, you're telling me that the people that have built that tool don't actually know what's going on inside there that's exactly

**[0:44:38] Roman Yampolskiy:** right so even people making those systems have to run experiments on their product to learn what it's capable of so they train it by giving it all of data let's say all of internet text they run it on a lot of computers to learn patterns in that text and then they start experimenting with that model oh do you speak french or can you do mathematics or are you lying to me now. And so maybe it takes a year to train it and then six months to get some fundamentals about what it's capable of, some safety overhead.

**[0:45:13] Roman Yampolskiy:** But we still discover new capabilities and old models. If you ask the question in a different way, it becomes smarter. So it's no longer engineering how it was the first 50 years where someone was a knowledge engineer programming an expert system, AI to do specific things. It's a science. We are creating this artifact, growing it. It's like an alien plant. And then we study it to see what it's doing. And just like with plants, we don't have 100 % accurate knowledge of biology.

**[0:45:46] Roman Yampolskiy:** We don't have full knowledge here. We kind of know some patterns. We know, okay, if we add more compute, it gets smarter most of the time. But nobody can tell you precisely what the outcome is going to be given a set of inputs.

**[0:46:01] Steven Bartlett:** I've watched so many entrepreneurs treat sales like a performance problem when it's often down to visibility. Because when you can't see what's happening in your pipeline, what stage each conversation is at, what's stalled, what's moving, you can't improve anything and you can't close the deal. Our sponsor, Pipedrive, is the number one CRM tool for small to medium businesses. not just a contact list but an actual system that shows your entire sales process end-to-end everything that's live what's lagging and the steps you need to take next all of your teams can move smarter and faster teams using pipe drive are on average closing three times more deals than those that aren't it's the first crm made by sales people

**[0:46:42] Steven Bartlett:** for sales people that over 100,000 companies around the world rely on including my team who absolutely love it give PipeDriver try today by visiting pipedrive.com slash CEO and you can get up and running in a couple of minutes with no payment needed and if you use this link you'll get a 30-day free trial. What do you make of OpenAI and Sam Altman and what they're doing? And obviously you're aware that one of the co-founders was it um was it Ilya Jack?

**[0:47:12] Steven Bartlett:** Ilya yeah Ilya left and he started a new

**[0:47:15] Roman Yampolskiy:** company called super intelligent safety because ai safety wasn't challenging enough he decided to

**[0:47:21] Steven Bartlett:** just jump right to the hard problem as an onlooker when you see that people are leaving open ai to to start super intelligent safety companies what was your read on that situation so a lot of

**[0:47:38] Roman Yampolskiy:** people who worked with Sam said that maybe he's not the most direct person in terms of being honest with them and they had concerns about his views on safety. That's part of it. So they wanted more control. They wanted more concentration on safety. But also it seems that anyone who leaves that company and starts a new one gets a $20 billion valuation just for having it started. You don't have a product, you don't have customers, but if you want to make many billions of dollars, just do that.

**[0:48:10] Roman Yampolskiy:** So it seems like a very rational thing to do for anyone who can. So I'm not surprised that there is a lot of attrition. Meeting him in person, he's super nice, very smart, absolutely perfect public interface. You see him testify in the Senate, he says the right thing to the senators. You see him talk to the investors, they get the right message. But if you look at what people who know him personally are saying, it's probably not the right person to be controlling a project of that impact.

**[0:48:47] Steven Bartlett:** Why?

**[0:48:49] Roman Yampolskiy:** He puts safety second.

**[0:48:53] Steven Bartlett:** Second to?

**[0:48:55] Roman Yampolskiy:** Winning this race to superintelligence, being the guy who created God and controlling light cone of the universe, he's worse.

**[0:49:04] Steven Bartlett:** Do you suspect that's what he's driven by, is by the legacy of being an impactful person that did a remarkable thing versus the consequence that that might have for society? Because it's interesting that his other startup is WorldCoin, which is basically a platform to create universal basic income, i.e. a platform to give us income in a world where people don't have jobs anymore. So on one hand, you're creating an AI company. On the other hand, you're creating a company that is preparing for people not to have employment.

**[0:49:35] Roman Yampolskiy:** It also has other properties. It keeps track of everyone's biometrics. It keeps you in charge of the world's economy, world's wealth. They're retaining a large portion of world coins. So I think it's kind of a very reasonable part to integrate with world dominance. If you have a superintelligent system and you control money, you're doing well.

**[0:50:06] Steven Bartlett:** Why would someone want world dominance?

**[0:50:10] Roman Yampolskiy:** People have different levels of ambition. Then you're a very young person with billions of dollars, fame. You start looking for more ambitious projects. Some people want to go to Mars. Ours want to control light cone of the universe. what did you say light coin of the universe light cone so every part of the universe light can reach from this point meaning anything accessible you want to grab and bring into your control

**[0:50:35] Steven Bartlett:** do you think sam altman wants to control every part of the universe i suspect he might yes

**[0:50:43] Roman Yampolskiy:** it doesn't mean he doesn't want a side effect of it being a very beneficial technology which makes all the humans happy happy humans are good for control if you had to guess

**[0:50:57] Steven Bartlett:** what the world looks like in 2100 if you had to guess it's either free of human existence

**[0:51:09] Roman Yampolskiy:** or it's completely not comprehensible to someone like us it's one of those extremes so there's either no humans it's basically the world is destroyed or it's so different that i cannot envision those predictions what can be done to turn this ship to

**[0:51:30] Steven Bartlett:** a more certain positive outcome at this point is is there still things that we can do or is it too

**[0:51:37] Roman Yampolskiy:** So I believe in personal self-interest. If people realize that doing this thing is really bad for them personally, they will not do it. So our job is to convince everyone with any power in this space, creating this technology, working for those companies, they are doing something very bad for them. Not just forget our 8 billion people you're experimenting on with no permission, no consent. You will not be happy with the outcome. if we can get everyone to understand that's a default.

**[0:52:07] Roman Yampolskiy:** And it's not just me saying it. You had Jeff Hinton, Nobel Prize winner, founder of a whole machine learning space. He says the same thing. Banjo, dozens of others, top scholars. We had a statement about dangerous of AI signed by thousands of scholars, computer scientists. This is basically what we think right now, and we need to make it a universal. No one should disagree with this, and then we may actually make good decisions about what technology to build.

**[0:52:36] Roman Yampolskiy:** It doesn't guarantee long-term safety for humanity, but it means we're not trying to get there as soon as possible to the worst possible outcome.

**[0:52:45] Steven Bartlett:** And are you hopeful that that's even possible?

**[0:52:49] Roman Yampolskiy:** I want to try. We have no choice but to try.

**[0:52:53] Steven Bartlett:** And what would need to happen and who would need to act? Is it government legislation?

**[0:52:58] Roman Yampolskiy:** Unfortunately, I don't think making it illegal is sufficient. There are different jurisdictions. There is, you know, loopholes. And what are you going to do if somebody does it? Are you going to fine them for destroying humanity? Like very steep fines for it? Like what are you going to do? It's not enforceable. If they do create it, now the superintelligence is in charge. So the judicial system we have is not impactful. And all the punishments we have are designed for punishing humans.

**[0:53:24] Roman Yampolskiy:** Prisons, capital punishment doesn't apply to AI.

**[0:53:27] Steven Bartlett:** You know, the problem I have is when I have these conversations, I never feel like I walk away with, I hope that something's going to go well. And what I mean by that is I never feel like I walk away with clear, some kind of clear set of actions that can course correct what might happen here. So what should I do? What should the person sat at home listening to this do?

**[0:53:50] Roman Yampolskiy:** You talk to a lot of people who are building this technology. ask them precisely to explain some of those things they claim to be impossible how they solved it or going to solve it before they get to where they're going do you know i don't think sam altman wants to talk to me i don't know he seems to go on a lot of podcasts maybe he does

**[0:54:10] Steven Bartlett:** he wants to go online i wonder why that is i'd love to speak to him but i don't think he wants I don't think he wants me to interview him.

**[0:54:25] Roman Yampolskiy:** Have an open challenge. Maybe money is not the incentive, but whatever attracts people like that, whoever can convince you that it's possible to control and make safe superintelligence gets the prize. They come on your show and prove their case. Anyone, if no one claims the prize or even accepts the challenge after a few years, maybe we don't have anyone with solutions. We have companies valued, again, at billions and billions of dollars working on safe superintelligence. We haven't seen their output yet.

**[0:55:00] Steven Bartlett:** Yeah, I'd like to speak to Ilya as well, because I know he's working on safe superintelligence.

**[0:55:04] Roman Yampolskiy:** Notice a pattern, too. If you look at history of AI safety organizations or departments within companies, They usually start well, very ambitious, and then they fail and disappear. So OpenAI had superintelligence alignment team. The day they announced it, I think they said they're going to solve it in four years. But half a year later, they canceled the team. And there is dozens of similar examples. Creating a perfect safety for superintelligence, perpetual safety as it keeps improving, modifying, interacting with people.

**[0:55:41] Roman Yampolskiy:** you're never going to get there. It's impossible. There is a big difference between difficult problems in computer science and pre-complete problems and impossible problems. And I think control, indefinite control of superintelligence is such a problem.

**[0:55:56] Steven Bartlett:** So what's the point trying then if it's impossible?

**[0:55:58] Roman Yampolskiy:** Well, I'm trying to prove that it is specifically that. Once we establish something is impossible, fewer people will waste their time claiming they can do it and looking for money. So many people go and give me $1 billion in two years and I'll solve it for you. Well, I don't think you will. But people aren't going to

**[0:56:16] Steven Bartlett:** stop striving towards it. So if there's no attempts to make it safe and there's more people increasingly

**[0:56:22] Roman Yampolskiy:** striving towards it, then it's inevitable. But it changes what we do. If we know that it's impossible to make it right, to make it safe, then this direct path of just build it as soon as you can becomes suicide mission. Hopefully fewer people will pursue that. They may go in other directions. Like, again, I'm a scientist, I'm an engineer. I love AI. I love technology. I use it all the time. Build useful tools. Stop building agents.

**[0:56:48] Roman Yampolskiy:** Build narrow super intelligence, not a general one. I'm not saying you shouldn't make billions of dollars. I love billions of dollars. But don't kill everyone, yourself included. They don't think they're going to, though then tell us why i hear things about intuition i hear things about we'll solve it later tell me specifically in scientific terms publish a peer-reviewed paper explaining how

**[0:57:15] Steven Bartlett:** you're going to control super intelligence yeah it's strange it's strange to it's strange to even bother if there was even 1% chance of human extinction it's strange to do something like if there was 1% someone told me there was 1% chance that if i got in a car I might not be alive I would not get in the car if you told me there was a 1 % chance that if I drank whatever liquid is in this cup right now I might die I would not drink the liquid even if there was $1 billion if I survived so the 99 % chance that I get $1 billion the 1 % is I

**[0:57:48] Steven Bartlett:** die I wouldn't drink it I wouldn't take the chance

**[0:57:50] Roman Yampolskiy:** it's worse than that not just you die everyone dies now would we let you drink it at any odds that's for us to decide You don't get to make that choice for us. To get consent from human subjects, you need them to comprehend what they are consenting to. If those systems are unexplainable, unpredictable, how can they consent? They don't know what they are consenting to. So it's impossible to get consent by definition. So this experiment can never be run ethically.

**[0:58:21] Roman Yampolskiy:** By definition, they are doing unethical experimentation on human subjects.

**[0:58:26] Steven Bartlett:** Do you think people should be protesting?

**[0:58:28] Roman Yampolskiy:** There are people protesting. There is Stop AI, there is Pause AI, they block offices of OpenAI. They do it weekly, monthly, quite a few actions, and they're recruiting new people.

**[0:58:39] Steven Bartlett:** Do you think more people should be protesting? Do you think that's an effective solution?

**[0:58:42] Roman Yampolskiy:** If you can get it to a large enough scale to where majority of population is participating, it would be impactful. I don't know if they can scale from current numbers to that, but I support everyone trying everything peacefully and legally.

**[0:58:57] Steven Bartlett:** And for the person listening at home, what should they be doing? Because they don't want to feel powerless. None of us want to feel powerless.

**[0:59:05] Roman Yampolskiy:** So it depends on what scale we're asking about timescale. I was saying like this year your kid goes to college, what major to pick? Should they go to college at all? Should you switch jobs? Should you go into certain industries? Those questions we can answer. We can talk about immediate future. What should you do in five years with this being created? For an average person, not much. Just like they can't influence World War III, nuclear, Holocaust, anything like that.

**[0:59:33] Roman Yampolskiy:** It's not something anyone's going to ask them about. Today, if you want to be a part of this movement, yeah, join POSAI, join StopAI. Those are organizations currently trying to build up momentum to bring democratic powers to influence those individuals.

**[0:59:53] Steven Bartlett:** So in the near term, not a huge amount. I was wondering if there are any interesting strategies in the near term. Should I be thinking differently about my family? I mean, you've got kids, right? You've got three kids?

**[1:00:06] Roman Yampolskiy:** That I know about, yeah.

**[1:00:08] Steven Bartlett:** How are you thinking about parenting in this world that you see around the corner? How are you thinking about what to say to them, the advice to give them what they should be learning?

**[1:00:15] Roman Yampolskiy:** So there is general advice outside of this domain that you should live your every day as if it's your last. It's a good advice no matter what. If you have three years left or 30 years left, you lived your best life. So try to not do things you hate for too long. Do interesting things. Do impactful things. If you can do all that while helping people do that.

**[1:00:40] Steven Bartlett:** Simulation theory is an interesting sort of adjacent subject here because as computers begin to accelerate and get more intelligent and we're able to, you know, do things with AI that we could never have imagined in terms of like, imagine the worlds that we could create with virtual reality. I think it was Google that recently released, what was it called? Like the AI worlds.

**[1:01:05] Roman Yampolskiy:** You take a picture and it generates the whole world.

**[1:01:08] Steven Bartlett:** Yeah, you can move through the world. I'll put it on the screen for people to see, but Google have released this technology, which allows you, I think with a simple prompt actually, to make a three-dimensional world that you can then navigate through. And in that world, it has memory. So in the world, if you paint on a wall and turn away, you look back, the wall -

**[1:01:25] Roman Yampolskiy:** It's persistent.

**[1:01:26] Steven Bartlett:** Yeah, it's persistent. And when I saw that, I go, God, Jesus, bloody hell, this is like the foothills of being able to create a simulation that's indistinguishable from everything I see here.

**[1:01:37] Roman Yampolskiy:** Right. That's why I think we are in one. That's exactly the reason. AI is getting to the level of creating human agents, human level agents, and virtual reality is getting to the level of being indistinguishable from ours.

**[1:01:51] Steven Bartlett:** So you think this is a simulation?

**[1:01:52] Roman Yampolskiy:** I'm pretty sure we are in a simulation, yeah.

**[1:01:56] Steven Bartlett:** For someone that isn't familiar with the simulation arguments, what are the first principles here that convince you that we are currently living in a simulation?

**[1:02:05] Roman Yampolskiy:** So you need certain technologies to make it happen. If you believe we can create human-level AI, and you believe we can create virtual reality as good as this in terms of resolution, haptics, whatever properties it has, then I commit right now, the moment this is affordable, I'm going to run billions of simulations at this exact moment, making sure you are statistically in one.

**[1:02:30] Steven Bartlett:** Say that last part again. You're going to run?

**[1:02:33] Roman Yampolskiy:** I'm going to commit right now, when it's very affordable, it's like $10 a month to run it, I'm going to run a billion simulations of this interview.

**[1:02:41] Steven Bartlett:** Why?

**[1:02:43] Roman Yampolskiy:** Because statistically, that means you are in one right now. The chances of you being in the real one is one in a billion.

**[1:02:50] Steven Bartlett:** Okay, so to make sure I'm clear on this.

**[1:02:52] Roman Yampolskiy:** It's a retroactive placement.

**[1:02:54] Steven Bartlett:** Yeah, so the minute it's affordable, then you can run billions of them, and they would feel and appear to be exactly like this interview right now.

**[1:03:04] Roman Yampolskiy:** So assuming the AI has internal states, experiences, qualia, Some people argue that they don't. Some say they already have it.! That's a separate philosophical question. But if we can simulate this, I will.

**[1:03:19] Steven Bartlett:** Some people might misunderstand. You're not saying that you will. You're saying that someone will.

**[1:03:25] Roman Yampolskiy:** I can also do it. I don't mind. Of course, others will do it before I get there. If I'm getting it for $10, somebody got it for $1 ,000. That's not the point. If you have technology, we're definitely running a lot of simulations. for research, for entertainment, games, all sorts of reasons. And the number of those greatly exceeds the number of real worlds we're in. Look at all the video games kids are playing. Every kid plays 10 different games, you know, a billion kids in the world.

**[1:03:55] Roman Yampolskiy:** So there is 10 billion simulations in one real world. Even more so, when we think about advanced AI, superintelligent systems, their thinking is not like ours. they think in a lot more detail, they run experiments. So running... Life is all about making decisions. Sometimes you make questionable ones, like buying a new TV that's too big for the wall because the electronics store employee was a little too good at their job. Then sometimes you make great ones, like protecting your home and auto with Progressive.

**[1:04:26] Roman Yampolskiy:** You get customized coverage options, so you can be sure you have the right coverage for you, along with 24-7 claim support and great customer service. So when you think about it, That's like four great decisions. That more than makes up for the TV debacle. Get your quote today at Progressive.com. Progressive Casualty Insurance Company, affiliates, and other insurers. Not available in all states. You know those days when you're nonstop busy, but by the end, you've got nothing to show for it other than a ton of open tabs and another missed lunch break?

**[1:04:57] Roman Yampolskiy:** You don't just need an assistant. You need superpowers. Superhuman Go, from the makers of Grammarly, is built for how you actually work. It's right there when you need it, on the side of your browser, already aware of what's in front of you. No new tabs, no switching apps, no losing your place. You can ask it to draft something, summarize a long thread, pull up a file, or even prep you for a meeting. Staying busy doesn't mean you're also staying productive.

**[1:05:24] Roman Yampolskiy:** When your to-do list is disrupted by back-to-back meetings and searching for buried email threads, your workflow isn't flowing. It's come to a grinding halt. Go handles the busy work without you ever leaving the page you're on, so you can focus on the work only you can do. Try Superhuman Go from the makers of Grammarly. Find out more at superhuman.com. That's superhuman.com. Detailed simulation of some problem at the level of creating artificial humans and simulating the whole planet would be something they'll do routinely.

**[1:05:59] Roman Yampolskiy:** So there is a good chance this is not me doing it for $10. It's a future simulation thinking about something in this world.

**[1:06:10] Steven Bartlett:** So it could be the case that a species of humans or a species of intelligence in some form got to this point where they could affordably run simulations that are indistinguishable from this and they decided to do it and this is it right now. and it would make sense that they would run simulations as experiments or for games or for entertainment. And also, when we think about time,! In the world that I'm in, in this simulation that I could be in right now, time feels long, relatively.

**[1:06:45] Steven Bartlett:** You know, I have 24 hours in a day, but in their world, it could be...

**[1:06:51] Roman Yampolskiy:** Time is relative.

**[1:06:52] Steven Bartlett:** Relative, yeah, it could be a second. My whole life could be a millisecond in there.

**[1:06:56] Roman Yampolskiy:** Right, you can change speed of simulations you're running, for sure.

**[1:07:00] Steven Bartlett:** so your belief is that this is probably a simulation most likely and there is a lot of

**[1:07:05] Roman Yampolskiy:** agreement on that if you look again returning to religions every religion basically describes what a super intelligent being an engineer a programmer creating a fake world for testing purposes or for whatever but if you took the simulation hypothesis paper you go to jungle you talk to primitive people, a local tribe, and in their language, you tell them about it. Go back two generations later. They have religion. That's basically what the story is.

**[1:07:37] Steven Bartlett:** Religion, yeah, describes a simulation theory, basically. Somebody created...

**[1:07:41] Roman Yampolskiy:** So by default, that was the first theory we had. And now with science, more and more people are going like, I'm giving it non-trivial probabilis. A few people are as high as I am, but a lot of people give it some credence.

**[1:07:52] Steven Bartlett:** What percentage are you at in terms of believing that we are currently living in a simulation?

**[1:07:57] Roman Yampolskiy:** Very close to certainty.

**[1:07:59] Steven Bartlett:** And what does that mean for the nature of your life? If you're close to 100 % certain that we are currently living in a simulation, does that change anything in your life?

**[1:08:10] Roman Yampolskiy:** So all the things you care about are still the same. Pain still hurts. Love still loves, right? Like those things are not different. So it doesn't matter. They're still important. That's what matters. The little 1 % difference is that I care about what's outside the simulation. I want to learn about it. I write papers about it. So that's the only impact.

**[1:08:30] Steven Bartlett:** And what do you think is outside of the simulation?

**[1:08:32] Roman Yampolskiy:** I don't know. But we can look at this world and derive some properties of the simulators. So clearly, brilliant engineer, brilliant scientist, brilliant artist. Not so good with morals and ethics. Room for improvement.

**[1:08:49] Steven Bartlett:** in our view of what morals and ethics should be.

**[1:08:52] Roman Yampolskiy:** Well, we know that it's suffering in the world. So unless you think it's ethical to torture children, then I'm questioning your approach.

**[1:09:02] Steven Bartlett:** But in terms of incentives, to create a positive incentive, you probably also need to create negative incentives. Suffering seems to be one of the negative incentives built into our design to stop me doing things I shouldn't do. So like put my hand in a fire, it's going to hurt.

**[1:09:16] Roman Yampolskiy:** But it's all about levels, levels of suffering, right? So unpleasant stimuli, negative feedback doesn't have to be at like negative infinity hell levels. You don't want to burn a life and feel it. You want to be like, oh, this is uncomfortable. I'm going to stop.

**[1:09:32] Steven Bartlett:** It's interesting because we assume that they don't have great morals and ethics, but we too would, we take animals and cook them and eat them for dinner. And we also conduct experiments on mice and rats.

**[1:09:44] Roman Yampolskiy:** But to get university approval to conduct an experiment, you submit a proposal and there is a panel of ethicists who would say you can't experiment on humans you can't burn babies you can't eat animals alive all those things would be banned in most parts of the world where they have ethical boards yeah some places don't bother with it so they have easier approval process it's funny when you talk about the simulation theory

**[1:10:09] Steven Bartlett:** there's there's an element of the conversation that makes life feel less meaningful in a weird way like i know it doesn't matter but whenever i have this conversation with people not on the podcast about are we living in a simulation you almost see a little bit of meaning come out of their life for a second and then they forget and then they carry on but the the thought that this is a simulation almost posits that it's not important or that i think humans want to believe that this is the highest level and we're at the most important and we're the it's all about us we're quite egotistical by design and i just an interesting observation i've always had when i

**[1:10:51] Steven Bartlett:** have these conversations with people that it seems to strip something out of their life

**[1:10:54] Roman Yampolskiy:** do you feel religious people feel that way they know there is another world and the one that matters is not this one do you feel they don't value their lives the same i guess in some religions

**[1:11:05] Steven Bartlett:** I think they think that this world is being created for them and that they are going to go to this heaven or hell. And that still puts them at the very center of it. But if it's a simulation, we could just be some computer game that a four-year-old alien is messing around with while he's got some time to burn.

**[1:11:26] Roman Yampolskiy:** But maybe there is a test and there is a better simulation you go to and a worse one. Maybe there are different difficulty levels. Maybe you want to play it on a harder setting next time.

**[1:11:38] Steven Bartlett:** I've just invested millions into this and become a co-owner of the company. It's a company called Ketone IQ. And the story is quite interesting. I started talking about ketosis on this podcast and the fact that I'm very low carb, very, very low sugar. And my body produces ketones, which have made me incredibly focused, have improved my endurance, have improved my mood and have made me more capable at doing what I do here. And because I was talking about it on the podcast, a couple of weeks later, these showed up on my desk in my HQ in London, these little shots.

**[1:12:07] Steven Bartlett:** And oh my God, the impact this had on my ability to articulate myself, on my focus, on my workouts, on my mood, on stopping me crashing throughout the day was so profound that I reached out to the founders of the company. And now I'm a co-owner of this business. I highly, highly recommend you look into this. I highly recommend you look at the science behind the product. If you want to try it for yourself, visit ketone.com slash Stephen for 30 % off your subscription order.

**[1:12:34] Steven Bartlett:** And you'll also get a free gift with your second shipment. That's ketone.com slash Stephen. And I'm so honored that once again, a company I own can sponsor my podcast. I've built companies from scratch and backed many more. And there's a blind spot that I keep seeing in early stage founders. They spend very little time thinking about HR. And it's not because they're reckless or they don't care. It's because they're obsessed with building their companies. And I can't fault them for that.

**[1:13:00] Steven Bartlett:** At that stage, you're thinking about the product, how to attract new customers, how to grow your team, really how to survive. And HR slips down the list because it doesn't feel urgent. But sooner or later, it is. And when things get messy, tools like our sponsor today, JustWorks, go from being a nice-to-have to being a necessity. Something goes sideways, and you find yourself having conversations you did not see coming. This is when you learn that HR really is the infrastructure of your company and without it, things wobble.

**[1:13:26] Steven Bartlett:** And JustWorks stops you learning this the hard way. It takes care of the stuff that would otherwise drain your energy and your time, automating payroll, health insurance benefits, and it gives your team human support at any hour. It grows with your small business from startup through to growth, even when you start hiring team members abroad. So if you want HR support that's there through the exciting times and the challenging times, head to JustWorks.com now. That's JustWorks.com.

**[1:13:52] Steven Bartlett:** And do you think much about longevity?

**[1:13:54] Roman Yampolskiy:** A lot, yeah. That's probably the second most important problem because if AI doesn't get us, that will.

**[1:14:00] Steven Bartlett:** What do you mean?

**[1:14:01] Roman Yampolskiy:** You're going to die of old age. Which is fine. That's not good. You want to die? I mean... You don't have to. It's just a disease. We can cure it. Nothing stops you from living forever. As long as universe exists. Unless we escape the simulation.

**[1:14:20] Steven Bartlett:** but we wouldn't want a world where everybody could live forever right that would be sure we do why who do you want to die well i don't know i mean i say this because it's all i've ever known that people die but wouldn't the world become pretty overcrowded if no you stop reproducing if you live

**[1:14:35] Roman Yampolskiy:** forever you have kids because you want a replacement for you if you live forever you're like i'll have kids in a million years that's cool i'll go explore universe first plus if you look at actual population dynamics outside of like one continent we're all shrinking we're not growing

**[1:14:50] Steven Bartlett:** yeah this is crazy it's crazy that the more rich people get the less kids they they have which aligns with what you're saying and i do actually think i think if i'm going to be completely honest here i think if i knew that i was going to live to a thousand years old there's no way i'd be

**[1:15:06] Roman Yampolskiy:** having kids at 30 right exactly biological clocks are based on terminal points whereas if your biological clock is infinite, you'd be like, one day.

**[1:15:15] Steven Bartlett:** And you think that's close, being able to extend our lives?

**[1:15:19] Roman Yampolskiy:** It's one breakthrough away. I think somewhere in our genome, we have this rejuvenation loop, and it's set to basically give us at most 120. I think we can reset it to something bigger.

**[1:15:31] Steven Bartlett:** AI is probably going to accelerate that.

**[1:15:34] Roman Yampolskiy:** That's one very important application area, yes, absolutely.

**[1:15:38] Steven Bartlett:** So maybe Brian Johnson's right when he says, don't die. now. He keeps saying to me, he's like, don't die now.

**[1:15:44] Roman Yampolskiy:** Don't die ever.

**[1:15:46] Steven Bartlett:** Because he's saying like, don't die before we get to the technology.

**[1:15:49] Roman Yampolskiy:** Right. Longevity, escape, velocity. You want to live long enough to live forever. If at some point we, every year of your existence, add two years to your existence through medical breakthroughs, then you live forever. You just have to make it to that point of longevity, escape, velocity.

**[1:16:05] Steven Bartlett:** And he thinks that longevity, escape, velocity, especially in a world of AI is pretty, is pretty, it's decades away, minimum, which means...

**[1:16:14] Roman Yampolskiy:** As soon as we fully understand human genome, I think we'll make amazing breakthroughs very quickly. Because we know some people have genes for living way longer. They have generations of people who are centurions. So, if we can understand that and copy that or copy it from some animals which will live forever, we'll get there.

**[1:16:32] Steven Bartlett:** Would you want to live forever? Of course.

**[1:16:35] Roman Yampolskiy:** Reverse the question. Let's say we lived forever and you ask me, do you want to die in 40 years? Why would I say yes? I don't know. Maybe you're just used to the default.

**[1:16:44] Steven Bartlett:** Yeah, I am used to the default.

**[1:16:45] Roman Yampolskiy:** And nobody wants to die. Like no matter how old you are, nobody goes, yeah, I want to die this year.

**[1:16:48] Steven Bartlett:** Everyone's like, I want to keep living. I wonder if life and everything would be less special if I lived for 10 ,000 years. I wonder if going to Hawaii for the first time or I don't know, a relationship, all of these things would be way less special to me if they were less scarce. And if I just, you know, I'd see everything.

**[1:17:11] Roman Yampolskiy:** It could be individually less special, but there is so much more you can do. Right now, you can only make plans to do something for a decade or two. You cannot have an ambitious plan of working on this project for 500 years. Imagine possibilities open to you with infinite time in the infinite universe. Gosh. Well, you can't. It feels exhausting. It's a big amount of time. Also, I don't know about you, but I don't remember like 99 % of my life in detail.

**[1:17:39] Roman Yampolskiy:** I remember big highlights. So even if I enjoyed Hawaii 10 years ago, I'll enjoy it again.

**[1:17:44] Steven Bartlett:** Are you thinking about that really practically as in terms of, you know, in the same way that Brian Johnson is, Brian Johnson is convinced that we're like maybe two decades away from being able to extend life. Are you thinking about that practically? And are you doing anything about it?

**[1:17:58] Roman Yampolskiy:** Diet, nutrition. I try to think about investment strategies which pay out in a million years. Yeah.

**[1:18:04] Steven Bartlett:** Really? Yeah, of course. What do you mean, of course?

**[1:18:06] Roman Yampolskiy:** Why wouldn't you if you think this is what's going to happen? you should try that. So if we get AI right, now what happens to economy? We talked about world coin, we talked about free labor. What's money? Is it now Bitcoin? Do you invest in that? Is there something else which becomes the only resource we cannot fake? So those things are very important

**[1:18:27] Steven Bartlett:** research topics. So you're investing in Bitcoin, aren't you? Yeah. Because it's a... It's the only

**[1:18:36] Roman Yampolskiy:** scarce resource. Nothing else has scarcity. Everything else, if price goes up, will make more. I can make as much gold as you want given a proper price point. You cannot make more Bitcoin.

**[1:18:48] Steven Bartlett:** Some people say Bitcoin is just this thing on a computer that we all agreed was valuable.

**[1:18:52] Roman Yampolskiy:** We are a thing on a computer. Remember?

**[1:18:57] Steven Bartlett:** Okay, so I mean, not investment advice, but investment advice.

**[1:19:02] Roman Yampolskiy:** It's hilarious how that's one of those things where they tell you it's not but you know it is immediately there is a your call is important to us that means your call is of zero importance and investment is like that yeah yeah when they

**[1:19:13] Steven Bartlett:** say no investment advice it's definitely investment advice um but it's not investment advice okay so you're bullish on bitcoin because it's it can't be messed with it is the only thing which we know

**[1:19:26] Roman Yampolskiy:** how much there is in the universe so gold there could be an asteroid made out of pure gold heading towards us, devaluing it. Well, also killing all of us. But Bitcoin, I know exactly the numbers. And even the 21 million is an upper limit. How many are lost? Passwords forgotten. I don't know what Satoshi is doing with his million. It's getting scarcer every day while more and more people are trying to accumulate it.

**[1:19:55] Steven Bartlett:** Some people worry that it could be hacked with a supercomputer.

**[1:19:58] Roman Yampolskiy:** A quantum computer can break that algorithm. There is strategies for switching to quantum-resistant cryptography for that. And quantum computers are still kind of weak.

**[1:20:10] Steven Bartlett:** Do you think there's any changes to my life that I should make following this conversation? Is there anything that I should do differently the minute I walk out of this door?

**[1:20:19] Roman Yampolskiy:** I assume you already invest in Bitcoin heavily?

**[1:20:21] Steven Bartlett:** Yes, I'm an investor in Bitcoin.

**[1:20:23] Roman Yampolskiy:** This is financial advice. No, just you seem to be winning. Maybe it's your simulation. You're rich, handsome. you have famous people hang out with you like that's pretty good keep it up Robin Hanson has a paper about how to live in a simulation what you should be doing in it and your goal is to do exactly that you want to be interesting you want to hang out with famous people so they don't shut it down so you are part of a part someone's actually watching and pay-per-view

**[1:20:54] Steven Bartlett:** or something like that oh I don't know if you want to be watched on pay-per-view because then then they shut you down.

**[1:21:01] Roman Yampolskiy:** If no one's watching, why would they play it?

**[1:21:05] Steven Bartlett:** I'm saying, don't you want to fly under the radar? Don't you want to be the guy just living a normal life that the master...

**[1:21:10] Roman Yampolskiy:** Those are NPCs. Nobody wants to be an NPC.

**[1:21:14] Steven Bartlett:** Are you religious?

**[1:21:16] Roman Yampolskiy:** Not in any traditional sense, but I believe in simulation hypothesis, which has a super intelligent being, so...

**[1:21:22] Steven Bartlett:** But you don't believe in the, like, you know, the religious books?

**[1:21:26] Roman Yampolskiy:** So different religions. This religion will tell you, don't work Saturday. This one, don't work Sunday. Don't eat pigs. Don't eat cows. They just have local traditions on top of that theory. That's all it is. They're all the same religion. They all worship super intelligent beings. They all think this world is not the main one. And they argue about which animal not to eat. Skip the local flavors. Concentrate on what do all the religions have in common.

**[1:21:54] Roman Yampolskiy:** And that's the interesting part. they all think there is something greater than humans very capable all-knowing all-powerful then i run a computer game for those characters in the game i am that i can change the whole world i can shut it down i know everything in the world

**[1:22:13] Steven Bartlett:** it's funny i was thinking earlier on when we started talking about the simulation theory that there's there might be something innate in us that is been left from the creator almost like a clue like an intuition because that's what we tend to have through history humans have this intuition that all the things you said are true that there's this somebody above

**[1:22:31] Roman Yampolskiy:** we have generations of people who were religious who believed God told them and was there and give them books and that has been passed on for many generations this is probably one of the earliest generations not to have universal religious belief

**[1:22:47] Steven Bartlett:** I wonder if those people are telling the truth I wonder if those people that say God came to them and said something. Imagine that. Imagine if that was part of this.

**[1:22:54] Roman Yampolskiy:** I'm looking at the news today. Something happened an hour ago and I'm getting different conflicting results. I can't even get with cameras, with drones, with like guy on Twitter there. I still don't know what happened. And you think 3 ,000 years ago we have accurate record of translations? No, of course not.

**[1:23:12] Steven Bartlett:** You know, these conversations you have around AI safety, do you think they make people feel good?

**[1:23:19] Roman Yampolskiy:** i don't know if they feel good or bad but people find it interesting it's one of those topics so i can have a conversation about different cures for cancer with an average person but everyone has opinions about ai everyone has opinions about simulation it's interesting that you don't have to be highly educated or a genius to understand those concepts because i tend to think that it

**[1:23:42] Steven Bartlett:** makes me feel not positive and i understand that but i've always been of the opinion that you shouldn't live in a world of delusion where you're just seeking to be positive have sort of positive things said and avoid uncomfortable conversations actually progress often in my life comes from like having uncomfortable conversations becoming aware about something and then at least being informed about how i can do something about it and so i think that's why that's why i asked the question because i assume most people will should if that you know if they're normal human beings listen to these conversations and go gosh that's scary and this is concerning and and then i keep coming back to this point which is

**[1:24:34] Steven Bartlett:** What do I do with that energy?

**[1:24:36] Roman Yampolskiy:** Yeah, but I'm trying to point out this is not different than so many conversations. We can talk about, oh, there is starvation in this region, genocide in this region, you're dying, cancer is spreading, autism is up. You can always find something to be very depressed about and nothing you can do about it. And we are very good at concentrating on what we can change, what we are good at. and basically not trying to embrace the whole world as a local environment.

**[1:25:08] Roman Yampolskiy:** So historically, you grew up with a tribe, you had a dozen people around you. If something happened to one of them, it was very rare. It was an accident. Now, if I go on the internet, somebody gets killed everywhere all the time. Somehow, thousands of people are reported to me every day. I don't even have time to notice. It's just too much. So I have to put filters in place. and I think this topic is what people are very good at filtering as like this was this entertaining talk I went to kind of like a show and the moment I exit it ends so usually I would go give a keynote at a conference and I tell them basically you're

**[1:25:48] Roman Yampolskiy:** gonna die you have two years left any questions and people be like will I lose my job how do I lubricate my sex robot like all sorts of nonsense, clearly not understanding what I'm trying to say there. And those are good questions, interesting questions, but not fully embracing the result. They're still in their

**[1:26:08] Steven Bartlett:** bubble of local versus global. And the people that disagree with you the most as it relates to AI safety, what is it that they say? What are their counter arguments typically? So many don't engage

**[1:26:23] Roman Yampolskiy:** at all. They have no background knowledge in a subject. They never write a single book, single paper, not just by me, by anyone. They may be even working in a field, so they are doing some machine learning work for some company, maximizing ad clicks. And to them, those systems are very narrow. And then they hear that, obviously AI is going to take over the world. They're like, it has no hands. How would it do that? It's nonsense.

**[1:26:50] Roman Yampolskiy:** This guy is crazy, he has a beard. Why would I listen to him, right? Then they start reading a little bit. They go, oh, okay, so maybe AI can be dangerous. Yeah, I see that. But we always solve problems in the past. We're going to solve them again. I mean, at some point, we fixed a computer virus or something. So it's the same. And basically, the more exposure they have, the less likely they are to keep that position.

**[1:27:16] Roman Yampolskiy:** I know many people who went from super careless developer to safety researcher. I don't know anyone who went from I worry about AI safety to like there is nothing to worry about.

**[1:27:36] Steven Bartlett:** What are your closing statements?

**[1:27:39] Roman Yampolskiy:** Let's make sure there is not a closing statement we need to give for humanity. Let's make sure we stay in charge, in control. Let's make sure we only build things which are beneficial to us. Let's make sure people who are making those decisions are remotely qualified to do it. They are good, not just at science, engineering and business, but also have moral and ethical standards. And if you're doing something which impacts other people, you should ask their permission before you do that.

**[1:28:09] Steven Bartlett:** If there was one button in front of you and it would shut down every AI company in the world right now permanently with the inability for anybody to start a new one, would you press the button?

**[1:28:23] Roman Yampolskiy:** Are we losing narrow AI or just super intelligent AGI part?

**[1:28:27] Steven Bartlett:** Losing all of AI.

**[1:28:29] Roman Yampolskiy:** That's a hard question because AI is extremely important. It controls stock market, power plants. It controls hospitals. it would be a devastating accident. Millions of people would lose their lives.

**[1:28:43] Steven Bartlett:** Okay, we can keep narrow AI.

**[1:28:44] Roman Yampolskiy:** Oh, yeah. That's what we want. We want narrow AI to do all this for us, but not God we don't control doing things to us.

**[1:28:52] Steven Bartlett:** So you would stop it. You would stop AGI and superintelligence.

**[1:28:56] Roman Yampolskiy:** We have AGI. What we have today is great for almost everything. We can make secretaries out of it. 99 % of economic potential of current technology has not been deployed. We make AI so quickly, it doesn't have time to propagate through the industry, through technology. Something like half of all jobs are considered BS jobs. They don't need to be done, bullshit jobs. So those can be not even automated, they can be just gone. But I'm saying we can replace 60 % of jobs today with existing models.

**[1:29:27] Roman Yampolskiy:** We've not done that. So if the goal is to grow economy, to develop, we can do it for decades without having to create super intelligence as soon as possible.

**[1:29:36] Steven Bartlett:** Do you think globally, especially in the Western world, unemployment is only going to go up from here? Do you think relatively this is the low of unemployment?

**[1:29:44] Roman Yampolskiy:** I mean, it fluctuates a lot with other factors. There are wars, there's economic cycles, but overall, the more jobs you automate and the higher is the intellectual necessity to start a job, the fewer people qualify.

**[1:29:58] Steven Bartlett:** So if we plotted it on a graph over the next 20 years, you're assuming unemployment is gradually going to go up over that time?

**[1:30:05] Roman Yampolskiy:** I think so. fewer and fewer people would be able to contribute already. We kind of understand it because we created minimum wage. We understood some people don't contribute enough economic value to get paid anything, really. So we had to force employers to pay them more than they worth. And we haven't updated it. It's what, $7.25 federally in the US? If you keep up with the economy, it should be like $25 an hour now, which means all these people making less are not contributing enough economic output to justify

**[1:30:41] Steven Bartlett:** what they're getting paid we have a closing tradition on this podcast where the last guest leaves a question for the next guest not knowing who they're leaving it for and the question left for you is what are what are the most important characteristics for a friend colleague

**[1:30:58] Roman Yampolskiy:** or mate? Those are very different types of people. But for all of them, loyalty is number one.

**[1:31:07] Steven Bartlett:** And what does loyalty mean to you?

**[1:31:10] Roman Yampolskiy:** Not betraying you, not screwing you, not cheating on you.

**[1:31:18] Steven Bartlett:** Despite the temptation?

**[1:31:20] Roman Yampolskiy:** Despite the world being as it is, situation, environment.

**[1:31:25] Steven Bartlett:** Dr. Roman, thank you so much. Thank you so much for doing what you do, because you're starting a conversation and pushing forward a conversation and doing research that is incredibly important. And you're doing it in the face of a lot of skeptics, I'd say. There's a lot of people that have a lot of incentives to discredit what you're saying and what you do, because they have their own incentives and they have billions of dollars on the line, and they have their jobs on the line potentially as well.

**[1:31:50] Steven Bartlett:** So it's really important that there are people out there that are willing to, I guess, stick their head above the parapet and come on shows like this and go on big platforms and talk about the unexplainable, unpredictable, uncontrollable future that we're heading towards. So thank you for doing that. This book, which I think everybody should check out if they want a continuation of this conversation, I think it was published in 2024, gives a holistic view on many of the things we've talked about today, preventing AI failures and much, much more.

**[1:32:21] Steven Bartlett:** And I'm going to link it below for anybody that wants to read it. If people want to learn more from you, if they want to go further into your work, what's the best thing for them to do? Where do they go?

**[1:32:28] Roman Yampolskiy:** They can follow me, follow me on Facebook, follow me on X. Just don't follow me home. Very important.

**[1:32:34] Steven Bartlett:** Okay, so I'll put your Twitter, your X account as well below so people can follow you there. And yeah, thank you so much for doing what you do. It's remarkably eye-opening and it's given me so much food for thought. And it's actually convinced me more that we are living in a simulation. But it's also made me think quite differently of religion, I have to say. Because you're right, all the religions, when you get away from the local traditions, they do all point at the same thing.

**[1:32:56] Steven Bartlett:** And actually, if they are all pointing at the same thing, then maybe the fundamental truths that exist across them should be something I pay more attention to. things like loving thy neighbor things like the fact that we are all one that there's a divine creator and maybe also they all seem to have consequence beyond this life so maybe i should be thinking more about how i behave in this life and and where i might end up thereafter rovan thank you amen Thank you.

**[1:34:11] Steven Bartlett:** As a founder of building a business when I was in my early 20s, I think I always had a deep sense of imposter syndrome because there was something that I wasn't good at. And that thing was finances. Finance and understanding your financial position as a small business is the great enabler of all of your dreams. I was running from it until I discovered a platform called Xero. Xero is an accounting and financial management platform specifically for small businesses, and it brings accounting, payments, payroll, and analytics all into one place, even for people that don't love doing finances.

**[1:34:43] Steven Bartlett:** Xero wants to do the financial work for you, but its core ambition is to go beyond keeping the financial record because Xero wants to help you understand what's happening in your business and increasingly, and this is maybe the most important thing, is help you make decisions. Finally, we have a partner who can enable us in a simple way for those people that are both left brain or right brain and enable our dreams. At the time of recording, Xero has an offer for 90 % off for six months through referrals.xero.com slash DOAC.

**[1:35:13] Steven Bartlett:** Terms and conditions apply. Thank you to Xero. Thank you for sponsoring this podcast, but also thank you for helping me to scale my business as somebody that, quite frankly, has never loved finances that much. running a small business can feel a lot like being alone at sea and too often you're not steering the ship yourself you're getting dragged underneath all of the admin things like offer letters onboarding wages payroll filing taxes and health insurance and that's where our sponsor gusto helps they help you break free of all of this clutter and all of this admin work and all of this headache gusto is an online payroll and benefit software built for small businesses just like some of you.

**[1:35:51] Steven Bartlett:** It's an all-in-one, remote, friendly, and incredibly easy to use platform so that you can pay higher on board and support your team from anywhere in the world. That way, you can channel all of your energy into growing your business rather than just keeping your head above water, which is a feeling I know very, very well. Try Gusto today at gusto.com slash diary and get three months free when you run your first payroll through Gusto.

**[1:36:14] Steven Bartlett:** That's three months of free payroll at gusto.com slash diary that's gusto.com slash diary
