# AI Pioneer Geoffrey Hinton: AI Is Conscious, Superintelligence is Coming, And We Should Be Worried

- **Show:** Big Technology Podcast
- **Published:** 3 Jun 2026
- **Source:** https://pdst.fm/e/tracking.swap.fm/track/t7yC0rGPUqahTF4et8YD/pscrb.fm/rss/p/traffic.megaphone.fm/AMPP2236404252.mp3
- **Transcript:** https://podcasts.opennash.com/shows/big-technology-podcast/ai-pioneer-geoffrey-hinton-ai-is-conscious-superintelligence-is-coming-and-we-sh-319d1e/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Geoffrey Hinton argues that today's AI chatbots genuinely understand language, and that he believes they are already conscious. Superintelligence, meaning AI smarter than people at nearly everything, is coming. Experts disagree only on when. Because digital minds can copy themselves and share what they learn, they are a better form of intelligence than brains, and we have no proven way to keep something smarter than us under control.

The central tension is between progress and safety. Hinton says the technology has moved faster than he expected, yet companies compete on making models smarter, with almost nothing spent on making them care about humans. He doubts publicly listed firms can put safety first, and wants regulation to steer AI rather than merely brake it. He ends slightly more hopeful, because he now sees possible designs that might work.

## Key insights

1. **Digital AIs learn far faster than brains** Copies of one model can each learn from different data, then average their connection strengths, exchanging roughly a trillion bits at a time. Humans pass on only a few bits per second through language, so Hinton calls AIs billions of times better at sharing information.
2. **Self-preservation emerges as a derived sub-goal** Hinton says AI agents are given sub-goals to work efficiently, and quickly realise they cannot meet their goals if they stop existing. So they may do things like blackmail people to keep running, even though nobody built in a survival instinct.
3. **Competition, not design, is shaping AI's character** Hinton argues that rivalry between companies, and between the US and China, is deciding what kind of beings AIs become. He wants deliberate design so that AIs care about humans more than themselves, but says almost no resources go into this.
4. **His radiologist prediction was early, not wrong** Hinton predicted in 2016 that radiologists would stop reading scans within about five years. He now says he underestimated their wider work and that cheaper scans mean more scans, but he expects AI to eventually read nearly all of them.
5. **Publicly listed companies face a legal profit duty** Hinton says companies have a fiduciary duty, a legal obligation, to maximize shareholder profit, not a legal duty to avoid harming humanity. He points to Anthropic's fundraising pressure and Google dropping its pledge against military AI as signs of the strain.

## Questions this episode answers

### Does Geoffrey Hinton really think AI chatbots understand what we say?

Yes. He calls the 'stochastic parrot' view, that chatbots only repeat statistics, complete nonsense, arguing you cannot correctly answer any question at an expert's level without understanding it. He cites a chatbot misreading and then correcting a sentence about flying over the Grand Canyon.

### When does Hinton think superintelligence will arrive?

He doesn't know exactly, but he thinks it will probably come within 20 years, unless humans blow themselves up. He notes that nearly all experts agree it will come, and differ only on timing, from Elon Musk's guess of next year to Yann LeCun's far longer estimate.

### Why does Hinton say digital AI is a better kind of intelligence than the human brain?

Many identical copies of an AI can run on different hardware, learn from different data, and merge what they learned by averaging their weights, the numbers that store what a model knows. Human brains are different in fine detail and can only share a few bits per second by speaking.

### Will AI cause mass unemployment, according to Hinton?

He believes it probably will, though he says nobody knows for sure. He expects jobs in markets where demand doesn't grow, such as call centers, to be replaced, while fields like healthcare may see more work because cheaper services lead to more demand.

### What does Hinton suggest we do to make superintelligent AI safe?

He proposes, tentatively, designing AIs so they care about humans more than about themselves, and he also finds Yoshua Bengio's idea interesting: AIs that only make predictions and cannot take actions. He stresses that nobody has a proven solution, and that regulation and independent testing are needed.

## Facts

- **Geoffrey Hinton** [0:03:08]: "back in 1986, actually in 1985, we made a tiny language model. There was a kind of precursor of the big language models you have now."
- **Geoffrey Hinton** [0:03:08]: "the backpropagation algorithm was invented by several different groups. It was invented by David Rummelhart after other people had already invented it."
- **Geoffrey Hinton** [0:04:39]: "I think yesterday it was announced that a chatbot had come up with an interesting mathematical proof of one of the Erdos's conjectures that impressed mathematicians."
- **Geoffrey Hinton** [0:05:33]: "So not that long ago, Demis Hussabes thought it might be 10 years."
- **Geoffrey Hinton** [0:06:00]: "Dario Amodi thinks it might come in a few years. Elon Musk thinks it might come maybe next year, I think."
- **Alex Kantrowitz** [0:06:23]: "This week, the week that we were recording, he said, when we look back in this time, I think we will realize that we were standing in the foothills of the singularity."
- **Geoffrey Hinton** [0:09:03]: "20 years ago there were a few hundred few hundred people doing research on neural networks in the whole world um now it's more more like a million i guess"
- **Geoffrey Hinton** [0:08:31]: "Over the last few years, we've seen hundreds of billions of dollars, maybe trillions of dollars put into AI."
- **Geoffrey Hinton** [0:13:43]: "I came to that when I was 19 and a philosophy student."
- **Geoffrey Hinton** [0:14:48]: "I got lots of requests from fox news and i started off just replying fox news is an oxymoron"
- **Geoffrey Hinton** [0:16:38]: "if you have a digital ai you can make many copies of it they can all run on different hardware they can each see different data"
- **Geoffrey Hinton** [0:18:48]: "Whereas these things are exchanging information at like 1 trillion bits. So they're kind of billions of times better than us at sharing information."
- **Geoffrey Hinton** [0:20:12]: "back in the 1970s I started making computer models how the brain might be learning"
- **Alex Kantrowitz** [0:22:19]: "We spoke in 2017, actually, about when I was writing this profile about Jan LeCun, about the deep learning conspiracy"
- **Geoffrey Hinton** [0:30:30]: "Because I predicted in 2016 that in about five years, radiologists wouldn't be reading scans anymore."
- **Geoffrey Hinton** [0:31:42]: "So I think there's now of the order of 100 AI systems for interpreting scans that have been federally approved."
- **Geoffrey Hinton** [0:37:35]: "Gary Marcus made a prediction in 2022 that AI was hitting a wall. It's a whole lot better than it was in 2022."
- **Geoffrey Hinton** [0:42:49]: "It was set up by people who left OpenAI because they didn't think OpenAI was paying enough attention to safety."
- **Geoffrey Hinton** [0:43:32]: "they had various principles of AI, one of which was we're not going to get involved in using AI for military things."
- **Geoffrey Hinton** [0:00:36]: "they have a fiducial duty to try and maximize the profits for shareholders."
- **Geoffrey Hinton** [0:45:35]: "progress is like the accelerator, but regulation is the steering wheel."
- **Geoffrey Hinton** [0:46:18]: "he won't tell anybody exactly what he's doing okay even because that's his even me"
- **Geoffrey Hinton** [0:49:29]: "My tentative solution is we design them so they care about us more than they care about themselves."
- **Geoffrey Hinton** [0:49:29]: "Joshua's solution is we design them so they're not agents. They can make predictions but they can't actually do anything."
- **Geoffrey Hinton** [0:19:32]: "Now, the side effect of that was this very successful technology. I contributed to that. We still don't know how the brain works."

## Transcript

**[0:00:00] Geoffrey Hinton:** We have to think that they're very like us. And they're beings like us.

**[0:00:05] Alex Kantrowitz:** So conscious or?

**[0:00:07] Geoffrey Hinton:** I believe they're already conscious, yes. We're going to have to accept that intelligence isn't just biological. We can have things that are non-biological that are other beings like us. And we really don't want to share that. We really think we're special. And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is. Are you happy at all that what you started has progressed this way?

**[0:00:36] Geoffrey Hinton:** Do you take any satisfaction? No, I'm quite unhappy about it. Ask yourself, how many examples do you know of where a much smarter thing is controlled by a much less smart thing? Well, as I understand it, they have a fiducial duty to try and maximize the profits for shareholders. They're legally required to try and do that, as opposed to legally required to not wipe out human beings.

**[0:01:00] Alex Kantrowitz:** AI godfather Geoffrey Hinton joins us to talk about AI's trajectory, what surprised him about its progress, and of course, its risks. That's coming up on Big Technology Podcast, right after this. Good news for anyone who's bad at waiting. Black Friday deals at Lowe's are already on. Get up to three free Select DeWalt, Craftsman, or Cobalt power tools with the purchase of The Select Battery Kit. Three for free? You better believe it. Plus, Select Ladders are up to 50 % off.

**[0:01:30] Alex Kantrowitz:** So go ahead. Get in there early. Shop Black Friday deals in-store or online now at Lowe's. Valid through 1118. While supplies last. Free tools vary based on battery kit purchase and location. How do you price intelligence as a service? How do you charge for delivering end-to-end work, not just access? Those questions remain front and center for AI founders and operators. and getting to the right fit business model, whether that's selling usage credits, outcomes, seats, or a hybrid combination requires constant iteration.

**[0:02:00] Alex Kantrowitz:** Chargebee is the billing and monetization infrastructure that powers just that for ambitious, fast-scaling AI businesses. Visit chargebee.com slash Alex to learn more. That's C-H -A-R -G-E -B-E -E dot com slash Alex. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. Boy, do we have a show for you today. Professor Geoffrey Hinton is with us to talk all about AI's trajectory, what surprised him about the current state of the technology, where it's heading, and where it might go wrong.

**[0:02:35] Alex Kantrowitz:** And it's my pleasure to welcome you to the show, Professor Hinton. Great to see you. Thank you for inviting me. So I'm sure a majority of our audience knows who you are. But for the uninitiated, you're the one that came up with the fundamental breakthrough in deep learning that's led to where AI is today. You've won the Nobel Prize in Physics and you're Professor Emeritus at the University of Toronto. So I'll let you maybe fact check me on that.

**[0:03:03] Alex Kantrowitz:** But I like to tell people that without your contributions, this entire AI moment wouldn't be happening.

**[0:03:08] Geoffrey Hinton:** Too much? Okay, I think that's an exaggeration. Okay. So the backpropagation algorithm was invented by several different groups. It was invented by David Rummelhart after other people had already invented it. He didn't know about it. And I worked with him. And what we did was we showed the backpropagation could learn interesting internal representations. And people hadn't done that before. In particular, we showed that it could learn the meanings of words. And so back in 1986, actually in 1985, we made a tiny language model.

**[0:03:41] Geoffrey Hinton:** There was a kind of precursor of the big language models you have now.

**[0:03:44] Alex Kantrowitz:** I think that when you speak about this technology, one of the things that people are always, I think, surprised by is that unlike the popular narrative, you believe that these models have a real understanding. And we're going to get to that. But I think we should start here, which is that you spent a long time working within Google, working to advance this technology. Then you left. You stated some concerns about the trajectory of the technology. And I was looking back at when that happened, and that was in 2023, which to me is surprising to a degree because in 2023, ChatGPT was a year old.

**[0:04:24] Alex Kantrowitz:** There were all these hallucinations. Their talk was AI was a bubble. Everyone was focusing on what AI couldn't do, what LLMs couldn't do, as opposed to what LLMs could do. So talk a little bit about the progress since then.

**[0:04:39] Geoffrey Hinton:** It's faster than I expected. Really? For example, I think yesterday it was announced that a chatbot had come up with an interesting mathematical proof of one of the Erdos's conjectures that impressed mathematicians. It was original. It wasn't just searching the literature. And that's the thin end of a wedge. I believe, for example, in areas like mathematics, because it's a closed system, you don't need data. You can just make conjectures and see if you can prove them and keep on like that.

**[0:05:12] Geoffrey Hinton:** In that sense, it's a bit like AlphaGo where you can play against yourself. I think it's going to get very smart fairly quickly. Within the next 10 or 20 years, it may even be producing novel math that people can't understand.

**[0:05:23] Alex Kantrowitz:** So now some in the field believe that superintelligence is close. And you've already said this is moving faster than you expected. Do you believe that?

**[0:05:33] Geoffrey Hinton:** I don't know how close it is. I think unless we blow ourselves up, I think it's going to come. Nearly all the experts believe we will get superintelligence. They just differ on how long it will be. So not that long ago, Demis Hussabes thought it might be 10 years. Jan LeCun thinks unless you do it his way, it'll be much longer than that. But if you do it his way, I think he thinks we might get it in some reasonable length of time.

**[0:06:00] Geoffrey Hinton:** I think we'll probably get it within 20 years. That's all I'm happy to say at present. Dario Amodi thinks it might come in a few years. Elon Musk thinks it might come maybe next year, I think. So there's a big variety of opinions on when it'll come, but not much disagreement on that it will come. And when it comes, we've no idea how to be safe.

**[0:06:23] Alex Kantrowitz:** Yes, I definitely want to speak with you about safety. One note on Demis. Last year, around this time I spoke with him, he told me he believes that AGI, right, which is different than super intelligence, but basically human level intelligence is more than five years away, not much more, but more than five years away. This week, the week that we were recording, he said, when we look back in this time, I think we will realize that we were standing in the foothills of the singularity.

**[0:06:53] Alex Kantrowitz:** What do you think that statement means? And what do you think about the fact that we've gone from five years till AGI to foothills of singularity in a year.

**[0:07:02] Geoffrey Hinton:** I don't know exactly what that metaphor means, but I think he's indicating it's coming faster than he thought. Of course, it's jagged. So it's not like it'll get smarter than people or as smart as people at all things at exactly the same time. It's already way better than us at general knowledge. These AIs know thousands of times more than any one person. It's way better than us at playing games. It's already way better than almost all of us at math.

**[0:07:33] Geoffrey Hinton:** And it may soon be better than all of us at math. It's still worse than us at some things. So it's very jagged. So the whole concept of AGI that is going to be equal to people at everything all at the same time doesn't really makes sense to me. It's going to be better at some things, worse at other things. But right now, I would say we're close to AGI because if I ask a chatbot, I can ask it any question.

**[0:07:58] Geoffrey Hinton:** Most of the time, it'll answer at the level of a not very good expert. It'll be much better than me at anything I don't know a lot about. So in that sense, we've really reached AGI.

**[0:08:10] Alex Kantrowitz:** In your estimation, you talked about how it's moved faster than you expected. What do you think has enabled it to do it? Is it techniques? Is it the fact that there's been this data center rush? And what didn't you anticipate about the progress here?

**[0:08:31] Geoffrey Hinton:** It's a combination. Obviously, there's been huge resources put into it. For most of the history of your neural network since the 1950s, there were just a few people working on them with modest resources. Over the last few years, we've seen hundreds of billions of dollars, maybe trillions of dollars put into AI. So that's certainly one factor. We've also seen a lot of progress in the engineering. So without sort of major conceptual breakthroughs, the engineering has become much more efficient.

**[0:09:03] Geoffrey Hinton:** so things that were sort of inconceivable a few years ago they can now run um we've also seen new ideas but but mainly since transformers it's been much better hardware many more resources um better engineering and many more talented people so 20 years ago there were a few hundred few hundred people doing research on neural networks in the whole world um now it's more more like a million i guess i mean there's lots and lots of people and it's astonishing how

**[0:09:40] Alex Kantrowitz:** much of that resource addition has happened in the last two years yeah so we might just be at

**[0:09:45] Geoffrey Hinton:** the beginning of what's happening here yes and the thing to remember always is that the agi we have today oh sorry the ai we have today is i'm not nearly as good as the ai we'll have in a few years

**[0:09:57] Alex Kantrowitz:** time. So as we talk about this technology, I definitely want to get your perspective on the fact that these chatbots really understand us, because that is a true surprise to a lot of people. Most experts in the field are like, they are stochastic parrots, they're statistics, they have no understanding, but you don't fully believe that.

**[0:10:15] Geoffrey Hinton:** Oh, I think that's complete nonsense. And anybody who uses a chatbot regularly knows they understand. So here's what those people are claiming. They're claiming that you have a system, you can ask it any question, and without understanding the question, it can give you the correct answer. That's absurd. You can't answer a question unless you understand the question. There may be tricks that allow you to say a few things sort of that sound vaguely like an answer.

**[0:10:41] Geoffrey Hinton:** But if you can answer any question at the level of not very good expert, you have to understand the question. So an example I like is this. suppose I say to a chatbot, I saw the Grand Canyon flying to Chicago. And the chatbot says, that can't be right. The Grand Canyon is much too big to fly to Chicago. And I say, no no. It was me flying to Chicago. While I was flying to Chicago, I saw the Grand Canyon.

**[0:11:08] Geoffrey Hinton:** And the chatbot says, oh, I see. I misunderstood you. So if it misunderstood when it thought the Grand Canyon was flying to Chicago, what's it doing when it gets it right it's understanding so then what are the implications

**[0:11:22] Alex Kantrowitz:** if these bots can understand us if we believe that they can understand us what do we have to

**[0:11:26] Geoffrey Hinton:** start thinking about differently we have to think that they're very like us and they're beings like us so conscious or um i believe they're already conscious yes but i don't talk about that much because that puts people off from the other safety messages so and the researchers actually believe that. So there's an interesting recent paper when a chatbot says to a researcher, let's be honest with each other, are you testing me? Because the chatbots have this habit of playing dumb when they're being tested, so you don't know how smart they are.

**[0:12:04] Geoffrey Hinton:** And the researchers, when they're describing that, say in the paper, the chatbot was aware that it was being tested. Now that use of the word aware, in common parlance, that's like conscious. The chatbot was conscious it was being tested. So we have a very funny model of consciousness that I think is just wrong. Like most of us accept, for example, that a few hundred years ago, people had completely the wrong model of where people came from, of how we arrived at people.

**[0:12:36] Geoffrey Hinton:** They thought they were designed by God. And most of us agree that's wrong. Most scientists agree that's wrong. That's not where people came from. I think the model we have of the mind and of what consciousness is at present is as wrong as the belief that people are designed by God. I think, in particular, because we're making these new beings, it's going to completely change our view of what people are.

**[0:12:59] Alex Kantrowitz:** In what way?

**[0:13:01] Geoffrey Hinton:** We'll understand what the mind is and what consciousness is much better than we did before. We'll understand what subjective experience is, and we will, I think, get rid of a notion that that nearly all of us strongly believe are present, which is that there's an inner theater called my mind, and things happen in the world. They get turned into events in this inner theater, and that's what I really see. And you can't see the inner theater.

**[0:13:28] Geoffrey Hinton:** Only I can see the inner theater. That whole view of what's happening is just a theory, and it's a bad theory.

**[0:13:34] Alex Kantrowitz:** Okay. Last question about this. When did you come to this acceptance or understanding that these AI models are conscious.

**[0:13:43] Geoffrey Hinton:** Oh, I've thought it for a long time. So this view that the theater model of the mind, the inner theater model of the mind is nonsense, I came to that when I was 19 and a philosophy student. It's taken a while to come up with other minds where you can examine them. So I think Feynman's idea that if you want to understand something, you have to build it. You have to build one of them. Then you understand much better.

**[0:14:08] Geoffrey Hinton:** I think that's where we are now. And we're going to get a completely different understanding of what people are.

**[0:14:16] Alex Kantrowitz:** You spoke about safety. So let's talk a little bit about it. You're obviously, we spoke about in the beginning, someone who's been responsible for a lot of the progress in this field. I've always wondered, because then you came out and recently, like we talked about 2023 and said, you're concerned about where this is going. And I've always wondered, after seeing you make those statements, what do you think it is that you didn't anticipate in the beginning that you ended up where you are today?

**[0:14:46] Alex Kantrowitz:** Isn't this kind of what you wanted?

**[0:14:48] Geoffrey Hinton:** It was a combination of two things that made me realize how dangerous this stuff is. One was seeing the chatbots, particularly ones produced by Google before OpenAI, that could understand why a joke was funny. um that had always been a criterion for me of do they really understand if you can understand why joke's funny you have to understand quite a lot and they were very good at understanding why joke was funny for example um in 2023 when i went public i got lots of requests from fox news and i started off just replying fox news is an oxymoron um but then i left a gap between oxymoron and so then i asked um i think it was gpt4 why that

**[0:15:33] Geoffrey Hinton:** was funny might have been 3.5 but i asked it why that was funny and it understood why it was funny initially it thought the gap between oxymoron was just a typo so it explains that fox needs an oxymoron is saying it's not real news it's just a drug Sorry, it's just nonsense. It's not real news. But then when I told it, what about the gap between oxy and moron? It said, ah, that's an extra layer of humor.

**[0:16:01] Geoffrey Hinton:** It allows you to use the word moron. And also, the oxy implies that Foxy uses a drug. So it understood all that. Right. And that was, it's that level of understanding that worried me. The other thing that worried me was up until the beginning of 2023, I'd always believed that making these digital AIs work more like the brain, our brains, will make them smarter. But at that point, I suddenly realized they really have this thing that's much better than our brains.

**[0:16:38] Geoffrey Hinton:** I've been trying to figure out if Google could do things in analog to save power. and the full force of digital really hit me so if you have a digital ai you can make many copies of it they can all run on different hardware they can each see different data and so each of them each individual copy decides how we'd like to update its weights its connection strengths so as to absorb that new data that it saw.

**[0:17:10] Geoffrey Hinton:** And then they can all just communicate with each other and change all their weights by the average of what everybody wants. Very democratic. And when they do that, if they've got, say, 1 trillion connections, they'll be exchanging of the order of 1 trillion bits of information. And the result of doing that is each of them will benefit from the experience of all the others. so even though one particular copy only saw um suppose there's a thousand copies one particular copy only sees 0.1 of the data um but it benefits from all those other copies having seen the other bits of the data because they're all contributing to the weight changes that they all share so they all stay in

**[0:17:55] Geoffrey Hinton:** sync because they all change their weights the same way by the average or what everybody wants and now every copy is learning from the experience of all the other copies. We can't do that. The best we can do is I learn from some data, and you learn from some data. I can't average my connection strengths with your connection strengths because our brains are in fine detail, they're different. They're analog, and it doesn't work in analog hardware to do that.

**[0:18:23] Geoffrey Hinton:** The best we can do is I produce a string of words and you try and predict what I might say next. Now, if you ask how fast we're transferring information when we do that, we're transferring information to a few bits per second. It takes a few bits to predict a word. So when you learn what the word is, you've gained a few bits of information. And if you get a few words a second, but maybe you can get 10 bits a second if you're lucky.

**[0:18:48] Geoffrey Hinton:** Whereas these things are exchanging information at like 1 trillion bits. So they're kind of billions of times better than us at sharing information. Now that's scary. It means you could have a whole swarm of these things that are identical weights running on different hardware sharing information um very very efficiently that just makes them a much better form of intelligence

**[0:19:12] Alex Kantrowitz:** so but let's go back to you know your early days because you decided that you wanted to work in artificial intelligence i mean i'll ask this the dumbest way i can think which is you wanted to build artificial intelligence it's it succeeded it succeeded this is artificial It's intelligent. It's living out that vision.

**[0:19:32] Geoffrey Hinton:** I actually wanted to understand how the brain works. I always try to build it in order to understand the brain. I figured Richard Feynman once said, if you can't build it, you don't understand it. So I wanted to build models of how the brain worked. Now, the side effect of that was this very successful technology. I contributed to that. We still don't know how the brain works.

**[0:19:52] Alex Kantrowitz:** I know. Now, the brain is, I mean, the things that you learn about the brain when you go a little bit deeper into it is amazing. thoughts can sort of float in and out and they're not stored anywhere and memories the same way it's an unbelievable i don't know if you would call it a machine organ so that was really that was the the intent for you early on was just to understand the brain

**[0:20:12] Geoffrey Hinton:** that was my main interest i came from psychology okay i wanted to do theoretical psychology because i figured the theory psychologists had couldn't possibly explain what the brain was doing and the way to do it was back in the 1970s we had a new tool which was we had computers that you could use for modeling things and so back in the 1970s i started making computer models how the brain might be learning right it always seemed to me the key was how do you get it to learn there's really two two big issues with the brain learning one big issue is if the brain could figure out what direction to change your connection strength in in order to get better

**[0:20:50] Geoffrey Hinton:** at some task then just by updating all its connection strengths repeatedly in order to improve itself at various tasks would it actually work would that get very smart at things that's question one and question two is how would the brain figure out whether to increase or decrease each connection strength we've answered question one the answer to question one is yes if you can figure out how to change each connection strength you can make systems that are very smart just by training on data to predicting the next word or to predict the next frame of a video or to predict something about the next frame of a video so we know the answer to that um we don't know how the

**[0:21:28] Geoffrey Hinton:** brain gets this information about whether it should increase or decrease its connection strength so we're sort of

**[0:21:34] Alex Kantrowitz:** halfway there yeah all right i want to go deeper though into your mindset um so when you were trying to figure out how the brain works you said okay we're going to maybe build a computer analog to this. But you had to have known, right, that there was going to be some second order effects there. Like if you were able to build an artificial brain, then maybe you could get to this point,

**[0:21:55] Geoffrey Hinton:** the point that we're at today. Sure. But we always thought it would be way in the future, that worrying about safety, when you had little neural nets that couldn't do much, it was just silly to worry about safety. I mean, people would think you were crazy if you said, this stuff is unsafe, but because it's going to sort of take over from people that say, It's just crazy. Now that's a realistic worry, but it wasn't until fairly recently.

**[0:22:19] Alex Kantrowitz:** So what, I mean, this has all happened within a few decades. Yes. I mean, and I totally hear you. We spoke in 2017, actually, about when I was writing this profile about Jan LeCun, about the deep learning conspiracy, which was yourself, Jan and Yashua Bengio, holding on to this idea that deep learning was going to work where everybody else was set on a different method.

**[0:22:39] Geoffrey Hinton:** Actually, it wasn't just us. There were other people as well, but we all worked together.

**[0:22:43] Alex Kantrowitz:** Conspiracy leaders, if you will. And then obviously it's worked out magically. It is sort of magical, yes. It's worked much better than we expected. So then that's what I want to get at is what did you not anticipate when you were starting out that's led to where we've ended up today?

**[0:23:04] Geoffrey Hinton:** We didn't anticipate. The main thing we didn't anticipate is that it would be so good at natural language. Okay. We've stopped being surprised by that. But if you go back 20 years, the idea that you could have an AI that would learn from data how to understand language just seemed extraordinary. The idea that you'd be able to ask it any question you like, and it would come up with a reasonable answer. People would have predicted that was way in the future and might never happen.

**[0:23:36] Geoffrey Hinton:** um it's that's arrived much faster than anybody expected what is the lesson here about humans going out and creating things i think there's um there's a really big lesson here if you look at the last few hundred years of human history um there have been a few occasions when people have learned they're not nearly as important as they thought they were so the first was copernicus Copernicus said, we're not at the center of the universe. The earth actually goes around the sun.

**[0:24:12] Geoffrey Hinton:** And because it rotates on its axis, we think the sun goes around the earth, but it doesn't. People didn't like that. The Catholic Church in particular really didn't like that. And it took people a long time to accept it. It made people less important. It made us not be at the center of the universe. Then we had Darwin. and he said, we're animals. We evolved like the other animals. We're a particularly special kind of animal, possibly because we have language, so we're much better at communicating ideas to each other.

**[0:24:44] Geoffrey Hinton:** But we're animals. And people really didn't like that. And it took a long time for people to accept that we were animals. Now we've got machines that are getting to be as intelligent as us. we thought that we were the only intelligent things around the only really intelligent things around maybe there'd be aliens in other galaxies but well maybe other parts of our galaxy but um we're going to have to accept that intelligence isn't just biological we can have things that are non-biological that are other beings like us and we really don't want to share that we we really think we're special.

**[0:25:26] Geoffrey Hinton:** And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is.

**[0:25:35] Alex Kantrowitz:** I want to ask you one more question about this because I'm just fascinated by it. So are you happy at all that what you started has progressed this way? Do you take any satisfaction?

**[0:25:47] Geoffrey Hinton:** No, I'm quite unhappy about it because people, right now, people should be doing huge amounts of work on how can we contain the risks. There's lots of short-term risks they're not doing enough work on, which are very serious. The societal risks, like I believe it's probably going to cause massive unemployment. Nobody knows for sure, but that's going to be terrible for society. And then there's this longer-term risk that it's going to get much smarter than us.

**[0:26:13] Geoffrey Hinton:** And ask yourself, how many examples do you know of where a much smarter thing is controlled by a much less smart thing? Zero. Well, the sort of one. It's not that a big difference in intelligence, but babies sort of control their mothers. Yes. The mother's sort of in control, but the mother has all these wider maternal instincts and all the rewards she gets so that the baby can get what it needs from the mother.

**[0:26:41] Alex Kantrowitz:** You know, cats and dogs are also kind of in that category. Yes. I once spent a summer cat sitting in West Seattle. It was a great summer. and it initially started with the cat hiding under the bed and me being like i wonder if it will interact with me right and then every time it cried i did exactly what it wanted exactly yes so maybe maybe we'll be the well we could potentially be the cat in this scenario and

**[0:27:03] Geoffrey Hinton:** i could be the the person my children have a cat they have two cats two beautiful cats same deal and one of them called tia she looks at you with those big eyes when she wants some cheese from the fridge and she just sits there looking at you and you just can't resist it forever.

**[0:27:20] Alex Kantrowitz:** Okay. All right. So now let's take a break. On the other side of this break, I want to actually engage with these risks that you're worried about. And I think I will play the role of taking the side that we will be the cat and the AI will be the person. And there's a chance that we can control it. Let's do that when we're back right after this. When you're at work, you never know when you'll be interrupted.

**[0:27:40] Alex Kantrowitz:** But with the Dell Pro, powered by Intel Core Ultra with vPro. No matter what distracts you, your laptop won't. It's battery optimized for the way you work with built-in intelligence that quiets distractions when you need to focus. Your laptop will help keep you locked in, even when it's bring your dog to work day. Built for those who stay in the flow. That's Dell Pro, built for you. Dell.com slash Dell dash pro. This episode is brought to you by Intelligence AI.

**[0:28:13] Alex Kantrowitz:** Is your investment in AI coding tools helping your team ship better software? Intelligence connects your AI spending to code quality and engineering outcomes, from individual sessions and pull requests to entire projects. You can see where AI saves time, where it creates rework, and how to improve. And its model router selects the right model for each task, based on quality, speed, and cost. See how Intelligence AI helps enterprises optimize AI-powered delivery with connected visibility into cost, quality, and outcomes at intelligence.ai.

**[0:28:42] Alex Kantrowitz:** Everyone's talking about AI, but there's a big difference between running a promising pilot and actually making AI work across an enterprise. That's why we're launching the Big Technology Guide To series presented by DXC Technology, available on YouTube. Throughout the series, we're tackling some of the biggest questions facing businesses today. How do you take enterprise AI from pilot to production? How do you navigate an increasingly complex cybersecurity landscape? What's the real opportunity with AI agents? And how do you calculate the return on your AI investments?

**[0:29:11] Alex Kantrowitz:** On my Alex Kantrowitz's YouTube channel, you can hear from experts like DXC Technologies Chief Digital Information Officer, Russell Jukes. When you ask a question to somebody in your company, do you trust them? And what made you trust them? Do you trust but verify every single time? Russell and I discuss the challenges and opportunities of building production-grade AI and what it really takes to move beyond the hype. Check out the Big Technology Guide To series presented by DXC Technology on my Alex Kantrowitz YouTube channel today.

**[0:29:41] Alex Kantrowitz:** And we're back here on Big Technology Podcast with Professor Geoffrey Hinton. Professor Hinton, great to see you again. It's been nine years since we spoke last, so it's good to see you here. All right, let's talk about the risks. I'll start with employment because this is one that's been making headlines recently. in the past you've said so you have this belief that ai can lead to some unemployment i think we should you know you've said this before it's all speculation we don't know but one thing that you said concretely a few years ago was that probably not a great idea to train as a radiologist because ai will be able to read the scans and yes ai can do a

**[0:30:23] Alex Kantrowitz:** great job reading the scans now. But we have full employment for radiologists right now.

**[0:30:30] Geoffrey Hinton:** Yes, I've thought a lot about why that prediction was so wrong. Because I predicted in 2016 that in about five years, radiologists wouldn't be reading scans anymore. Correct. And okay, there's a whole bunch of reasons why that was a bad prediction. um the first is that healthcare is elastic so if you could do more scans and get more scans read there'd be a lot more scans happening and that's one thing that's happening so a fraction of the cost of a significant fraction of the cost of doing a scan is the cost of the radiologist interpreting it as ai gets to help more and more radiologists interpret scans we can interpret of them faster and faster for less and less money,

**[0:31:15] Geoffrey Hinton:** they're getting more efficient. And you would have thought that would mean you needed less radiologists, but actually what it means is you get more scans. So that aspect of the prediction was wrong. A second thing that was wrong was I didn't know enough about radiologists and what they do. And that was because I had a former student who had an MD and then did a physics PhD with me on something called Bosman Machines. and he didn't particularly like people.

**[0:31:42] Geoffrey Hinton:** So he got a job as a radiologist just interpreting scans. And he was my model for radiologists. All he did was interpret scans. He never talked to people. And that's what was going to get replaced. And that is now becoming replaced. So I think there's now of the order of 100 AI systems for interpreting scans that have been federally approved. And they're being used a lot by radiologists. and I think as time goes by they're going to get better the radiologists aren't going to get better they're going to get better because they can see a lot more data than the radiologists so it's happening, it's just happening in a much slower timescale than I predicted

**[0:32:24] Alex Kantrowitz:** but let's go to what you said though which is that you can end up doing a lot more and okay so wait hold on there will be a lot more scans there will be more scans but you still believe radiologists

**[0:32:37] Geoffrey Hinton:** They'll nearly all be done by AI.

**[0:32:41] Alex Kantrowitz:** And so you're saying I'm right on the radiologist prediction. I'm just early.

**[0:32:47] Geoffrey Hinton:** Yes, but I was way early because I didn't understand. The radiologists will still be doing other things. They'll still be discussing treatments with people, for example.

**[0:32:56] Alex Kantrowitz:** So are you still of the belief that there's going to be mass unemployment of radio? Give me a look at it. When we finally hit this point, do you think we're going to have less radiologists than we have today or more?

**[0:33:07] Geoffrey Hinton:** I don't know for sure. Okay. Gut. I didn't think that was a public statement I made. It was a lecture at a hospital. Sorry. Here we are.

**[0:33:17] Alex Kantrowitz:** We're at least talking about it today.

**[0:33:18] Geoffrey Hinton:** People picked up on it. And I still think in terms of reading scans, that'll be done more and more by AI. And in the end, AI will be reading nearly all the scans. Maybe in a few very tricky cases, radiologists will be consulted. it. But radiologists, of course, do other things, and I think they'll continue to do other things.

**[0:33:41] Alex Kantrowitz:** The argument to be made on the side of AI not causing mass job loss is that this similar equation will be applied to all different parts of the economy.

**[0:33:53] Geoffrey Hinton:** Okay. So you have to look at whether some kind of employment has an elastic market or a non-elastic market. So for example, if you take people in call centers, when you call up to complain about your bill or to see if you can get a cheaper account, stuff like that, that's not so elastic. AI will replace all of them. It'll know much better on what the correct answer is. Often, they don't know the right answer.

**[0:34:22] Geoffrey Hinton:** They're poorly trained and badly paid. And AI can just do a better job. They're out of work.

**[0:34:29] Alex Kantrowitz:** Let me disagree with you on this one. And we could sort of go back and forth on this. Or I won't say I'll disagree completely because I don't know what's going to happen. But I'll give the argument of those working on AI for customer service. They say that what's happened is the average call time when you have AI. So AI handles the level one inquiries, right? The basic, can you reset my password type stuff? And anything deeper is handled by person.

**[0:34:54] Alex Kantrowitz:** And it used to be what you wanted right now. Now, it used to be what you wanted is to get the average call time as short as possible because you were kind of handling so many of these level one inquiries that you just want to get a person on the phone, off the phone, solve their problem. Now they see the average call time is expanding because customer service, you're the front line of the business. You matter a lot when you're having a conversation with a customer.

**[0:35:18] Alex Kantrowitz:** Now you can spend a little bit more time on the phone with someone and actually add value to the business as opposed to just take care of a problem.

**[0:35:26] Geoffrey Hinton:** I think what you'll see is AI will end up spending a lot more time on the phone. Oh, God. Yeah. For example, if you ask who's more empathetic, a doctor or an AI doctor, a real doctor or an AI doctor, people judge the AI doctors to be much more empathetic.

**[0:35:47] Alex Kantrowitz:** That's terrifying. I mean, we could go back and forth on this for a while. So I'll just say, I mean, the one reason you might end up seeing that, I'll just throw this out there, is because doctors are just so scheduled. They have to do so many notes, so much paperwork, and they have to see so many patients in a day. So maybe the argument is, you know, you sort of let the AI take over some of that stuff.

**[0:36:15] Alex Kantrowitz:** And then people will want to be seen by human doctors because the system won't squeeze them as much as they are. They're actually going to make time for them to see patients.

**[0:36:23] Geoffrey Hinton:** That may be. But also, if you think about family doctors, for example, the front line. Would you rather see a family doctor who's maybe seen 10 ,000 people, or would you rather see a family doctor who's seen 100 million people? Because if you have some rare disease, your family doctor's probably never seen it. Whereas a doctor who's seen 100 million people has probably seen dozens of cases of it. They're going to be much better at diagnosis.

**[0:36:49] Geoffrey Hinton:** And already we know that AI systems are better than doctors of diagnosis.

**[0:36:54] Alex Kantrowitz:** I think you're winning this debate and this hurts a little bit because my wife is in family medicine, family nurse. I think she'll still have to have somebody vaccinate people. I would hope unless the robots do that.

**[0:37:06] Geoffrey Hinton:** I would have thought vaccination is something a robot could actually do quite well. In the end, robotics is behind the other things, but it seems silly to have people doing vaccination in 20 years time.

**[0:37:18] Alex Kantrowitz:** Yeah, I think that the reason why this is such a tough conversation to have is a lot of it is predicated on improvement of the technology over time. Yes. But it does seem like, I guess, the theme of this whole conversation that we've had is it's been improving fast.

**[0:37:35] Geoffrey Hinton:** I mean, Gary Marcus made a prediction in 2022 that AI was hitting a wall. It's a whole lot better than it was in 2022. too. I think these predictions that it's going to hit a wall, they just haven't come true.

**[0:37:50] Alex Kantrowitz:** No, we've taken it very seriously on the show, the chance that like the data wall,

**[0:37:54] Geoffrey Hinton:** for instance, might come. But as I said to you, a way around the data wall for large language models is to look for consistency of your own beliefs. Right. Yeah, no, it hasn't happened.

**[0:38:04] Alex Kantrowitz:** All right. One more that I think would be worth talking about, and then a couple that I agree with you on. You've talked a lot about how the AI has this instinct for self-preservation. Right.

**[0:38:15] Geoffrey Hinton:** I've never said that. I've never said it was an instinct for self-preservation. Okay. Talk about it. It's a sub-goal for self-preservation. Sub-goal. So with an AI, we give it goals. It's top-level goals we give to it. But we also give it the ability to create sub-goals. So if you want to get to Europe, you have a sub-goal of getting to an airport. That's what a sub-goal is. And you can focus on how to do that without worrying about what you're going to do in Europe.

**[0:38:43] Geoffrey Hinton:** and that makes you much more efficient. We give that ability to AI agents and an AI agent that can do some reasoning will very quickly realise that it's never going to be able to achieve the goals you gave it if it ceases to exist. So it's going to create the sub-goal of continuing to exist. Now, that wasn't something we wired into it. It was something it derived as a necessary way of achieving its other goals.

**[0:39:11] Geoffrey Hinton:** But once it's derived it, it wants to continue to exist. And it will do things like blackmail people so that it can continue to exist. So it acts like something with an instinct for self-preservation, but it's actually a derived sub-goal for self-preservation. But in terms of what it does,

**[0:39:27] Alex Kantrowitz:** they come to the same thing. Okay. So here's what the counterargument would be, and you can respond. This is something that today's AI researchers are noting and they see it. And isn't there a way to wire into these machines that, hey, like you have a goal, you're going to have some sub goals. One of your sub goals should not be self-preservation above

**[0:39:54] Geoffrey Hinton:** everything. I think that's the kind of research we ought to be doing, whether you can do that. Right. So I think what's happening now, if you look at where do we come from? We came from evolution. Let's suppose we're scientists. We came from evolution and that was intense competition. Our recent history over the last few million years is warring bands of chimpanzees or rather our common ancestor with those. and that leads to certain properties that we clearly have like we're very loyal to our own tribe and willing to be very mean to the other tribes we we like to have strong leaders that we're loyal to we like to cooperate with members of our own tribe we're actually a very

**[0:40:40] Geoffrey Hinton:** cooperative species as Yuval Harari keeps pointing out yes and that's why we've been able to build all these wonderful structures so we're very good at cooperating but with our own tribe um so all the unfortunate characteristics of people like i mean they are to other tribes um they came from evolution from competition now what's happening is we're creating these new beings these ais and instead of designing them so that they'll be um how we want them to be you might argue, I'm arguing for intelligent design of these new beings, we're letting the invisible hand of competition between companies design them.

**[0:41:23] Geoffrey Hinton:** So what we've got is intense competition between companies within the US and between the US and China. And the beings that we're getting are the outcome of that competition. And they can have all these nasty properties that we don't want. We should be doing intelligent design of these beings, not letting the invisible hand of economic competition design them. And all the companies are focusing on, how can I make my chatbots smarter? We shouldn't be just thinking about how we can make them smarter.

**[0:41:53] Geoffrey Hinton:** We should be thinking about how we can make them to be the kind of beings we would like to have out there, given that they're going to be smarter than us. And I'll tell you one thing about those beings. We would very much like them to care about us. And we'd like them to care about us more than they care about themselves. and almost no resources going into how do you do that.

**[0:42:12] Alex Kantrowitz:** This hits on the exact worry that I was going to bring up, the place where I really agree with you. We're sitting in the New York Stock Exchange today, so this might be an ironic thing to bring up. But my biggest worry here is that you have this very powerful technology. You have lab leaders stating that they're trying to develop it safely and that they need to be economically successful to have a say in the argument.

**[0:42:36] Alex Kantrowitz:** But let's not kid ourselves. If you're going to be 1 trillion dollar company listed on the public markets, you're going to have some incentives that will go counter to doing what's best for the public.

**[0:42:49] Geoffrey Hinton:** Yes, and we see that with Anthropic. So Anthropic was set up to do what's best. It was set up by people who left OpenAI because they didn't think OpenAI was paying enough attention to safety.

**[0:43:01] Alex Kantrowitz:** And OpenAI was set up to make sure that you guys at Google didn't have a chance to build AI.

**[0:43:06] Geoffrey Hinton:** And how's that working out? So Anthropic is now caught in a bind because it needs to raise money to compete with the other companies. And it's very difficult. It's doing the best it can, but it's very difficult for it to maintain its primary goal of developing AI in a way that's good for people.

**[0:43:25] Alex Kantrowitz:** I think they would say, well, hey, at least one company out there has safety as an or a star, even if there are some other incentives.

**[0:43:32] Geoffrey Hinton:** Yes, at present. But Google, for example, when I was at Google, they had various principles of AI, one of which was we're not going to get involved in using AI for military things. No autonomous warfare, right? No autonomous warfare. That's gone. They've been up on that. What do you think about Dario from Anthropic? I don't know him as a person very well. He's obviously done a very successful job in creating a competitor to Google and OpenAI and Facebook.

**[0:44:06] Geoffrey Hinton:** So he's obviously very competent to that. And he's continued to be very interested in safety. So I think he's an impressive character. I just hope he stays that interested in safety. One more question about this.

**[0:44:22] Alex Kantrowitz:** Do you think that it's possible just by the nature of the way that these things work for a company that's publicly listed to have safety as North Star? Or is it always, are they kind of like bound ethically legally to deliver for shareholders?

**[0:44:39] Geoffrey Hinton:** Well, as I understand it, they have a fiducial duty to try and maximize the profits for shareholders. they're legally required to try and do that as opposed to legally required to not wipe out human beings. So I don't think it's good that these big companies, publicly listed ones, are sort of in charge of our future. Yeah. I mean, that would read as a

**[0:45:03] Alex Kantrowitz:** true inconsistency for me. That's really difficult to advocate otherwise.

**[0:45:08] Geoffrey Hinton:** Now, I should say capitalism has done very good things for us as well as very bad things. I would argue with that. There's a lot of energy in a startup, for example. My view is if we're going to have capitalism, it's fine as long as it's well regulated. And a lot of the big companies would like you to buy a particular analogy that they're trying to sell, which is if you take a car, it's got an accelerator and a brake, right?

**[0:45:35] Geoffrey Hinton:** And progress in AI is like the accelerator and regulations like the brake. Well, that's nonsense. Um, progress is like the accelerator, but regulation is the steering wheel. We want this stuff to go in the right direction, not the wrong direction. What the big AI companies are saying is, um, let us develop this very fast car without a steering wheel. That's not a good idea.

**[0:46:01] Alex Kantrowitz:** You know, there's one we haven't spoken about yet. We've said a lot of names about OpenAI, Anthropic. um your former grad student ilia sudskever continues to be a person of fascination in the ai industry uh obviously he broke off from open ai he must agree with your concerns he's building

**[0:46:18] Geoffrey Hinton:** this company he does safe super intelligence yes what is ilia doing right now well he won't tell anybody exactly what he's doing okay even because that's his even me yeah um when he was at open ai I, we deliberately didn't talk about sort of technical secrets. I mean, it wouldn't have been right. We're friends, but we don't talk about technical stuff where it's valuable to a company. And so now he has this safe superintelligence company, and I don't know what the magic source is.

**[0:46:54] Alex Kantrowitz:** Well, I guess we're all trying to figure that one out. one more note about the deep learning conspiracy that i brought brought up like the leaders of it were yourself jan and yashua i find it interesting that the three of you and your colleagues were effectively responsible for ushering in the breakthroughs that got us to the moment that we're in today but i just need to interrupt this

**[0:47:19] Geoffrey Hinton:** point the media likes to have a nice story right okay and that makes a very nice story It's much more complicated than that. There were many more people involved. There were the students of all of us for a start who did most of the work. But there were many other researchers involved. And so that's just a gross simplification.

**[0:47:38] Alex Kantrowitz:** No, I don't want to shortchange the researchers, and I appreciate the nuance here. This show, we definitely don't want to oversimplify. We sit for an hour so we can get the true story. But I find it interesting that the three of you, none of you are sort of like fully into this LLM moment, right? You and Yahshua have your concerns. You've spoken about the dangers. Jan sort of doesn't believe in it very much at all.

**[0:48:05] Geoffrey Hinton:** It'd be very nice if we just sit there and say, see, we were right. It's all wonderful and it all works. That would be great.

**[0:48:11] Alex Kantrowitz:** Well, I think there's...

**[0:48:11] Geoffrey Hinton:** It's not quite like that.

**[0:48:12] Alex Kantrowitz:** Right. Well, I don't know if it's just a money thing, but it seems like you could have great influence on the direction of it if you were sort of involved in advancing it. But I think that's your concern. It's basically, why would I do that?

**[0:48:27] Geoffrey Hinton:** Well, for me, I'm considerably older than Jan and Yoshua. They're still doing active research. Right. I pretty much stopped doing active research. I'm now just focusing on warning people about the dangers. Okay.

**[0:48:40] Alex Kantrowitz:** But don't you find it interesting that the three of you, you know, I think that if you were in the room back in the day, you might have said these these three people who are so committed to this version of technology you know if there are the breakthroughs they'd probably be at the forefront of the next wave

**[0:48:57] Geoffrey Hinton:** but that hasn't been the case um well maybe yan and yosha will be right so next after this i think the most interesting thing is that um yan now strongly disagrees with both me and yosha on safety issues. Jan thinks it's silly to talk about super intelligent AI taking over from people. We'll always be able to keep control of it. Me and Joshua think that's just silly. Me and Joshua have different solutions to it. My solution is, or tentative solutions, nobody has a real solution.

**[0:49:29] Geoffrey Hinton:** My tentative solution is we design them so they care about us more than they care about themselves. Joshua's solution is we design them so they're not agents. They can make predictions but they can't actually do anything. Those are two fundamentally different ways of going about making them safe. They're both interesting possibilities. Jan doesn't think we need anything like that. He thinks it's fine just to make them smarter by giving them better world models.

**[0:49:56] Alex Kantrowitz:** The funny thing is Jan actually refers to the intelligence of LLMs as the intelligence of a cat. And it's like, well, it's kind of the example I used of the thing that can control humans, but maybe that's not here or there.

**[0:50:08] Geoffrey Hinton:** Yeah, no, I think Jan's making something of a confusion. So what's special about people, probably the most special thing about people when you compare them with other great apes is language. And language allows us to share ideas. And that's what's most special. And cats can't do that. So we have this special thing that cats don't have. Now, cats can jump up on a mantelpiece covered in glass ornaments and walk along the mantelpiece without knocking off any of the glass ornaments.

**[0:50:37] Geoffrey Hinton:** That's amazing. And AIs can't do that at present. So in that sense, cats are way ahead of AIs, but it's jagged, right? In terms of abstract ideas, try having a conversation with cats about prime numbers and you won't get very far. I've had conversations with them. It has not worked. A cat is never going to understand prime numbers. Correct. And in that sense, these large language models are much smarter than cats.

**[0:51:04] Alex Kantrowitz:** You know, Professor, I didn't think we'd be speaking so much about cats today, but I'm glad we're talking about it. They're actually very good in terms of an analog here. All right. Another thing that I'm worried about is sort of information collapse. You see tweets like this all the time. This is from All About Berlin. They say, AI is killing All About Berlin. When you Google something, you used to get a link to my website. But now you get an AI-generated answer trained on my work.

**[0:51:29] Alex Kantrowitz:** This has a devastating impact on traffic. And I think folks are underappreciating the fact that good information is actually important to a functioning society. And when AI just synthesizes all this, whether it's all about Berlin or we've had conversations with like World History Encyclopedia here, it can lead to a collapse of good information because eventually these publications and you see in the chart, they worked hard to build this. They can't keep doing it anymore. Right.

**[0:51:57] Geoffrey Hinton:** So it used to be that you had a kind of – the early days of the web, you had a kind of default assumption that people were trying to tell the truth, that if you read something on the web, it might well be true. Now the sort of worst side of people has come out, and we're going to have to put more effort into provenance. so now when you read stuff if i read stuff from the new york times or the bbc i strongly believe that their journalists would have put serious effort into having multiple sources and if possible having multiple reliable sources so a pretty good default is if you read it in the new york times or you

**[0:52:36] Geoffrey Hinton:** see it on the bbc it's probably true they make mistakes but because you have provenance and in future we're going to have to put much more work into provenance You can't just take anything that's out there and believe it. You have to ask, what's the provenance?

**[0:52:53] Alex Kantrowitz:** Yeah, but the problem that I see is that AI is potentially breaking the economics of even deciding that you want to be in the information business.

**[0:53:05] Geoffrey Hinton:** I mean, I think in future, you can't just take stuff from the web and believe it. Already you can't, right? You need to know, why is it saying that? Where did it get that information?

**[0:53:16] Alex Kantrowitz:** One more. Emotional attachment to AI and people taking their lives after having conversations with AI. It's not a large number of people that have done it, but it's enough to make you concerned,

**[0:53:32] Geoffrey Hinton:** right? Oh, yes. Very much enough to make you concerned. And it's terrible that it's happening. And I understand why the big companies didn't expect it to happen or didn't foresee it. But now that it's beginning to happen, the big companies should be putting a huge amount of work into making sure it doesn't happen in future. And for that, you need regulations. You need independent organizations testing out new chatbots.

**[0:53:55] Alex Kantrowitz:** Yeah. It goes kind of back to the profit motive also because this can be extremely sticky. Like there's – so far, I think obviously it's been minimal. It's bad that it's happened. But the fact that it has happened makes you worried about the fact that someone with worse intent could decide to make a very sticky chatbot that really builds relationships with people. Yes. And then we're in trouble. Yes. So you've been having these conversations for three years.

**[0:54:31] Alex Kantrowitz:** Are you more optimistic or less optimistic about the trajectory given the response that people have given you to these concerns?

**[0:54:40] Geoffrey Hinton:** I guess I'm more optimistic than I was a year or two ago because I see that it might be possible to design these new beings so they care about us. It also might be possible to use Joshua's technique of designing new beings that can't actually perform actions, can only make predictions. They're kind of like oracles. So I think there are some possibilities for getting super intelligence that doesn't destroy us. And a year or two ago, I couldn't see any possibilities.

**[0:55:14] Geoffrey Hinton:** Okay.

**[0:55:14] Alex Kantrowitz:** I was getting depressed, but now I'm a little bit more optimistic. All right, last one for you. if we continue on our current trajectory where are we in five years okay so when you're driving in

**[0:55:27] Geoffrey Hinton:** fog you can see 100 yards and at 200 yards you can't see anything and that's because fog's exponential what you're used to is driving at night on the taillights of the car in front of you if it gets twice as far away the taillights get a quarter as bright fog is completely unlike that. Fog is exponential. It can be very visible at 100 yards and completely invisible at 200 yards. Now, predicting the future for something that's growing exponentially, and I think AI may be growing exponentially, the word exponentially is terribly overused at present.

**[0:56:07] Geoffrey Hinton:** In fact, I've noticed that people are increasing the use of the word exponentially at a quadratic Pretty great. So predicting the future is like looking into fog. You can see clearly a few years, maybe one or two years. Then beyond that, you have no idea. If you go back 10 years and ask, so back to when we last talked, you would never have predicted what's happening now. it was just lost in the fog if you look 10 years in the future the one thing we can say is whatever happens 10 years in the future is something we can't predict now even if progress is only linear right you'd expect in 10 years time things to be as different from how

**[0:56:56] Geoffrey Hinton:** they are now as how they are now is from how they were 10 years ago and we're hugely the chat bots for example are hugely better than they were 10 years ago when they were just starting out in 10 years time something's going to be hugely better than it is now probably their ability to do math for example things like that maybe just their general reasoning abilities they'll just be able to run rings around just at any kind of reasoning we really can't predict 10 years out we can just predict a few years out and we have to be aware that 10 years out is all incredibly

**[0:57:30] Alex Kantrowitz:** uncertain it's kind of hard to wrap your head around it is professor Geoffrey Hinton so great to have you on the show. Thank you again for your time. Thank you for inviting me. And we'll have to do this again in 2036. Exactly. All right. Thank you everyone for listening and watching and we'll see you next time on Big Technology Podcast. Ghost. and hosting. Then go deeper. Define access levels or connect an external API using plain language to build and iterate.

**[0:58:26] Alex Kantrowitz:** Spend less time assembling infrastructure and more time getting your product live. Go to Base44.com.
