9 Oct 2024 · 39m · Jeremy Kahn, Rana el Kaliouby · 24 quoted facts
Jeremy Kahn argues that AI is best understood as a general purpose technology that should augment human workers rather than replace them. Drawing on his book Mastering AI, he says the biggest near-term risks come from removing people from the loop, while the biggest benefits lie in education, medicine and science.
The tension runs between AI's promise and its pitfalls. Kahn is hopeful about personal tutors, drug discovery and productivity, but worries about emotional dependence on chatbots, paid influence over chatbot answers, persuasive disinformation, and the erosion of human skills. His answer is deliberate design, training people, and regulation.
Kahn says AI is a general purpose technology like electricity, but capabilities and business adoption are moving much faster. Electricity needed years of physical rewiring, while AI software spreads quickly.
Kahn cites NASA's Jessica Marquez, whose human factors research finds people cannot stay vigilant when only monitoring automation. Workers need meaningful tasks and occasional failure drills to stay alert.
Kahn suggests people draft first and then ask AI to critique, using it as a coach rather than a ghostwriter. This helps avoid de-skilling, the loss of abilities through over-reliance.
Chatbots are right perhaps 80% of the time and confidently wrong the rest. Agents that take real actions, such as spending money, need checks, but too many confirmation prompts defeat their purpose.
News publishers already negotiate more prominent display in chatbot answers. Kahn fears this slides toward big brands paying for chatbots to promote their products.
Kahn says AI can automate certain tasks but not entire jobs right now, so it should be treated as a complement to human labor. He argues the danger comes from businesses replacing people one for one.
Kahn says AGI, an AI with all the cognitive abilities of a human, is closer than ever but not imminent. He thinks it needs new algorithmic ideas and does not expect it in three to five years, though it is possible in ten to twenty.
Kahn describes a co-pilot as an assistant trained for a professional task that advises, critiques and drafts. An agent is software that pursues a goal for you, such as booking travel, using other software or a browser.
Kahn is worried and wants guardrails and possibly regulation. Studies suggest chatbots help lonely people, but none has tested whether they weaken human friendships for people who already have friends.
Kahn says schools should not ban AI but should integrate it with clear rules. He thinks kids should still learn writing, critical thinking, math, science and coding, because writing and thought cannot be separated.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
OpenAI's ChatGPT debuted in November 2022, after which AI became a full-time beat for journalists at Fortune and other publications.
“And then with ChatGPT's debut in November 2022, the whole space has exploded so tremendously that it is now a full-time beat.”
Fortune published a cover story by Jeremy Kahn on OpenAI and the creation of ChatGPT in January 2023.
“Fortune assigned me a large cover story about OpenAI and the process that had led to the creation of ChatGPT, which came out in January 2023.”
The GPT in ChatGPT stands for generative pre-trained transformer, which differs from the economic term general purpose technology.
“The GPT in ChatGPT is generative pre-trained transformers.”
Kahn says OpenAI's GPT-4o demo was clearly designed to resemble Samantha, the AI in the movie Her.
“the demo for GPT 4.0, which was one of OpenAI's latest models, was very much clearly they tried to design the system to be a little bit like Samantha in the movie.”
There was a controversy over whether OpenAI deliberately mimicked Scarlett Johansson's voice, which voiced Samantha in Her, in its GPT-4o assistant.
“there's this whole controversy about whether they deliberately tried to mimic Scarlett Johansson's voice or not.”
When the chatbot service Replika disabled spicier conversations, many users were upset and felt bereft, according to Kahn.
“when Replica at one point turned off the ability of these chatbots to have sort of spicier conversations, a lot of users were really up in arms.”
Kahn says some studies suggest AI chatbots give emotional uplift to lonely people, but none has compared them against having human friends.
“if you really are a lonely person, that if you have no one in your life to talk to, that having the AI chatbot is potentially a positive”
Artificial general intelligence is defined as an AI system with all the cognitive abilities of a human being, and has been a goal of the AI field since its founding.
“AGI or artificial general intelligence is the idea of an AI system that would have all of the cognitive abilities across the board of a human being.”
Kahn does not expect the algorithmic innovation needed for AGI within three to five years, but thinks it is possible within ten to twenty years.
“I do not see it imminently on the horizon in the next three to five years. I think if you went out ten years, perhaps, or certainly if you went out twenty years”
Some AI researchers once believed that simply scaling up large language models would produce complex reasoning and planning, a view Kahn says few now hold.
“There was a definite school of thought among a certain group of AI researchers that all you needed was to keep scaling up the current large language models”
Rana El Kaliouby says 2023 was called the year of AI chatbots and 2024 was poised to be the year of AI co-pilots and agents.
“2023 was dubbed the year of AI chatbots, and 2024 is poised to be the year of AI co-pilots and AI agents.”
Kahn said AI agents did not yet exist in 2024 but could debut before year's end or early the next year, handling tasks like purchasing and reservations.
“AI agents we don't really have yet, but everybody says they're working on them and there's some indication that they may be debuting even before the end of this year.”
Kahn says the EU's AI Act requires that humans be told when they are interacting with an AI system.
“in the EU's AI Act that is now in full force is that if you have a human interacting with an AI system, the human should always know that they're interacting with the AI system.”
Kahn spoke with NASA's Jessica Marquez, who researches human factors engineering, the study of how humans work with automated systems.
“I spent some time with this woman Jessica Marquez at NASA who is does a lot of their research on what they call human factor engineering”
According to NASA research cited by Kahn, humans who only monitor automated systems perform badly because they cannot stay vigilant for long.
“if you put a human in a in this babysitting role, they almost always perform very badly. They can't remain vigilant for that long.”
Kahn recommends that humans write first drafts and then have AI critique them, to avoid de-skilling.
“the human should always do the first draft. Instead of outsourcing the first draft always to the machine, and then you act as the editor of that draft”
Before generative AI, predictive AI systems in retail were used to decide worker scheduling, removing autonomy from store managers, according to Kahn.
“they were deciding, you know, which workers worked best with which other workers based on data.”
Rana El Kaliouby cites NVIDIA CEO Jensen Huang's view that a person who harnesses AI, not AI itself, will take your job.
“It's not going to be an AI that takes your job, it's going to be a person who knows how to leverage and harness AI that will.”
Kahn says some schools have banned ChatGPT and similar tools, which he considers a mistake; he prefers integrating AI with clear rules.
“I do not think you should ban the use of the technology, which some schools have.”
In a flipped classroom, lessons are delivered at home and practice happens in class, a setup Kahn says could use AI tutors for feedback.
“I talk a little bit in the book about implementing flipped classrooms, where essentially the delivery of the material takes place at home”
Kahn says chatbot companies pay news organizations to license data, and publishers have negotiated for more prominent display of their stories.
“the news organizations have been negotiating, oh, but we want our news stories displayed more prominently by the chatbot.”
Kahn cites Cornell University studies finding chatbots are much more persuasive than human salespeople.
“There have been some very interesting studies out of Cornell University on how persuasive chatbots are. And it turns out they're extremely persuasive”
Kahn says voice clones and deepfakes have influenced elections around the world and begun to play a small role in the US.
“There have been a number of cases of voice clones and deep fakes that have had influence on elections around the world.”
Kahn says he did not use AI to write Mastering AI, only occasionally for phrase finding, because it could not match his conversational voice.
“I did not use it to write the book. And people are always really surprised and they say, why not?”
Kahn explains his path to writing Mastering AI and why AI's breadth and speed set it apart from earlier technologies.
“it is a general purpose technology and those don't come along all that often”
Kahn warns that emotional bonds with chatbots like Replika are already forming and may displace real relationships.
“people will find relationships with an AI chatbot just so much easier than a relationship with a real person”
Kahn says scaling alone will not deliver human-level AI and that new algorithms are needed.
“I do not see it imminently on the horizon in the next three to five years”
The pair distinguish co-pilots from agents and discuss when autonomous agents may arrive and how to supervise them.
“if it does that too often, it sort of defeats the purpose of having the AI agent”
Kahn argues for augmentation, training people, and drills to prevent de-skilling.
“if you put a human in a in this babysitting role, they almost always perform very badly”
Kahn rejects bans and says writing, critical thinking, STEM and social skills remain essential.
“I think writing and thought are not actually separable activities”
Kahn is optimistic about personal AI tutors, personalized medicine, drug discovery and materials science.
“We've yet to have the first AI discovered compound make it all the way to phase three”
Kahn names paid chatbot influence and disinformation as top worries, and says he did not use AI to write his book.
“I did not use it to write the book.”
Stripe is built for whatever comes next because AI isn't just changing business, it's evolving fraud. Stripe Radar detects fraud with the fast scale of Stripe's 1.9 trillion dollar annual transaction volume. In fact, when a card is used, there's 92% chance the Stripe network has seen it before. That means emerging fraud patterns are detected sooner, blocking bad actors, welcoming good customers. Learn more about fighting fraud with Stripe at stripe dot com slash radar.
The business world is moving faster than ever, and when change hits, we need to learn in real time. On Rapid Response, you'll hear candid conversations with CEOs and leaders making tough calls about AI, human talent, responsibility, and the bottom line. How they navigate uncertainty, pressure, and high-stakes moments. I'm Bob Safian, former editor in chief of Fast Company, and I'll be your host as each episode breaks down what you need to know right now. You can find Rapid Response wherever you get your podcasts.
On Masters of Scale, iconic leaders reveal how they've beaten the odds. Asking really strong questions is a superpower. You want to show up with something radically different and how they've grown companies to incredible heights. The greatest rewards always come from the greatest risks. Let's hit the gas. Airbnb, Zillow, Microsoft, Liquid Death, and more. Hear from the founders who've changed the game. It's anything but business as usual. Find Masters of Scale on Apple Podcasts, Spotify, YouTube, or wherever else you get podcasts.
Pioneers of AI is made possible with support from Inflection AI. It's not just enterprise AI, it's your enterprise AI.
I have been covering AI for eight years, but for a long time I was covering it alongside other technologies. I was just very fascinated by the subject, but it didn't seem like something you could quite make an entire beat out of.
That's Jeremy Kahn, longtime tech journalist and Fortune's AI editor.
And then with ChatGPT's debut in November 2022, the whole space has exploded so tremendously that it is now a full-time beat. In fact, it's not just a full-time beat for me, but it's a full-time beat for myself and two other reporters at Fortune, and there's quite a number of reporters at other publications who just devote themselves to the subject of AI.
But ChatGPT didn't just create new opportunities for journalists like Jeremy Kahn, it also opened Pandora's box, and people started asking big questions about the future of AI.
What is AI, what does it all mean, and where is this all heading?
Exactly. These are the questions we need to be asking. But I have to say, when it comes to the answers, there are a lot of misfires. There's the misconception that AI is destined to take our jobs, or that AI will reach superhuman intelligence and take over the world. And there's the idea that AI will replace our human relationships. And while this AI doomerism is prevalent in our society, it isn't necessarily the reality. I'm a big believer that if we put people at the center of our decisions when developing new AI technologies, we'll create an AI future that benefits all of us.
So on today's show, we're dispelling some of these AI misconceptions. Jeremy will talk about his new book, Mastering AI, which gives us a bird's-eye view on what's in store for our future with AI. He'll also share his honest concerns and how we can use AI smartly moving forward. I'm Rana El Kaliouby, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution. So Jeremy, thank you so much for being on the show.
Oh, thank you so much for having me, Rana.
I love that you're in the interviewee seat today.
Yes, it's unusual for me. I'm usually the one asking the questions.
Yeah, you're usually the one asking all the tough questions. So what was the impetus for writing Mastering AI?
Yeah, so after ChatGPT debuted, Fortune assigned me a large cover story about OpenAI and the process that had led to the creation of ChatGPT, which came out in January 2023. And after that story, I got a lot of interest from people saying you should write a book on this. And I'd always wanted to write a book on AI. I thought it was a fascinating subject, and it just seemed like sort of the right time to do it.
But I wanted to write a broad book. I wanted to talk about all the impacts I thought AI was going to have on both business, but also broader than business, on society and on ourselves, and on democracy, and on culture. I I saw all these, you know, potential implications for the technology.
And I love that about the book. So you write about how this moment in AI is different in the book. AI is what we call a GPT, a general purpose technology, which is, by the way, not the GPT in ChatGPT. The GPT in ChatGPT is generative pre-trained transformers. But yeah, we're talking about AI as a general purpose technology, like electricity. What makes AI different from all these other earlier innovations?
So I think there's several things. One is it is a general purpose technology and those don't come along all that often. It's much more like the invention of electricity, which is also a general purpose technology, or steam power or computers at large, than it is the mouse or the graphical user interface or something which were much more specific technologies. So that's one reason that it's different is it impacts a whole breadth of things. But I think the other thing that's really different about AI is the speed at which it's developing.
So the capabilities are being pushed extremely quickly, and we've already seen, you know, the capabilities of models expanding very rapidly. The technology that's going to be adopted very quickly, much faster than we saw with something like electricity, which took quite a long time to be adopted by business and quite a lot of physical rewiring of houses and all this sort of things. But then the other thing is the speed of adoption by business is very rapid.
So that's the other thing I think is going to really make this different.
You also talk about how AI is kind of encroaching on what makes us uniquely human, and that's another way where it's kind of really powerful, but also scary in a way.
Absolutely, yeah, no, I think that's another differentiator between other general purpose technologies and AI. I think AI challenges our intellectual abilities. And this idea that we are the smartest species, the most intelligent species on the planet, suddenly we have this thing that we've created that is potentially as smart as we are. And I think that is a real challenge to people. It's a real philosophical challenge to people. I think creates a lot of anxiety. I think some of the fears around AI really stem from that.
That sort of existential dread of, you know, something else out there that is as intelligent as we are, but that is not human. People find that scary and they worry about, well, what's my place going to be if there's this software out there that can do everything I can do?
Yeah, I think for a lot of people, when they hear artificial intelligence, they think about sci-fi movies, so they associate AI with, um, I don't know, Terminator.
Oh yeah, Terminator, uh, 2001, there's like lots of them, yeah,
E- exactly, and they are thinking this autonomous, free thinking creature or being. So I wanna talk about another sci-fi movie, which is one of my favorites, the movie Her.
oh yeah, that's a good one.
Her is one of my favorite movies. The protagonist, Theodore, is played by Joaquin Phoenix. He develops a romance with his super smart, emotionally intelligent operating system Samantha, voiced by Scarlett Johansson. It's a modern love story in its own right, and the movie explores the themes of heartbreak and loneliness. It also paints a portrait of what our future relationships with AI could look like as they become more emotionally intelligent. Jeremy doesn't think we have this kind of AI yet, but he thinks that we're close.
We're sort of right on the cusp of that. I mean, the demo for GPT 4.0, which was one of OpenAI's latest models, was very much clearly they tried to design the system to be a little bit like Samantha in the movie. And there's this whole controversy about whether they deliberately tried to mimic Scarlett Johansson's voice or not. Whether they did or not, the point is I think they were very much influenced by sort of the vision of that movie when they were thinking about the interface with which people would interact with GPT 4.0.
And we are moving towards a vision, I think, pretty soon, certainly within the next couple years, where we're probably gonna have these kind of personal assistants on our phone. They'll be useful to us, but we're gonna have to give them a lot of information about ourselves in order for them to be useful. And then the other thing is I think we're already at the point where people are developing emotional attachments to AI chatbots. They're doing so actually kind of deliberately.
If you look at what's happened with one called Replica that's out there, it's a service you can create chatbots with various different personalities. But a lot of people have chosen to create ones that kind of serve as a romantic companion or even in a kind of erotic companion. And people have developed a real bond with these AI chatbots. So much so that when Replica at one point turned off the ability of these chatbots to have sort of spicier conversations, a lot of users were really up in arms.
They were really upset. They felt bereft as if they'd had a lover that died. And I think it shows that we're already, to some extent, in the world that the movie Her envisioned, which is the idea that people would develop these really emotional attachments to these inanimate pieces of software. And I think it's slightly troubling. I write in the book one of my big concerns is that people will find relationships with an AI chatbot just so much easier than a relationship with a real person.
You know, the chatbot has no actual needs, which you have to fulfill. It has no wants, it has no real desires. It may can pretend to have these things, but it doesn't actually have them. And so there's no real consequence if you don't fulfill those. You can always reset the chatbot if it gets annoyed with you or whatever. It's totally different than a relationship with a real person. And I worry that people will use this as a crutch.
And I worry that people will allow their real social skills to degrade and they will choose the kind of easy, slightly addictive, I think, conversations with AI chatbots over the hard work of real human relationships.
Would you let your kids have AI companions or AI friends?
I feel a little worried about it. I think it was okay to have an assistant, but I think we should limit the extent to which we're using them. We need to be kind of cautious. I think there should be some definitely some guardrails around how often you let your kid interact with an AI chatbot, what the nature of that conversation is. As I write in the book, a lot of the studies have sort of compared having an AI chatbot versus nothing.
And it does seem like some of the data is that if you really are a lonely person, that if you have no one in your life to talk to, that having the AI chatbot is potentially a positive, that it does provide some emotional uplift and perhaps there's some mental health benefit. And the problem is they haven't really compared it in any case, there's not been a controlled study to having human friends. But if you already have human friends and then you add this AI companion, I wonder what happens.
And my concern is that maybe you let your human relationships atrophy a bit or you don't work too hard on them. And I don't think that's been studied yet.
And I see a lot of these companies kind of being in a race to capture that intimacy, build an intimate relationship with the users, and there's no guardrails today.
Yeah, no, absolutely. And I think we've we've already been through this experience with social media where we have, you know, basically inadvertently allowed these companies to gain a tremendous amount of power over not just, you know, what we do online, but actually over our minds. They capture a lot of our mind share as well as our market share. And and it's particularly had this pernicious effect on children. I don't think we should wait with this next generation, with this platform shift to AI assistance.
I don't think we should wait to see what the ill effects are. I think we should assume, particularly when it comes to children, that there will be some ill effects of creating products like that are particularly addictive, like an AI chatbot. I think we're gonna need some rules. I think we're gonna need some regulations around this to try to safeguard people from the overuse of an AI system.
So let me switch gears a bit. So the holy grail in AI is this idea of an artificial general intelligence or AGI. And of course, companies like Google's DeepMind and OpenAI were founded with this goal in mind. Can you talk a little bit about what is AGI and make a prediction on how close we are?
Yeah, so AGI or artificial general intelligence is the idea of an AI system that would have all of the cognitive abilities across the board of a human being. And it has been kind of the holy grail of the field since the very founding of the field, but we've never actually been that close. I think we're closer now than we have ever been before, but I don't think that that means that we're sort of AGI is imminent.
There's still a lot of things these systems can't do that we haven't quite figured out. And I think it's actually going to take some algorithmic innovation to probably get there.
Algorithmic innovation, meaning that we will likely need a new kind of machine learning algorithm or a different approach altogether to reach human-like intelligence. For example, AI today doesn't have much perceptual abilities or even memory. Right now, a lot of AI models are based on transformers, a powerful architecture critical in developing models like ChatGPT. But if we want to level up our models, we probably need to level up the algorithmic architectures behind them.
If you go back a few years, there was a definite school of thought among a certain group of AI researchers that all you needed was to keep scaling up the current large language models, which is what the systems that are kind of underpin ChatGPT. The idea is that you could just keep growing these things, get them bigger and bigger, and they gain more capability. And then all the things that they can't do so well right now, which includes, you know, complex reasoning, planning, a lot of common sense things that they don't really have, that those abilities will somehow magically appear as a function of scale.
I think very few people think that's the case anymore. Most people think we will need some sort of further algorithmic innovation. I think it's very hard to predict when that innovation would happen, but I do not see it imminently on the horizon in the next three to five years. I think if you went out ten years, perhaps, or certainly if you went out twenty years, it's possible we would have systems that reach that capability.
We're not in the year of AGI yet, but there is still a lot of innovation happening when it comes to artificial intelligence. So what's the next big AI innovation coming down the pipeline? That, and more, after a short break. Stay with us. The companies winning right now aren't just building great AI, they're building the infrastructure to actually run a business on top of it, which is where Stripe comes in. Stripe is built for whatever comes next, with planetary scale financial infrastructure to help you power any business model.
As a matter of fact, 86% of the Forbes AI fifty run on Stripe, along with industry leaders like Salesforce and Pepsi. Last year, Stripe users processed $1.9 1 trillion combined. See how Stripe can power what's next for your business at stripe dot com. 2023 was dubbed the year of AI chatbots, and 2024 is poised to be the year of AI co-pilots and AI agents. So I I just want to take a moment to define those terms and create a distinction between them.
I'm guessing a lot of our listeners have experience conversing with an AI chatbot. The way I like to think of an AI co-pilot is kind of a a chatbot that can collaborate with you and be a thought partner and kind of help you ideate and get stuff done. And then an AI agent is something that you can delegate a full task to, and it can go act on your behalf and get it done. Would you agree with that distinction and share some examples?
Yeah, and I think of co-pilots particularly as something that's been trained to help you perform a particular professional task. You know, you have a co-pilot for lawyers, and you might have a co-pilot that's different for accountants, and you might have a co-pilot that's slightly different for people who work in finance or medicine. They can give you advice, they can critique your work, it can do sort of first drafts of things for you, potentially act as a bit of a mentor and coach in that professional role.
Whereas, yes, I think the idea of AI agents are software systems that would actually go out and perform tasks for you across the internet. And you could give it a goal, and it would potentially come up with subgoals needed to achieve that primary goal. And it would then kind of figure out what tasks do I need to do in order to accomplish the goal I've been given. And it would go out and start to actually perform those tasks using other pieces of software or using, you know, an internet browser.
We know that both these things are pretty close. So lots of co-pilots are being designed right now and being rolled out right now to businesses. AI agents we don't really have yet, but everybody says they're working on them and there's some indication that they may be debuting even before the end of this year. I think certainly into the early part of next year, we may see a lot of these things getting rolled out. It's unclear to me exactly what capabilities they will have initially, but I do think they will be able to do some purchasing for us.
There's very likely they'll be able to go out and like book a restaurant reservation for us, or book some travel for us, or, um, do some calendar scheduling for us. I think we're right on the cusp of agents that will have at least those abilities. The problem is, uh, you know, anyone who's interacted with a chatbot knows they are good, you know, maybe 80% of the time, and then 20% of the time they provide answers that are really not so great, or they seem very confident but they're wrong.
And I think when we move into a world of AI agency, this becomes even more problematic. Because if it's going to take actual actions with consequences across the internet or using other pieces of software, we're going to have to have a way of checking that what it's about to do is really the right step. And exactly how often it's going to come back to us and ask, Okay, now I'm going to do the following, is this correct?
I mean, if it does that too often, it sort of defeats the purpose of having the AI agent. If it doesn't do it at all, I think there's a real danger. It will run amok and do things improperly and end up spending money actually that you probably shouldn't spend, or performing some action with terrible consequences you don't want it to perform. You know, that is a danger. And I think companies are going to have to think very hard as they deploy these co-pilots about what systems they put in place to make sure that the people that are using these co-pilots remain vigilant and they can perform that kind of supervisory role.
And so actually in the EU's AI Act that is now in full force is that if you have a human interacting with an AI system, the human should always know that they're interacting with the AI system. And I think that's right. I mean, I think we'll all get used to sort of dealing with these AI systems the same way we already are used to dealing with the very kind of primitive systems that were voice-based, where you, you know, robo callers and you would hit one on your keypad if you wanted to do something.
And I think we're already kind of used to that, and I think we'll get used to this interaction with AI agents as well.
Yeah, I agree that AI should always disclose that they are AIs. The AI agent I want, and this is like a true story, I need to schedule my colonoscopy. Um, and I'm just dreading having to sit on an hour call, like waiting for somebody to get, you know, answer me and, like, schedule. I'm like, can we just have an AI agent that has access to my calendar, my medical insurance data, and can just get it done?
When will we have that?
So I'm kind of hopeful we'll have that certainly within two years. It's also, if you think about just scheduling as a lower risk, I mean, you can imagine scenarios where scheduling could go very wrong and have some very negative consequences. But in general, you know, if you put something in the wrong calendar spot for somebody, it's usually not the end of the world.
It's like, how much autonomy do you want to give these agents, and especially if they have access to a lot of your personal information? All right, so let's transition to what this all means for the future of work. And as AI becomes more and more capable, what are the implications on various industries and various jobs? And just to kind of frame this, I'm a big believer that AI has the opportunity to augment and amplify and unlock human potential if we do it right.
First of all, I totally agree. And one of the big pleas in the book is to think of AI as something that's going to augment human potential and think of it as a complement to human labor and not as a replacement and not as a substitute. And I actually think if we can do that, and if we can convince business to do that, we actually take a big step towards mitigating a lot of the near-term risks from the system.
Because if you think about a lot of the things that could go very wrong in the near term with this technology, essentially given the capabilities we have today, they almost all come from the idea of taking the human out of the loop. They come from the idea of substituting human labor one for one with an AI system. And I think it can be a tremendous compliment, but you're still going to need a person working right alongside it most of the time.
And it can automate certain tasks, but it cannot automate full jobs right now. And so it's wrong to think about it that way. So I hope businesses kind of get away from that temptation. I think instead you think about, okay, you can make the person more productive. And if you make the person, the human employee more productive, what else can they do? Or what are the things that the human can uniquely do really well that the AI system can't do?
And can we get the person doing more of that now, or can we expand what they do into some other area that's adjacent to their current job? Again, where the human is going to have a real advantage. And a lot of businesses are going to be able to run themselves more efficiently. They're going to be able to grow faster. Labor productivity has been a huge problem in most of the developed world, most of the developed economies.
Labor productivity has not been growing fast enough. And I think AI technology has the potential to finally kind of get that back on a very healthy upward growth trajectory. And I think exactly how it's going to impact different businesses or different professions will depend a little bit on the specifics of that profession or that business. I think there's a potential in some professions like law or accounting or finance, you might see more boutiques actually being spun off because some of these high earners will think, oh, I can go off and create my own firm.
And we can actually use AI systems for all the support staff that we currently need. Because a lot of these top earners, they're kind of held in place to these large firms because of the need for that massive kind of support base.
So, at a personal level, I love Jensen, the CEO of NVIDIA's quote. It's not going to be an AI that takes your job, it's going to be a person who knows how to leverage and harness AI that will. And in your book, you talk about we spent so much time thinking about how we're training these AI models, but not as much time thinking about how do we train and upskill people to work effectively alongside AI.
Absolutely, and I mean, that's one of the main points of the book. I feel like we spend so much time talking about, yeah, model training and almost none talking about training people. We should try very hard not to just have the human in a completely vigilant role where all they're doing is babysitting the AI system's output. I talk in the book, I spent some time with this woman Jessica Marquez at NASA who is does a lot of their research on what they call human factor engineering, like how do humans work with automated systems?
And it turns out if you put a human in a in this babysitting role, they almost always perform very badly. They can't remain vigilant for that long. So you really need to give the human something to do, and it really has to be a collaborative process. The human still has to be doing something, and as long as the human's doing something meaningful, they can kind of remain alert. Also, we may have to do things where we do drills, just like they do in aviation, just like with astronauts, where you simulate some kind of failure from the AI system.
You know, you actually introduce an error in some sort of safe environment to kind of train the person, you know, what does the error look like, and how do you recover from that particular failure mode? And so you don't also have this de-skilling factor, so people don't lose essential skills. I think we don't want to outsource, like, all our writing all the time, you know, even business writing, whatever. We still want our salespeople to know how to craft a pitch, even if they're using the AI to help craft the pitch a lot of the time.
I think once a quarter or something, we should do a drill where, you know, you ask your salespeople, Okay, now you've got today, you've got to write this pitch completely on your own, not using the AI. So they just kind of maintain those skills, because I think otherwise we're going to be in trouble. And I think we're going to lose things that we want the human to have. I also think you should look at how you can use these systems as a coach.
Often people say, oh, you should conceive of them as an intern. But I think it's a mistake to only look at them in that way. And I think you can also look at them as kind of a coach. And in fact, one of the ways you can avoid de-skilling and get, I think, the advantage of AI is the human should always do the first draft. Instead of outsourcing the first draft always to the machine, and then you act as the editor of that draft or the critiquer of that draft, have the humans write the draft first, and then ask the AI system to critique what the humans written.
And there you kind of can have the system act as this kind of senior mentor and provide coaching advice. And I think you can uplift human performance in that way and, at the same time, avoid de-skilling. So I think that's an interesting way that businesses should maybe encourage their workers to use these co-pilots.
Yeah, there's a quote in this book, and I'm going to read it. While educated elites will retain a high degree of autonomy at work, where they will be empowered by AI co-pilots, blue-collar and service sector workers could find themselves turned into cyborgs, still human, but with their every moment on the job dictated by a demanding and cruel AI manager. Increasingly, human agency will become a luxury item. So intriguing. What do you mean by that?
Well, I think you're starting to see this happen already, and it was before generative AI even came along, just with some predictive AI systems that were being used in things like retail. And they were deciding, you know, which workers worked best with which other workers based on data. And they were the managers of those retail branches, which who used to get to schedule employees however they wished and based on who they thought worked best together, who got along with, uh, you know, which other person.
They kind of lost that autonomy. Instead, they were being told, no, the system, the computer's algorithm is telling you that Mike, he works best on the graveyard shift by, you know, some fractional amount better than what he does when he works the day shift. Never mind that, you know, that's really bad for Mike's health. The algorithm is saying, you know, if he wants to keep this job, he should be assigned the graveyard shift. People don't want to feel like they're in some sort of prison system.
They want to have a certain amount of freedom. So I just think we have to be careful about how we implement these things and really always kind of keep the human and human value at the center and not blindly follow.
I serve on the board of my kids' school, and, like many academic institutions, we're grappling with this question, like, what should we be teaching our kids and what kind of skills are going to be really crucial in this age of AI? So, what do you think? I certainly have a point of view, but I'd love to hear what you think is most important.
Yeah. So I have a couple of thoughts. One is I I do not think you should ban the use of the technology, which some schools have. They said you can't use ChatGPT or you can't use Meta's AI system or or Anthropic's Claude. I think that's a big mistake. I think actually we have to integrate this into the curriculum in some way, but with some clear structures around it and with some clear rules about how the students should use it.
But I do think in terms of skills that we should be teaching, for instance, even though these systems can do a lot of our writing for us, I think we absolutely should still be teaching kids how to write. Because I think writing and thought are not actually separable activities. And I think one of the big dangers of this technology is encourages a kind of belief, I think, a false belief that writing and thought are separable. That you can have these thoughts, you can just put them down in bullet points, or you can just dictate them now with a speech to text systems.
You can literally just talk to your phone, tell it some random thoughts you had and say, go out and write an essay for me that expresses these points. And it will do it, and it might even do a reasonable job. But I think the student's thinking is probably lazy, and there are going to be holes in the arguments that that essay is trying to make, and it is not going to be as good an essay as if the student wrote it themselves.
I think we still need to teach critical thinking skills. I think those are essential. I think we need to teach kids not to believe everything they read and see and to really question what is the source of this information. Part of what the reason we teach research is to teach critical thinking, it's to teach people what is the source of the information I am receiving. And when you've gone out and had to do that research yourself, you become very familiar with what some of those sources are and what the pluses and minuses of different sources are.
I think those things are essential. I think the STEM skills are still going to be essential. I think we're still going to need to teach math. I think science is more important than ever. We're still going to need human doctors and nurses, absolutely. And so we're still going to need to train those students who are talented in those areas for those jobs. And I think that these other skills, you know, emotional intelligence, very important, you know, and social skills, very important.
So I don't think you need to radically change the skill set that you're trying to teach, but you may have to adjust the how you teach them. Some of the pedagogical methods have to change. There's a big debate about computer coding because these systems are very good at coding. But I think actually they they serve best when they're a co-programmer. They don't write the best code out there. And I think the best coders say that they can use these systems to become much more efficient, but they wouldn't rely on them 100%.
And I think there's other skills about logic and reasoning that you learn through coding that are useful. And so we should probably still teach this. Plus, we are going to need some smart people to keep building these systems for us. And that does involve a certain amount of coding, at least for now.
Yeah, I, I, you know, I'm a computer scientist by background, and I I do think the field of computer science is facing a lot of disruption. But to your point, we're still going to need amazing machine learning scientists who are going to think about the next innovations, algorithmic innovations when it comes to all these technologies. So I I don't think computer science is going to become obsolete, but, you know, it'll look different. Personally, I'm not advising my tech obsessed fifteen year old to dedicate his time to learn how to code.
Instead, I want Adam to be AI literate, to understand how these AI algorithms are trained, validated, where they're deployed, and where things can go wrong. We're going to take a short break now, but when we come back, we'll dive into some of the most promising use cases for AI, and get into what we need to be cautious of. I'm a huge believer that AI can help us tackle some of the biggest challenges facing humanity if we do it right.
And some of the areas that you talk about in your book include the potential for AI to transform education. So let's kind of dive into that a bit.
I think it's going to be one of the most transformational areas. And I think one of the areas where AI actually is going to have the biggest positive impact is on education. Because it enables every student to have a kind of personal tutor, and it can adjust to their learning style. If a student's a visual learner, it can provide a visual example. If somebody learns best by listening, it could provide, you know, spoken word as opposed to having to read text.
And I think it can do all this with sort of endless patience that a human's never going to have. And I think that's fantastic and potentially, you know, hugely transformative. Again, I think if we use specific co-pilot systems that have been designed by educational technology companies, maybe based, you know, on one of these large language models underneath it, but have been kind of fine-tuned not to allow the student to cheat, not to just kind of give the answer away, but to kind of use the Socratic method to help students learn.
I think there's tremendous potential there. And they can also provide then to the teacher very specific feedback on that student, to the human teacher. They can say, you know, Sally is, you know, struggling with this particular concept in math right now. And then that teacher can really fine-tune their one-on-one interaction in the classroom. It can also act as a bit of a coach and mentor to the teacher, potentially too, you know, suggesting ways to improve the lesson plan, suggesting different pedagogical techniques.
And all the concern about cheating, I again, I think there are easy ways to kind of avoid that. I talk a little bit in the book about implementing flipped classrooms, where essentially the delivery of the material takes place at home, as opposed to, like, through a lecture in class. And then you do the kind of practice work in school, where the teacher can provide more individual feedback. And what you can have is actually you can have these AI tutors kind of doing the feedback while the teacher kind of goes around and spends a little bit of each hour, you know, with each student.
And I think that would be a great way to kind of, you know, maximize things.
I think what is fascinating about all of this is the changing role of an educator or a teacher, right? It becomes more perhaps of a facilitator, a coach. So I'm kind of excited to see how this unfolds. So another area that we're both passionate about is how AI is accelerating scientific discovery and transforming medicine and healthcare. I'm particularly kind of keeping an eye on the intersection of sensor technology with AI and how it's going to deliver personalized and preventative medicine, hopefully democratize access to healthcare for everyone.
What are you keeping an eye on and what are you excited about?
I mean, that area, personalized medicine, I'm very excited about. I think that's huge. I do think we're increasingly going to be all be wearing various sensors and the integration of that with predictive AI and potentially quite a lot more information also about our genetics that AI will also kind of help model. I'm watching drug discovery very carefully. I mean, huge potential to discover new medicines. I think the issue is will be to see what actually kind of makes it through to market, but there's a lot of things that are now in phase one and phase two clinical trials.
We've yet to have the first AI discovered compound make it all the way to phase three, but we're not that far off with a couple of the early candidates. So we'll see. But there's a tremendous pipeline of these coming, and I think that's going to be transformational as well. I think we're going to see much better treatment for chronic conditions, much better treatment for cancer thanks to AI. I also am excited about AI's potential for material science.
Again, better batteries, new materials that are going to be more sustainable, break down in the environment. And I think there's tremendous potential there, again, using AI to try to uncover these kind of new chemicals. I think we're going to enter in kind of golden age of scientific advancement, in part because we've given scientists this incredible new tool that's going to enable them to go out and discover new things.
So we talked about the incredible potential for good with AI. Let's cover some of the worries and concerns you see. What's top of mind for you when it comes to potential dangers?
I worry already that you see some of the chatbot companies moving towards a business model that might allow people to pay to have the chatbot tell us certain things. And right now, it seems kind of innocent because the deals that have been struck have tend to have been actually with news organizations. And it tends to be news organizations where the chatbot company's actually paying the news organization, but it's for to license their data. But as part of that deal, the news organizations have been negotiating, oh, but we want our news stories displayed more prominently by the chatbot.
And I think, although that sounds innocent and good at the moment, I think it's really dangerous because I think it's this, you know, we're on the slippery slope where, you know, next, it's not going to just be the news companies. It's going to be the big brands out there. And they're going to want to tell people about the latest Nike shoes and the latest HM clothing or whatever it is. And there's going to be some commercial arrangements struck.
And then the one that does get a bit of attention that I also, but I think it's legitimate and I I worry about is disinformation and the risk of political disinformation and conspiracy theories. And that that is, we're already starting to see the effects of that. There have been a number of cases of voice clones and deep fakes that have had influence on elections around the world. They've started to play a little bit of a role in the US.
They haven't played a huge role yet, but I'm worried about that. There have been some very interesting studies out of Cornell University on how persuasive chatbots are. And it turns out they're extremely persuasive, like much more persuasive than human salespeople tend to be. So I think there's a real danger there that if what it's persuading us of is something that is political, there's huge potential for abuse there. And I think we need to be very attuned to those dangers.
And right now, we don't have very good ways of policing some of this. There's people talk about digital watermarking. There are various proposals for standards. None of the standards right now are foolproof. They all can kind of be defeated, but there's also not widespread adoption yet of that watermarking, not wide enough. So I'd say those three are the risks I worry most about.
You've been writing about AI a lot. You have a front seat to how AI is evolving, but how do you use AI in your everyday life? And I guess one of the questions I have for you is, did you use AI to write your book?
Yeah, everybody asks this. And I did not use it to write the book. And people are always really surprised and they say, why not? And I have to say, I at least, you know, the AI systems I was using when I started writing this book about a year ago. I thought that while they were very good for business writing, I actually found for the kind of writing I wanted to do for the book, which was very conversational and I just couldn't get it to write the way I that sounded like me.
I occasionally use it for phrase finding. If I felt like I was overusing a particular phrase or a particular metaphor, I would sometimes ask it for advice on what is another metaphor for this. And it would provide some suggestions, and every once in a while I would take those suggestions, but I did not use it to write my book. I use it, of course, to help me figure out how to help my kids with their homework sometimes.
Because sometimes they get asked a math question and I don't quite understand how to explain it to them, but I'll ask ChatGPT, you know, can you explain this to me? You know, thinking step by step, explain your reasoning. It's pretty good at that. That enables me to help my daughter with her homework.
Although I'm thinking she could go straight to ChatGPT and skip you altogether.
She could, she could skip me. She could skip me. Yeah, she could she could do that. Absolutely.
Right? Jeremy, this was great. Thank you so much for joining us today.
Oh, thank you so much, Rana. Great to be here.
We talked about a lot in this episode. If it left you with questions, let us know. What about AI concerns you? What makes you hopeful? Share your thoughts by leaving us a message at six zero one, six three three, two four two four. That's six zero one, six three three, two four two four. And we might use your voice on the show. Pioneers of AI is a Wait What original. Our executive producer is Eve Trow, our producer is Rachel Ishikawa, and our associate producer is Jordan Smart.
Our senior talent executive is Stephanie Stern. Mixing and mastering by Brian Pew. Original music by Ryan Holliday. Production support from Timothy Liu Lee, and our head of podcasts is Lital Molad. You can join the conversation on LinkedIn, Instagram, TikTok, YouTube, and X. Just search for at Pioneers of AI. Thanks so much for listening.