# The Risks of Too Much AI: Fortune’s Jeremy Kahn - **Show:** Me, Myself, and AI - **Published:** 29 Oct 2024 - **Source:** https://sloanreview.mit.edu/audio/the-risks-of-too-much-ai-fortunes-jeremy-kahn - **Transcript:** https://podcasts.opennash.com/shows/me-myself-and-ai/the-risks-of-too-much-ai-fortunes-jeremy-kahn-19ca0f/ - **Provider:** OpenNash Podcasts (machine-generated) ## In brief Fortune's AI editor Jeremy Kahn argues that generative AI is so fluent and easy to use that people will lean on it too much. That weakens critical thinking, because the tools can't show where their answers come from and make it tempting to skip the hard thinking that happens while writing. He also worries about empathy and social habits, since chatbots only imitate real human connection. The conversation moves from individual habits to what companies and schools should do. Kahn suggests drafting your own work first and using AI as a critic. He says training people to see AI's limits matters more than any single tool, and that adaptability needs practice. The hosts push back that advice like "be flexible" is vague, and argue that company training must fit different uses, not one generic boot camp. ## Key insights 1. **Chatbots hide where their answers come from** Unlike Google, which shows links you can inspect, a chatbot gives a smooth summary with no visible source. Kahn says the system itself often can't explain how it knows what it knows, so users are less likely to question it. 2. **Use AI as critic, not first drafter** Kahn suggests writing the first draft yourself and then asking the AI to critique it like a senior colleague. This gets the benefit of its broad knowledge while avoiding the skill loss of handing it the drafting work. 3. **BCG found AI helped creativity but hurt strategy** In a BCG controlled study with about a thousand consultants, those using ChatGPT did better on idea generation. On business strategy problems they did worse, reinforcing wrong answers through over-reliance, according to Shervin Khodabandeh. 4. **Adaptability has to be practiced, not just advised** Kahn says the only way to become adaptable is to practice adapting, for example by changing how students learn from term to term. He points to NASA, which kept astronauts alert around automation only by introducing random faults. 5. **Companies need tailored AI training, not one boot camp** Khodabandeh argues firms should map which uses are off-limits, which are low-risk, and which need a human in the loop, then train people for each. He says most non-digital-native companies lack this kind of curriculum. ## Questions this episode answers ### Why does Jeremy Kahn think using AI chatbots could make us worse at thinking? He says chatbots give quick, plausible answers that are easy to accept without questioning, and they don't show where the information came from. He also says writing is part of thinking, so having AI write for us can skip the work that sharpens our reasoning. ### What does "provenance" mean, and why does it matter for AI? Provenance means the source of a piece of information, or how an analysis was reached. Kahn says generative AI often can't tell you its sources, unlike a search engine that shows links, which makes its answers harder to check. ### What is the safest way for an employee to use AI at work? Kahn recommends doing the first draft yourself and asking the AI to critique it, like a senior colleague would. He also says employees should be trained on what the tools are good and bad at, including examples of them making things up. ### Does Kahn expect AI to cause mass unemployment? No. He says he is not a big believer in mass unemployment from AI and calls the job-loss fear the biggest misconception, though he expects most jobs to change and flexible workforces to do best. ### Why do the hosts say a single corporate AI boot camp isn't enough? Shervin Khodabandeh says companies should sort AI use into zones, such as no-go areas, low-risk areas and human-in-the-loop areas, and train people for each situation. Sam Ransbotham adds that one-size-fits-all corporate training lacks the personalization the tools themselves offer. ## Facts - **Sam Ransbotham** [0:01:11]: "He's the AI editor at Fortune magazine. At Fortune, he writes its weekly newsletter Eye on AI." - **Sam Ransbotham** [0:01:11]: "He's also the author of a book, Mastering AI, a survival guide to our superpowered future." - **Shervin Khodabandeh** [0:00:44]: "Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies" - **Jeremy Kahn** [0:03:12]: "one of the good things about Google is at least you have these links, so you have to think about, you know, where's the information coming from?" - **Jeremy Kahn** [0:04:17]: "That's why Jeff Bezos had famously kind of banned PowerPoint presentations at Amazon because he was very worried about the kind of sloppy thinking" - **Shervin Khodabandeh** [0:04:47]: "BCG did one of the largest controlled studies with MIT, Harvard and Wharton, where we looked at a thousand strong consultants and they were given chatgpt to work with." - **Shervin Khodabandeh** [0:05:17]: "But on things that consultants are supposed to excel at, like, you know, business strategy, problem solving, like, what is the best strategy for a company in Southeast Asia that's trying to enter a market?" - **Jeremy Kahn** [0:07:05]: "the companies developing the product have no incentive really to get you not to use it." - **Shervin Khodabandeh** [0:10:42]: "we talked about the AI gap. And like the 10% who are getting significant value versus most who aren't" - **Jeremy Kahn** [0:12:01]: "I'm not a big believer in the idea that we're gonna see mass unemployment from AI." - **Jeremy Kahn** [0:12:44]: "I don't think that's gonna necessarily be true in ten years, because I think we will have perfected away some of that, optimized away some of the need for that." - **Jeremy Kahn** [0:15:37]: "Except by introducing faults at like random intervals so that the people were constantly on guard, which was an interesting insight." - **Sam Ransbotham** [0:15:58]: "people followed robots who were clearly demonstrating that they were making mistakes over and over again, and yeah, they still followed the robot even after the robot was exhibiting these mistakes." - **Jeremy Kahn** [0:17:42]: "Particularly, there's some interesting studies about men who have been left alone with Alexa, um, basically the Amazon digital assistant." - **Jeremy Kahn** [0:18:15]: "it's one of the reasons there was actually some pressure put on Amazon to introduce a mode to Alexa that a parent can turn on, where the Alexa will only respond if the child is polite" - **Jeremy Kahn** [0:19:03]: "I think drug discovery is maybe the biggest opportunity" - **Jeremy Kahn** [0:19:39]: "I think there's a great example with that Google Olympics ad where they had the kid writing the fan letter" - **Jeremy Kahn** [0:19:22]: "I wanted to be an archeologist at one point." - **Sam Ransbotham** [0:25:33]: "We've had guests, for example, Katia Walsh at Levi's had a boot camp, Michelle McCracken at Delta, they also had a boot camp." - **Shervin Khodabandeh** [0:26:08]: "let us segment the types of use, let us segment the context, let us identify the various zones that gen AI is going to be leveraged in our company." - **Sam Ransbotham** [0:30:29]: "spreadsheets are what opened up the world of computers to most people. They were part of the killer app that opened up the use of personal computers for most people." ## Transcript **[0:00:01] Shervin Khodabandeh:** Stay tuned after today's episode to hear Sam and I break down the key points made by our guest. **[0:00:09] Sam Ransbotham:** In what ways are we over reliant on AI technologies? On today's episode, we chat with a journalist who shares some of his fears about AI. **[0:00:19] Jeremy Kahn:** I'm Jeremy Kahn from Fortune, and you're listening to Me, Myself and AI. **[0:00:25] Sam Ransbotham:** Welcome to Me, myself and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review. **[0:00:44] Shervin Khodabandeh:** And I'm Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate. **[0:01:11] Sam Ransbotham:** Hey everyone, thanks for joining us today. Shervin and I are excited to be talking with Jeremy Kahn. He's the AI editor at Fortune magazine. At Fortune, he writes its weekly newsletter Eye on AI. He's also the author of a book, Mastering AI, a survival guide to our superpowered future. Jeremy, we're looking forward to talking today. **[0:01:31] Jeremy Kahn:** It's great to be here. **[0:01:33] Sam Ransbotham:** So I'll raise the complaint first. I had planned to just to skim the book and just, you know, try to pull out a highlight or two, but it was interesting. And so it. It unfortunately cost me a lot more hours this this weekend preparing than I than I planned. So that's my that's my angry start to this podcast **[0:01:48] Shervin Khodabandeh:** That's a good complaint. episode. This is this is what Sam's compliments always sound **[0:01:51] Jeremy Kahn:** Yeah, yeah, I was gonna say, sorry, not sorry on that one. **[0:01:55] Sam Ransbotham:** Well, but maybe go negative here. We heard a ton about the benefits of AI, but let's start with some of these risks that you've foreseen. I thought you did a nice job outlining those. For example, you cautioned against letting AI cognitive prowess diminish our own prowess. Can you walk us through how some powerful tool might diminish our own human abilities? **[0:02:15] Jeremy Kahn:** Yeah, absolutely. It's one of the things I'm concerned about is that this technology is, in some ways, so easy to use and so seductive that we'll sort of overuse it. And as a result, lose some of our important cognitive abilities. W- One I worry about is just critical thinking. I mean, anyone who's used one of these chatbots from OpenAI or Anthropic or any of the others, it gives you a very fluid, plausible pat answer and a very capsule summary of whatever information you want. **[0:02:43] Jeremy Kahn:** And I think it's just really easy to kind of take that answer and not think too hard about it and go off and say, Oh, I've got the answer. And in the way these systems are currently designed, there's no real prompting of the user to think about the provenance of the information that they're receiving. And in fact, in some cases, the way these things are designed, there's no way. The system itself doesn't really understand the provenance of the information it's giving you, so it can't actually tell you how it knows what it knows. **[0:03:10] Sam Ransbotham:** Explain provenance, though, just start with **[0:03:12] Jeremy Kahn:** Yeah, so province, I mean, what was the source of this information? Any particular fact, it's telling you, where did that come from? or its analysis, how did it arrive at that? And, you know, people said some of these things about Google when Google came along. There was some criticism that Google was gonna make us stupid. But one of the good things about Google is at least you have these links, so you have to think about, you know, where's the information coming from? **[0:03:34] Jeremy Kahn:** At least there's something there in the UX that kinda prompts you to think about the source of information. If you wanna know more than just the few lines that the link displays, you have to click the link. You don't have to do any of that with a generative AI based chatbot response, and I worry about that. I worry it's we're just not gonna think too hard about where's this information come from. So that's one of the ones I'm concerned about. **[0:03:54] Jeremy Kahn:** The other is the tendency with these LLMs to think that writing and thinking are completely separate activities. I'll just give the system a few bullet points and it will write the whole essay for me, and it will. It will do all the hard work of thinking through the arguments. And I worry about that because, again, I think writing and thinking are not actually separable activities. That it's really through writing that we refine a lot of our reasoning. **[0:04:17] Jeremy Kahn:** Very often you find holes in your own argument when you actually go down to write something long form that you you haven't thought of when you just outlined a few bullet points. And again, there's some tendencies that were already there with other technologies. Powerpoint most famously had this tendency to reduce things to bullet points, often with some negative repercussions. That's why Jeff Bezos had famously kind of banned PowerPoint presentations at Amazon because he was very worried about the kind of sloppy thinking that came in when you just had people delineate their thoughts in bullet points. **[0:04:47] Shervin Khodabandeh:** first of all, I agree with both of your points, and I think the first point you made about is it making us stupider. I think we've actually seen that documented. BCG did one of the largest controlled studies with MIT, Harvard and Wharton, where we looked at a thousand strong consultants and they were given chatgpt to work with. And in certain tasks that included broad creative thinking, ideation, they actually did better when they were told to use the tool versus those who weren't. **[0:05:17] Shervin Khodabandeh:** Cause this expands the realm of possibilities for many in terms of like, how do you create a campaign? or how do you consider this from different angles? But on things that consultants are supposed to excel at, like, you know, business strategy, problem solving, like, what is the best strategy for a company in Southeast Asia that's trying to enter a market? You know, here's the revenues, here's the competitors, et cetera. Gen AI didn't do really as good, and in fact, what we saw is that group, there were sort of a reinforcement of the wrong thing because it was an over reliance, as as you said, on something that doesn't work. **[0:05:52] Shervin Khodabandeh:** So, for sure, we've seen it will take critical thinking away. But I also feel like the paradigm change is when before all these large language models, AI was a tool that would help us predict better or optimize or do things that humans couldn't do. I think, as you say now, Jeremy, we've entered a phase where AI is doing that which humans could do too, and in many ways better. We think of it as more of a coworker than a tool. **[0:06:22] Shervin Khodabandeh:** And so now you have a coworker, and just like with a coworker, you need to challenge them. And when you're having a conversation, you know, I find the best outputs that I get from a gen AI chat is when I really engage with it. The same way I would engage like a young associate or a young apprentice. And you challenge them. And and it often it says, Yep, I made a mistake, I'll go deeper, I'll go deeper. **[0:06:47] Shervin Khodabandeh:** And if you don't take it, as you were saying, at face value, you get to the answer. What sort of worries me in the long term is the first thing you said, which is, are we overusing it? What have humans invented that they haven't overused? **[0:07:05] Jeremy Kahn:** Yeah, I mean, that's a good question. I I think there is that risk. That's why in the book I talk a little bit about why I think there should be very specific design choices to try to prompt people not to overuse it. But of course, the companies developing the product have no incentive really to get you not to use it. So it's a it's a bit of a problem. But, you know, it would be useful if, as part of the design, there was something that sort of prompted you to think about the source of this information. **[0:07:29] Jeremy Kahn:** I wanted to use talk about using it as a kind of a junior colleague, that's kind of interesting. And I actually think one of the ways you can avoid some of these risks is if you don't use it that way. So you do the initial drafting of of the document or the proposal, or, you know, in your example of the strategy for the company, like the consultant would, the human consultant would do that initially. And then ask the system to act as almost like a senior colleague To critique to critique it. **[0:07:53] Jeremy Kahn:** And I think, actually, sometimes you both get the benefits of this large knowledge base. It can draw from. It does sometimes point out things that you've missed or that, you know, you didn't think about. And you don't have the de-skilling risk that you have when you do treat it as the kind of initial drafter of things, as the the one who's gonna do the work of the junior colleague. That's maybe a way that we should train people to use the systems to think about it more as a coach or an editor, um, auditor of our work, as opposed to the initial drafter. **[0:08:23] Sam Ransbotham:** Yeah, that's interesting because I've noticed that my own use of a plain old text editor has gone up versus because I find that, you know, all the spelling correction and grammar correction is a distraction. When I'm trying to think about what I'm thinking about. And so I've gone kind of old school and bare bones here. Maybe I'll be using Vim before too long. So AI is not gonna go away, though. What what can people practically do to avoid this? **[0:08:48] Sam Ransbotham:** I guess you one suggestion is draft it first. Are there other things we could suggest? **[0:08:52] Jeremy Kahn:** I think draft it first. I think training of people about what these systems are actually good at and what they're not. So at least you have that awareness. That should be part of how employees are kind of trained to use these systems as co-pilots, as well as students, as this gets introduced into education that they're taught. What are the strengths and weaknesses of these systems and what can't they do? And, and, you know, show people some examples of how it hallucinates or how it can miss whole lines of argument, maybe because they're not, you know, represented as well in the data set. **[0:09:20] Shervin Khodabandeh:** I agree with everything you're saying, and because I feel the same Well, that's way, I feel **[0:09:24] Sam Ransbotham:** no fun. Come on. like, I **[0:09:25] Shervin Khodabandeh:** mean, no, but. But I there was gonna be a. But because I have children in high school and I see the benefits, I see the risk, right? Like my son was saying, I saw him on it. I'm like, What are you doing? You're doing math? But he says, Look, I don't understand how to solve. This problem is teaching me, and now I'm gonna do it on my own, and then it's gonna critique me. I'm like, great, that's fine. **[0:09:43] Shervin Khodabandeh:** But the point that it can take a significant amount of cognitive load away from humans, and sometimes that's good and sometimes that's dangerous. I think that point is not lost on people. My worry is that in reality, it's not gonna happen because this is very early stages, right? We're talking about something that's two years old that right out of the gate was great, is getting better. It's improved by, like, orders of magnitude, and it has some flaws. **[0:10:13] Shervin Khodabandeh:** But the other thing I see is, I mean, we're talking about individual use of it, right? But when you think about all of these investments are basically done for the enterprise or being used by the enterprise or being consumed by the enterprise. And that's where, like, billions and trillions continue to go in it. I still think that's in very early stages. And what I worry about there is that the gap between the haves and have-nots is going to increase significantly at the company level, right? **[0:10:42] Shervin Khodabandeh:** Like we had, you know, Sam, in our research, we talked about the AI gap. And like the 10% who are getting significant value versus most who aren't, and what do they do? I feel like that's even gonna narrow more because of the sort of exponential power that this new generation has. And I think that the challenge that many companies are going to have is also the challenge of rethinking their skill set and their talent. And, you know, now that I could use AI to do things that a thousand people could do, maybe make it fifty more effective in a coworker mode. **[0:11:23] Shervin Khodabandeh:** What kind of talent do I need? How do I hire them? I think the 10% that we talked about in our prior You're predicting generation of **[0:11:30] Sam Ransbotham:** that it gets smaller. **[0:11:30] Shervin Khodabandeh:** AI, Sam, right, those who get, I think those 10% are going to be those who, in the long term, figure out the talent play and the skill play. And they become sort of new companies, **[0:11:40] Jeremy Kahn:** yeah, I mean, I mean, I think you you're probably right about that. One of the things I highlight in the book is the risk that thems have got got shall get sort of thing. Those that already are sort of ahead and have the largest pool of data and have figured out how to use this technology well already are more likely to continue to pull ahead of others. I think that's true both sort of within professions and across industries. **[0:12:01] Jeremy Kahn:** And I do take your point about talent, but I think you need both. I think you both need a workforce that is going to be extremely flexible. I think that's the lesson here is that, you know, I'm not a big believer in the idea that we're gonna see mass unemployment from AI. In fact, quite the opposite. But I do think most people's jobs are going to change, and you have to be flexible. And I think the companies that have the most resilient and flexible workforces, in some ways, will do the best. **[0:12:27] Jeremy Kahn:** Because, you know, I'm talking about how these systems work right now, and they might work very differently in twenty years. Absolutely, they will. And if you're just starting your career, I think you have to be prepared to be riding this wave of this evolution of this technology. Today, you know, it would be helpful to you to have quite a lot of prompt engineering skills the way the systems **[0:12:43] Shervin Khodabandeh:** that's right. **[0:12:44] Jeremy Kahn:** work right now. I don't think that's gonna necessarily be true in ten years, because I think we will have perfected away some of that, optimized away some of the need for that. **[0:12:54] Shervin Khodabandeh:** When I was listening to you, Jeremy, my mind went into not just the workers being reskilled or retrained, or maybe just new talent coming in, like companies hiring people that they normally wouldn't hire because they have spikes and things that AI doesn't have. But my mind went into actually the younger generation. And I was gonna ask you, Sam, like, as an educator, as a professor, what do you think we could do in schools now for these young people? **[0:13:20] Shervin Khodabandeh:** You know, when we went to school, if we thought deeply and we memorized a lot of things, you know, read a lot and do a lot of math and exhibit hard work, we pretty much did well. And I think, **[0:13:30] Sam Ransbotham:** spoken like a true engineer. **[0:13:31] Shervin Khodabandeh:** well, okay, fine for for like science and engineering fields, that, you know, that was the case. And and what do you think this is gonna mean for, you know, high school and college, sort of education disciplines in, like, five to ten years from now? **[0:13:47] Sam Ransbotham:** Yeah, sure. And I have kids approximately the same age, and you know, we deal with university students all the time. And I worry about this advice to be flexible. I mean, not that I disagree with it, but I wonder if if it has about the same amount of substance that buy low and sell high does. You know, I think no one's gonna take the the counterargument that, oh yeah, you should be more rigid and less flexible. **[0:14:09] Sam Ransbotham:** But how how do we get them to do that? I mean, I have some techniques that we use in class, but Jeremy, how do we go beyond? Sort of a yes, be flexible? Is there something that people should do? Someone's listening right now? What should they do tomorrow to be more flexible? **[0:14:23] Jeremy Kahn:** That's a very good question, and I'm not sure I have really great answers for that. Um, except that I think people should have to change fairly frequently. I mean, I think the only way to become adaptable is to actually practice adapting. I then I think some jobs are naturally more like that than others. Some have kind of rote processes or ways of doing things and they don't change very often. Or a person's not asked to change very often what they do. **[0:14:48] Jeremy Kahn:** Or, and I think some others have to kind of learn on the fly. You have to constantly evolve. Maybe you're facing different challenges or questions every day. And I think for students, you could actually force them to learn in different ways at different points, you know, even year to year or term to term. But I think that sort of thing, like building that into the educational system, would be a way to teach people to to change and to adapt. **[0:15:11] Jeremy Kahn:** I don't really know other ways of doing it. I spent some time talking to folks at NASA about how they try to teach astronauts to work alongside automated systems. In the book, I was talking about this issue of about vigilance. If you're constantly just putting the person in this mode where all they have to do is kind of babysit an automated system, humans are really terrible at it. Um, so how does how? And NASA tried all sorts of different ways to get people to be better at it, and they they ultimately couldn't get people to be better at it. **[0:15:37] Jeremy Kahn:** Except by introducing faults at like random intervals so that the people were constantly on guard, which was an interesting insight. But I think maybe we have to do something similar in terms of training students or the workforce, where you kind of artificially introduce disruption and change in order to teach people to get used to disruption and change. **[0:15:58] Sam Ransbotham:** that's an interesting idea. It actually reminds me, I think you had an example in the book about a experiment at Georgia Tech, which I, of course, noted, cause that's my background. But people followed robots who were clearly demonstrating that they were making mistakes over and over again, and yeah, they still followed the robot even after the robot was exhibiting these mistakes. And I I thought that was pretty, pretty fascinating, you know? One of the things you're saying is it's incumbent on an educators in the educational system to introduce doubt in these systems and introduce, you know, the idea that they will fail. **[0:16:29] Sam Ransbotham:** That makes a lot of sense. And if I guess if NASA does it, then I believe it. **[0:16:32] Jeremy Kahn:** I think we could draw a lot of lessons on, you know, from both the space industry and the aviation industry, which have worked alongside these automated systems of various kinds for decades. I think one of the the most undervalued aspects of this is what they call human factor engineering or human computer interaction. I think those things are really gonna come into into prominence. **[0:16:52] Sam Ransbotham:** One of the things about the interacting with the device versus interacting with the people is this empathy question and it, in some ways, interacting with a device makes us less empathetic. **[0:17:07] Jeremy Kahn:** Yeah, I I speak a lot about empathy in the book. It is ultimately, I think, the thing that is uniquely human, the fact that we can relate to one another based on our own lived experience and relate to others lived experience. And the problem with AI technology is it has no lived experience. And chatbots can mimic empathy pretty well. They can sound sympathetic, empathetic, but it's not authentic, it's ultimately kind of a fraud. It the empathy isn't real, and I think we need to keep in mind this distinction between real empathy and its imitation. **[0:17:42] Jeremy Kahn:** And I worry, particularly with people using chatbots as kind of substitute friends, as kind of companions, that this is another kind of disc. And never have to talk to anybody else and never have to deal with real humans who are, you know, much messier creatures to deal with. Particularly, there's some interesting studies about men who have been left alone with Alexa, um, basically the Amazon digital assistant. And, you know, it's an environment where they didn't realize they were being recorded in any way, and they were really abusive to, particularly the female voiced AI Alexa assistant. **[0:18:15] Jeremy Kahn:** Very quickly, things descended into kind of misogynistic language and talk. And then there were some studies of those men and their interactions later with real people, and they carried over sometimes some of those habits of dialogue to real relationships. And I think that's dangerous, and people are very, very worried about that with kids. Actually, it's one of the reasons there was actually some pressure put on Amazon to introduce a mode to Alexa that a parent can turn on, where the Alexa will only respond if the child is polite and says please and thank you. **[0:18:45] Jeremy Kahn:** Again, that's one of those things we might want to consider in in other AI systems, is, you know, are we using these systems to kind of prompt us to, as humans, to be polite in our interactions? **[0:18:55] Sam Ransbotham:** So let me transition here. We've got a series of rapid fire questions. What's the first thing I that comes to your mind? What do you see as the biggest opportunity for AI right now? **[0:19:03] Jeremy Kahn:** Uh, I think drug discovery is maybe the biggest opportunity **[0:19:06] Sam Ransbotham:** Man, we've certainly seen some examples of that with Covid. I hope we don't have another similar opportunity coming up. What's the biggest misconception that people have about artificial intelligence? **[0:19:15] Jeremy Kahn:** that we're all gonna lose our jobs. **[0:19:17] Sam Ransbotham:** Yeah, that whole fear mongering of the job loss. So what was the first career that you wanted? **[0:19:22] Jeremy Kahn:** The first career I wanted, so I I wanted to be an archeologist at one point. **[0:19:28] Sam Ransbotham:** yeah, so that actually that really ties into your desire for data provenance. Then you can make a connection between the roots of data and the roots of anyway. All right, when is there too much artificial intelligence? **[0:19:39] Jeremy Kahn:** I think there's too much if it's doing things where you want human emotion and connection. I think there's a great example with that Google Olympics ad where they had the kid writing the fan letter and the dad had the kid in the ad. Use AI to write the fan letter. And and there was such backlash against that, and I think rightfully so. I mean, that's something where, you know, you want the kid to do that. That's like this connection with their hero, and there's a lot of learning that takes place in the writing of that letter. **[0:20:07] Jeremy Kahn:** And that's one of these, you know, fundamental human experiences is based on connection between two people. You don't want AI doing that. **[0:20:14] Sam Ransbotham:** yeah, I mean, this is why we have the machines to do these other things, not to do the fun parts. Is there one thing you wish that artificial intelligence could do right now that it can't? **[0:20:23] Jeremy Kahn:** Yeah, I would like it to actually do my shopping for me, which it can't quite do. I think it's going to very soon be able to do that. I'm really looking forward to it. **[0:20:30] Shervin Khodabandeh:** Jeremy, it's been wonderful having you on the show. Thanks for coming on and for inspiring a lot of deep thoughts about the future. **[0:20:38] Jeremy Kahn:** Great, it's been fantastic to be here. **[0:20:46] Sam Ransbotham:** Hey, sure, that was fun. It was a different conversation. We often focus on some of the positives from artificial intelligence, but I think Jeremy had some good risk that didn't go crazy. Existential robots are gonna take over the world types of risk, but some very subtle how AI is changing our behavior types of risk that I think were interesting to think about. What do you think people could learn from what Jeremy said? **[0:21:10] Shervin Khodabandeh:** I mean, just like any piece of technology, you cannot treat it like a black box. The users have to have a certain level of understanding of what are the limitations, what are the risks, what kinds of things can I not worry about the output. And where are some areas where I need to sort of probe and push and go a bit deeper. **[0:21:32] Sam Ransbotham:** I think a certain level of understanding is a problem, though. What is that certain level? I mean, I don't think we're asking people to understand how, you know, I don't know how silicon transistors work. That doesn't seem necessary for understanding the modern world of AI. On the other hand, we are coming pretty hard against the idea of just accepting everything AI says. So I think finding that that middle seems tough. Do you need to be able to make a model yourself? **[0:21:57] Sam Ransbotham:** Do you need to be able to use a model? Where's the line? **[0:21:59] Shervin Khodabandeh:** I don't think the understanding that I was talking about is a technical understanding, but it is a tool that has limitations. And just like when you're driving a car, you need to have some understanding that when you're driving thirty miles an hour and you apply the brake, you'll stop much sooner than if you're driving a hundred miles an hour. And **[0:22:24] Sam Ransbotham:** I think if it's wet, that'll change that equation. **[0:22:25] Shervin Khodabandeh:** and it will change that equation depending on the condition of the road and your tires and all these things. I mean, in reality, some people learn those the hard ways, and that's why they teach these things in driving schools and they teach people you know what to do in case of a skid. I think that's sort of the analog. So while I think, first of all, I don't think the conversation was necessarily dark. I mean, I think there's just so much over exuberance over AI that, like, a little bit of **[0:22:55] Sam Ransbotham:** That's a better way of phrasing like, back to it. **[0:22:56] Shervin Khodabandeh:** reality, I think you'll be good. And and I think, you know, a better way to paint, maybe even, which is not something Jeremy did, but the most extremely potential negative situation to create that dialectic that you could then start thinking of it the other way. But **[0:23:14] Sam Ransbotham:** Switching to your tire example, I think that's a good one. I mean, to me, I don't know, and I think probably you're the same with your engineering background. You, you're thinking about friction, you're thinking about coefficients of friction that change as the surface changes. I'm in the middle of teaching my our sixteen year old how to drive, and there's so much that you take for granted that I have forgotten about. **[0:23:36] Shervin Khodabandeh:** yeah, I mean, honestly, so much of my driving, even now, I think about things that I was taught. And like, when the conditions change, most of them, thank God, are things that I haven't needed to do myself, like experientially. But I was told it's raining, there's a thin film of oil, you will not see it or feel it un unless it's too late, so slow down. But that's my point, like, when you're driving, you're not thinking about those things, but you do know that, hey, it started raining, I'm gonna slow down. **[0:24:13] Shervin Khodabandeh:** Like, that's what most people on the freeway do, like, as soon as it starts raining a little bit, cause they've been told you just have to slow down, cause there's gonna be a film of oil and it's gonna be dangerous. And so **[0:24:25] Sam Ransbotham:** you drive different places than I do, I think. **[0:24:27] Shervin Khodabandeh:** well, what I was gonna say, though, is that be adaptable, I still think, is a goal and an outcome we drive to. But the question is, how do you become adaptable? And I think it starts with education. And I think that, just like with any tool, just like when they taught people how to use spreadsheets, they said, Look, you can have the totals, but then you need to have some kind of a check. And if you don't, you might have a great spreadsheet and it's gonna make a mistake. **[0:24:58] Shervin Khodabandeh:** And if you're linking spreadsheets, I do feel like that is the piece that's going to unlock this level of adaptability and caution, et cetera, which is just by demystifying it and by teaching. By having multiple situations of an answer gone wrong, an over reliance creating a real unrealistic outcome from AI. I mean, there are many YouTube videos where there's a conversation with ChatGPT and it's being convinced that two plus two equals five. **[0:25:33] Sam Ransbotham:** Coming back to your education point, I think that was a good one. Because to be able to do that, to be able to work in these ways, is going to require education. And I think most of our discussion has been about individual level interaction. But you know, what do we think about companies and how they should be educating? We've had guests, for example, Katia Walsh at Levi's had a boot camp, Michelle McCracken at Delta, they also had a boot camp. **[0:25:58] Sam Ransbotham:** What kinds of things are people needing to do to promote that education, and what kind of education should it be? So boot camp is one answer, but even then, I'm not sure what the boot camp should have in it. **[0:26:08] Shervin Khodabandeh:** I don't think the answer is a simple let's everybody have a boot camp. I think the answer is let us segment the types of use, let us segment the context, let us identify the various zones that gen AI is going to be leveraged in our company. There are some no fly zones that strategically it's not gonna happen. There are some places where it has maybe not carte blanche, but it is areas where the risks are so low and and **[0:26:46] Sam Ransbotham:** Fraud detection. **[0:26:46] Shervin Khodabandeh:** it doesn't really matter. Yeah, you don't wanna use it for fraud detection, right? Exactly right. And then there are areas where it is, as always, it's been human in the loop, human and AI. And now what can go wrong? And then what are the guardrails and what is the proper education for the type of user and the type of situation? I think that's the kind of curricula that's needed. **[0:27:13] Sam Ransbotham:** what I liked about what you said was this pushback against. We hear a lot about boot camps, and I don't want to be negative about boot camps, but they strike me as lacking some of the personalization that the tool offers. That we have a variety of cases, a variety of ways. You've listed a bunch of them that we can use these tools and the monolithic boot camp or the sort of classic corporate education doesn't really feel right here. **[0:27:41] Sam Ransbotham:** You I'm sure you have to do these things in your organization, I have to do them too. These, um, I don't want to complain about them too much. I understand why many of them exist, but they're many of these corporate education tools are not particularly tailored to what you know. **[0:27:55] Shervin Khodabandeh:** Yeah, but look, I mean, this dates me as a person. **[0:27:58] Sam Ransbotham:** It's okay, you're in a safe space with me, **[0:28:00] Shervin Khodabandeh:** But like, I do think about spreadsheets, and I do think about the education I went through as a young consultant, learning how to get the best out of spreadsheets. Because **[0:28:13] Sam Ransbotham:** right? You had to miss a total at some point to get that right. **[0:28:16] Shervin Khodabandeh:** but and how to use a formula, or how do you do it better or faster? And as the tool evolved, then more education and more education, and I do actually think those things help. And **[0:28:29] Sam Ransbotham:** Did you get to be a guru of spreadsheets by by classes or by doing? **[0:28:33] Shervin Khodabandeh:** by both, but the classes really helped, and and talking to people who were in those classes really helped. And then it depends on what you're using it for. It wasn't the same, you know, for everyone. I mean, I think most consultants, when they start this, they're in the world of pretty heavy, numbers and spreadsheets, and they do that and they check and debug. And now the thing is so advanced that it's making like debugging itself, and, you know, it's telling you, are you sure you wanna do this? **[0:29:00] Shervin Khodabandeh:** But I do think that right now, this is a black box. And I think there is such a big variance in within an organization in people's understanding of risks and benefits and what it can do and what it can't do. And is it gonna replace me, or is it not gonna replace me? Like, there's just such noise and level of misunderstanding that the only way to break through that is through real, thoughtful design of content and education that's company wide. **[0:29:40] Shervin Khodabandeh:** And I agree with you that I think, like for most corporations, the way they deal with this right now is by creating **[0:29:49] Sam Ransbotham:** kind of a monolithic package. **[0:29:51] Shervin Khodabandeh:** right, exactly, well, creating a monolithic package for education. But I don't think we have yet developed the right curricula at scale for the learning and development of the enterprise. I mean, many companies have done it, you know, we have done it at BCG because we use these tools all the time and and we need to know. But for a typical company, for a non digital native company, I don't think these curricula exist. And I think that this is a real big concern for the chief human resource officers of these companies to put the put in place these kinds of curricula. **[0:30:29] Sam Ransbotham:** Yeah, but it seems time because, I mean, by analogy, I liked your spreadsheet analogy, because, you know, spreadsheets are what opened up the world of computers to most people. They were part of the killer app that opened up the use of personal computers for most people. And I I expect that the large language models are gonna have that same sort of effect with artificial intelligence. That's the gateway, the inroad for most people to artificial intelligence. And and you're right, I think most companies don't have good curricula for that. **[0:30:57] Sam Ransbotham:** We've got a bonus episode coming up in a couple weeks talking about exactly this talent in the world of generative AI, so everyone can pay attention to that. Thanks for listening today. Next time, Shervin and I talk with Rebecca Finley, CEO of the Partnership on AI, please join us. **[0:31:17] Narrator:** Thanks for listening to me, myself and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MITSMR and BCG. You can access it by visiting MITSMR dot com forward slash AI for leaders. **[0:31:48] Narrator:** We'll put that link in the show notes, and we hope to see you there.