Ethan Mollick: AI Urgency, Leadership Responsibility, and Why Companies Must Act Now

Hi everybody, uh, Nicolai Tangen from the Norwegian Sovereign Wealth Fund, and today I am here with Ethan Mollick, one of my favorite professors, a professor at, uh, Wharton, and who is, uh, who was out, uh, not long ago with a book on co called Co Intelligence, Living and Working with AI. And actually, you can see it, um, behind Ethan there, down to down to the right. If you haven't got it, run and buy it, Ethan. If you were, uh, a chief AI officer, uh, in a company for the next three months, what kind of top actions would you take straight away?

So I think that the the most important thing is to get people actually aware of where the state of the art in AI is. I talk to companies all the time, and I think that a lot of executive level people may have tried AI a while ago or didn't use it personally and don't realize how potentially transformative it is. Um, and I think that I I have a sort of general idea that you need to involve your team leadership, you need to involve a, uh, set up a lab that's doing research, and you need to think about how to roll this out to the crowd, to everybody in the organization.

So you've gotta kinda bring the whole company with you, which is not always an easy thing to do.

And how do you maximize the uptake, the uptake in the organization?

So I think that's a it's a really good question, and I think we're still figuring out the answers. But it's kind of like any other thing you wanna do. You have to think about incentives, and you think have to think about leadership, right? So why are people incentivized to use this? Now, the thing that makes AI interesting is everybody's already using it. So there's a new study that just came out that showed in a representative sample of Americans, uh, American workers, the usage went from 30% in February to 40% of people are using AI at work, uh, a little over 40% as of April.

So it's used everywhere. The thing is, people aren't just aren't showing you they're using it because they're hiding it, uh, because they're not incentivized to show you. They're worried they'll be fired if they use it either because they'll be told that they're using it improperly, or because people will realize they're not necessary or there's less workers needed. Um, and so there's incentive problems, and then there's also for everyone else who isn't using it, there's role modeling problems.

What do I do to, you know, to get started? How do I use this? Why is it important? So you've gotta both deal with the skeptics and the people already adopting it who may not be showing you they're using it yet.

Why would people not wanna show that they use it? I mean, here, if people don't use it, I'm like, you know, that's that's bad, right?

Uh, you know, so if you think about the incentives of the average worker who's, uh, who's using AI, first of all, they look like geniuses when they're using AI. And one problem that we see is people don't wanna show you they're using it because, uh, it makes them look like they're not geniuses, like the the people get give the AI credit. The second reason is IT, in many companies, is viewed as a thing that is a cost-cutting measure.

So if I could show you that the AI does some of my work, do I get a reward for that? Or do we you fire employees, right? Am I do I feel safe to reveal it? And, uh, a lot of people are just using it to work less. So you're working 90 less. Am I gonna show you I'm using AI so I have to do more work as a result? Am I gonna get a credit for doing this?

So there's a lot of incentives to not share, probably a lot more than to share.

Well, how do you measure AI usage in a company?

So the the metric thing is really interesting, right? So, uh, what I see a lot of people drive towards we're we're in a world of sort of new innovation, right? So it can be a little challenging, um, to to have a direct, like, easy answer to that question, um, of, like, what does AI do? How should we be using it immediately? So, chief AI officer, you're doing experimentation. That said, I think raw usage actually matters. Um, I mean, the typical KPIs you see are how many people are using our AI app internally.

The downside of that is in most companies I talk to, that maxes out at 20 or 30 of the population. And that's cause an an additional third is probably secretly using AI or and doesn't wanna use your apps, your apps aren't good enough. And another third is kind of waiting for instructions, like they're happy to use it, but they're the opposite of the people who were early adopters. They want clear rules about how I should use it and why.

So I think you sh- can measure adoption by use of your app. I think you can measure adoption by other internal measures of how much you're using AI at work. Um, but I think you have to be aware that there's still this secret cyborg problem of people not revealing their AI use.

Uh, so how are the best companies going about this?

So I'm seeing some really interesting examples of of how this works. I could tell a few of the stories, I can't tell all of them. Um, one example is, you know, radically changing incentives. So I've seen companies that offer 10,000 bonuses at the end of every week to every employee that uses the employee who best uses AI to automate their job, right? And they think they're saving money versus other approaches. I've seen people build this into their hiring process.

So before you hire somebody, you have to your team has to try and use AI to do their job. And then you adjust your, uh, your request for hiring based on that experience. Or before you request money, you need to show how you're using AI to do it. Um, I've Moderna has this really great example that they they put together, they use AI for everything, and what they did was build around the process of, uh, annual reviews.

So basically, they built a whole series of GPTs that help people uncover their own performances, it's improvement needs and what they've done over the year and talk to the right people about their jobs and things. So they could write a really good yearly update about themselves. And they said, Well, if you don't use these GPTs, you're probably not gonna do as well, uh, on your performance reviews, and that will hurt your annual salary. And everybody ended up using this series of things, which introduced them to AI.

So putting these bottlenecks in place where people have to use it, um, thinking about building into internal processes in a way that is po- encourages positive use rather than negative use, those tend to be really effective methods.

So you need a combination of, uh, the stick and the carrot there.

I I think you do, and I think you also need role modeling, right? A leader who uses AI will make sure AI seems critical. Uh, someone who doesn't use AI and says, use it, um, is is is gonna be a problem. I mean, you mentioned the chief AI officer at the beginning. One of the things that worries me a little about that is there is no ability to have a chief AI officer really at this point.

I mean, generative AI has been around three years. There everybody who's on their very first project. So if you hire a chief AI officer, that is often somebody who is actually doing machine learning beforehand, which is great, like AI writ large, but not generative AI. But they're not gonna have any idea of how to make an organization transform with AI. It has to be at the leadership level, I think it has to be at the C level in the company.

But does it make sense to have a central unit which kind of disperses, uh, the best use cases and make sure that everything everybody is is at it?

So that's the concept of the lab that I was talking about earlier. So you do want a a centralized unit for experimentation, um, but that that lab has to be staffed partially by the crowd, by the people who are the best users inside your company. And they're gonna be sort of ambidextrous, they're gonna be shipping out, like, here's a great use case. And also building a full agentic system that doesn't quite work yet that automates the entire job, right?

Um, but I think the traditional view of of having IT produce things, I think that actually is on its way out. Um, some of the most interesting experiments I'm seeing companies are dispersing engineers from the IT department out into work with subject matter experts. Because vibe coding has strong limitations, but it's abso- it's absolute best when you have a senior coder working with a subject matter expert. You can do incredible things in a couple of days that, you know, used to be whole processes.

So I think there's gonna be centralization of some AI functions, but it has to disperse to other parts of the organization in a more decentralized way.

yeah, yeah, yeah, we do we, um, we do that. But w- what about what about, uh, ambassadors and people across the firm being being trained regularly?

So, yeah, and I think one of the things we've been struggling with a lot as we think about AI use in observant at companies has been what training means. So I think one of the one of the things I noticed approach that you take that I think is smartest about training is using the opportunity for contact hours with AI. I think one of the smartest things I said in the book that didn't know was the smartest thing at the time, but has turned out to be quite useful, has been that you just need to use AI to get it right.

You need, you need to bring it to every work task, and you'll figure out what it's good or bad at ten plus hours. And people are very resistant, even very smart, very well-meaning, very, you know, self-motivated people are often very resistant to using this for reasons they can't fully explain, right? It's a weird technology, it's not that easy to use, it's uncomfortable in some ways to sort of confront. I I talk about the need for an existential crisis.

So I think that it's hard to see adoption is strange that way, right? So training can kind of get you into adoption, but we don't actually have that much we can teach people about AI. Like, a lot of the prompting techniques you learn don't really work that well or aren't that important as the model gets better. So it's not that experience, and I think you will develop this idea of of these champions, people, some people will just get it, and it's really important, like you said, to get those people out there.

They're the cutting cutting edge, they're the representatives of the lab in the world.

Will the top performers, well, w- will the people who were top performers without AI, be the top performers with AI?

So this is one of the biggest questions we're facing. If you think about it, there's four possibilities for what happens in an AI world with, uh, uh, unskilled. So the first effect that we saw, and we saw this in our in our study, the we, uh, the Boston Consulting Group study I did with my friends at Harvard and MIT and University of Warwick, where we found big performance gains. And this came out like a year and a half ago, kind of made a big stir.

40% improvement in quality for people who used GPT-4 versus not, um, big speed improvements, and a lot of other studies like that have shown a leveling effect. So bottom performers get the biggest boost. When you really look at what's happening, it's actually the AI doing the work of the bottom performers, right? So the AI is pretty good, so it moves everybody up to the 80th percentile. So one option is it boosts the bottom performers. A second option, um, that could exist simultaneously, is the idea that top performers get some sort of massive returns.

We have a couple of studies show that, but not that many. It's hard to study. Um, there's actually a, uh, one of the best pieces of evidence for that was actually turned out to be a fraudulent paper from MIT, uh, that that didn't, that didn't exist, right? But I think there's a lot of suspicion that top performers using AI can get a huge boost. It's just harder to measure. So there's a possibility that maybe there's 100 times return if you're already a good coder.

Um, and we'll know more about that in the near future. There's also a possibility that, um, that, you know, that AI lifts everybody up, so everybody's performance goes up by a similar amount. And then there is this sort of other possibility that there's AI whisperers who are just good at AI, and they're the ones who get all the returns. So we don't know whether it's concentrating the lower end on the top end, whether it lifts everybody up, or whether there's just sort of magical AI whisperers who are just built to do this.

Um, and then agents are coming to replace everybody, if you listen to the AI labs.

And what do you think? Do you think that will happen?

So I, you know, I think that I mean, I mean, there's,

fi- well, first of all, first of all, just, uh, tell us. W- an agent, what? what exactly does an agent do?

yeah, so, um, nobody has a good definition of anything in AI. So, um, people, if you're, if you are, um, a leader in a company, you will have vendors selling you things with every possible label, and they'll say it's agentic and everything else, because there's no clear definition. But the simplest version, and that sort of overlaps with what the labs think, is imagine an AI any AI system that ha- could be given a goal and could autonomously go and try and accomplish that goal without further human intervention, using its own judgment and tools.

That's what an agent does. So an agent, I would say, prepare me for this podcast, and it would do all the research necessary. We can even demo a little agent right now if we wanted to do that, uh, a little bit of agentic work, um, but it's, uh, it's, it's a so it's a tool that goes out and kind of does things in the world. And the idea with an agent is if an agent could go out and do work for us, then I get to skip the whole problem of trying to figure out how to integrate AI in with my workers, because the AI will basically be a worker.

I'll say, write this code for me, and then deploy it, and then test it and come back. I think we're further from that than the labs think, um, but I do think narrow agents are already very possible. So deeply

Have you got, have you, have you got a have you got a good one available?

I I mean, I think, yeah, we let's, let's let's pick an agent for fun here. Um, I will actually share my screen, and I'm gonna log into an agent while we talk. Okay, so let's let's let's look at a an early version of an AI agent called Manus. Um, Manus is Pa- is, uh, is a small Chinese company. Actually, I don't know how big they are, uh, but they're they are, um, they are the, uh, they use Claude as their underlying data, so Anthropic's data.

And I just want to show you what an agent would do, right? So in this one case, I asked it to come up with 20 ideas for marketing slogans for a new mayoral cheese shop. Select the best one, build a financial marketing plan, build a website carrying cheeses. Um, and so what you'll see is what the model actually does. Here is it comes up with a to-do list, and then it starts coming with a plan. So here it comes with a bunch of slogans and ranks those slogans and then decides on the best one.

Then it goes out and actually does market research on the cheese industry. Then it goes ahead and puts together a whole financial plan for us. Uh, that's many pages long. Uh, then it goes ahead and, um, and and figures out, here's a to-do list that goes through to do this. It does market research, it comes up with a color scheme for the site. And then ultimately, what it does is, without any further intervention from me, uh, launches a website.

And this is 100 created by the AI, right? I can go shop cheeses. There's a little sh- built a shopping cart functionality. Um, I've got a subscription model I can do that has forms built in, and this is without any further intervention from me at all, right? So the idea same. is that this isn't So how? So you know,

how big was your prompt? How was big was your prompt here on this one?

uh, you saw it, this was the the prompt was literally this one, right? Jeez, And right, I mean, that's not that complicated. It figured out all my intent and everything else from this. So this is an example of agent at work, right? Um,

And when will these be widely, when will these be widely available?

I mean, the this is available right now, and I would also argue that, um, O3, which is the model that everyone can use right now, is basically an agentic model already. So if we just go to ChatGPT's O3 and I can just ask it a question saying, you know, um, that, um, uh, you know, um, let's let's ask it to do something. Let's say something like, you know, um, give me ten ideas for a new on-trend, uh, shoe design based on market research, um, uh, and develop pricing, a pricing strategy, and pro forma financials for the best one, and show me a photo shoot of it.

Uh, and I'm not so interested in the prompt being amazing as to show you what I mean by this. So this is, uh, O3 is an example of both a reasoning model, the model that can that goes through this process of thinking. We can talk more about why that's important or interesting, if that, if you want to. But what you'll see, uh, what I want to show you about this is it's come up with a plan, right?

I'm gonna look up latest trends in footwear, and then I'll come with ten designs, then I'll pick the best one, then I'll create a pricing strategy. So it's come up with a plan, and what you'll see is that it's gonna go through this stepwise. So just on that command, right, it's going to it's you can see it's doing web searches already about, uh, to do market research. Um, now it's choosing a design based on the research that it did.

So it's this stepwise agentic approach, right, um, that the AI takes, even for just a simple command. This is this is kind of how an agent works, so you can see it's doing multiple web searches. Um, it's it's thinking about what this all means for the financials. Um, it's figuring out, you'll notice also that it made a mistake in doing a search, so it's looking for different things that it did before. That's an agent at work.

So this agent could have done this whole podcast, right, and pretended to be you and me.

So almost right. Um, there's a weird disconnect. Video and audio is a little bit different than, uh, than other approaches to, uh, to AI, um, like large language models themselves, but very increasingly so. Yes, I've there. You can now bring a live agent onto a call, and it has a pretty good conversation with you. Um, and video is live video is getting much better. So if not s now, within the next months, that's a pretty plausible thing to be able to do.

Do you think he would do better than you and me?

No, um, and I mean, you know, not just cause I wanna keep our jobs, right? Um, and I'm not just trying to flatter you because we still have interview left and I want you to look make me look good. But a aside from those issues, right? Um, I think the thing about AI is that mostly it's not superhuman. So you're a very good podcaster, and I would say that anyway. Um, but if you're at the top of your field, whatever you're top at, you're definitely probably better than AI.

And by the way, we got our our shoe here. Um, and so, um, I think that it's not as good as us yet. That's the real question is, does it get to be? and does it get to be good across, uh, uh, is good as us across every field, uh, which we'd call, uh, artificial general intelligence, AGI, a machine smarter than a human expert across every field? And that's the biggest question in AI right now.

Absolutely, absolutely. When will we be there?

Um, so that is a thing I don't know the answer to. If you ask the AI labs, they think two years. You ask AI skeptics, they think ten years, which is a weird place to be for skeptics to be like, Yeah, this is definitely happening, it's just not happening right now.

Yeah, so what? What aspects of human work do you think AI will complement, and what will it replace?

So I think that is a really interesting question. Um, the whole premise of the book is about co-intelligence, the idea that that the machine, you know, works best with a human, and that's the same way, right? Um, and so right now, because the a AI has these, uh, has what we call jagged frontier. It's really good at some stuff you'd expect and not good at other stuff, and it's also missing functions, right? Like, it's hard to have it kind of connect the world together.

If I ask it to prepare for the podcast, I'm gonna get only so far with it. It will probably give me great preparatory advice, but then I still need to go on the podcast and, you know, and have this conversation with you. And I still it's not, doesn't use my email well yet, so I can't let it interact completely with your team. So there's all these jagged edges that make it hard to use as a universal person replacement.

Again, the goal of the AI labs is agents will solve this problem by doing the work for for us. I again, I'm, and I'm a little reluctant to believe that we're gonna be there as fast as they think. But um, I think compliments is is pretty wide. I think the question, uh, uh, compliment, you know, compliments across a wide range of tasks. I think where you're weakest is where it will substitute, but almost at the individual level.

So if you're not good at idea generation, the AI is probably better at idea generation than you. If you're good at idea generation, you will definitely get value out of using AI, but you should probably be using your own idea generation as well. If you're real if you are terrible at email communication, the AI is probably better at that than you. But that doesn't mean that you don't have a role to play in making sure the email is shaped properly.

Yeah, no, I use it for a lot of my emails just to help, cause my English isn't so good, so it just really improves, uh, my emails a lot. Um, now do you think, um, to which extent do is it now being used for CFOs trying to cut cost? And in to which extent is it just amplifying power and, um, helping us to do things better?

So this is a this is where companies get to make choices. And one of the things I worry about with AI is if the leadership isn't well informed in companies about how they work, they view this as another normal technology. Uh, in in the sense of like, this is a cost-cutting measure. So I can increase productivity by 20, so I can fire 20 of my staff. I think there's two things that worry me about that approach, outside of any sort of moral or other kinds of concerns you might have, which is that, first of all, no one knows how to use this, right?

There is no off-the-shelf product that just does things for you with AI yet. Um, they'll, they'll come, but you have to figure out how to use it inside your own company. And doing that requires you to actually have experts figure out how to use it, and the experts are your own organization, your HR department, your R D. So if you start firing people for using, you know, by because AI makes it more efficient, everyone will just stop showing they're using AI, and you're gonna be in trouble.

So I think there's some danger in making a cost-cutting move right away. That doesn't mean people aren't doing it. The second big danger for making cost-cutting is if we you believe we're on the edge of a real revolution in how work gets done, which I I do. Then the idea that you're going to slim yourself down. So if I get 20 per performance improvement, I'll cut 20 of people feels like a really bad solution in a world where everybody else is gonna have 20 performance gain overnight.

And I so I think that organizations that are in growth mode will tend to outperform those who are using this as a cost-cutting technology. But we don't have all the models yet, people are still figuring this out.

hm. When we had, uh, breakfast recently, we talked about, um, the role of compliance or general counsel. Um, how are you seeing that?

So the opinion that will probably get in the most trouble is the two most, uh, not universally, but the two most, uh, risky places to assign all of your AI responsibility to, is, uh, often IT and legal. And that's not true in every case, right? But legal and compliance, the issue is this is a weird technology. Um, a lot of people people know about is based on rumor. So the number of companies I talk to that will refer that where the legal office will refer to an incident where Samsung's data was stolen by ChatGPT, which never happened, right?

What actually happened was that that people in Samsung were worried about ChatGPT using their data, so they banned use in the very early days, you know? So it's all rumor-based right now. These AI models are, you know, I can tell you, they're being used at Moderna and they're being used at JP Morgan. Like companies that are very worried about data use with le- legal restrictions are using it, right? You guys are using AI. There are ways to get around at this point, the legal issue.

So if the legal team is holding you back, it's because they don't fully understand the problem. And on the IT side, there are some incredibly brilliant and innovative IT people out there who will run with this. But the traditional way that IT handles a project, right, is they'll build a product for you around this and check out vendors and do this approach, and they'll keep the they'll make AI and IT technology as opposed to everybody technology.

And that's another danger is if you just like, we need to build an application for AI. Well, we actually need to figure out use cases, everybody needs to be using this to get there. So those are two different danger spots. They're not universal. I've seen some incredible compliance officers who lead directions with AI, but I've also seen resistance happen from there.

Yeah, no, I I think we we got, uh, uh, tremendous, uh, compliance here, uh, really, really good. But it's interesting because th- there are very few cases where a compliance officer can kill a company. I mean, here, if if you hold back the usage, you kill your company. Because competition is just go, you know, pulling apart by 20 a year, and within two years, you're, you're dead.

I, I mean, I, I I think that that urgency you feel is is really interesting. I talk to lots of executives and you see this light switch go off for them. A lot of them are treating this as like they put this down seven levels of their organization, or they've hired a consultancy who's gonna produce a report on their AI readiness. And then you see the executives who kind of get it, and there's just night and day.

Because once you get what's happening here, it's very hard to not feel urgency, um, and you know, and to not be anxious about resistance everywhere. And that's when our in our previous conversation, that's one of the things that struck me was that feeling like, Oh, this is, this is the big one, and we need to figure this out. And organizations that haven't put that on the list aren't gonna be in trouble.

What proportion of companies have got it now, you think?

I am surprised by how quickly the religion is spreading, but not as many as you think. I mean, I I talk to a lot of top executives, I would say, you know, it's gone from like two or three of people, you know, getting it to we're at 20 of executives in a lot of the firms that should feel urgency, feeling it. Um, but that's a pretty big increase in a short time. So, I mean, this technology is remarkably rapidly adopted.

slight change of, uh, um, topic. How do you, um, stay updated through these extraordinarily fast-changing times?

Uh, so I think I'm probably one of the most kind of, uh, you know, current people on, uh, using this. I've got, I've got this virtuous cycle going, right? Which is as somebody who does a lot of work in AI and is influential in the space. All the AI labs come to me. I don't take money from any of them, but they all give me early access to stuff, so I know what's coming. I'm on all these weird private Discords and conversations, I'm on X and Blue Sky and so.

But, you know, I'm a professor who is on sabbatical and spends a lot of time thinking about AI stuff, and I'm having trouble keeping up. Um, I think keeping up is challenging. Uh, the other hand, I don't think you need to that much, right? I actually think if you go with, like, a ChatGPT or Gemini and just use it a bunch, um, those gen tend to have the really up-to-date models, but it's hard to kinda keep up otherwise.

I mean, I've got a newsletter, so people can subscribe to that, um, but there isn't one sort of, you know, great source on this. And AI is still it's sort of like how it's treated in organizations. For a lot of publications, it's like one. It's spread across multiple parts of their organization, so it's reported a little bit in politics, a little bit in technology, a little bit in business. So nobody really has a full picture, I think, including people in the AI labs.

People move from one company to the other, right? Uh, and it seems like these models are not ahead for a long period of time. They are being overtaken all the time by other things, right? Do you expect this will? It is something is this something that will continue?

So a lot of questions about the future are unclear. I think, you know, so the frontier models, the best models, at any one point, there's only a few companies that can afford to make them at this point. And so generally, you wanna stay close to one one of the model makers, right? So the people who make frontier, uh, closed-source models are, you know, OpenAI and Anthropic and Google, by and large, right? There are some other options out there, but those are sort of the three big closed-source ones.

Generally, if you go with one of those, they're they're going to stay on the frontier for the foreseeable future. There's not a reason to suspect they're they might fall four months behind for a little while. If that matters to you, that's what your lab is supposed to be doing in your company is like, how good is the new model? Should we switch over? Somebody else has to be doing that testing 2047. Another thing that always surprised me in companies is how few of them have people assigned 2047 to just working with AI.

Like it just there's lots of other departments that work on things, but there's very few people whose job is to stay on top of these things.

Hm, hm. So, uh, you're in the lead, you get all these stuff, you're invited to, uh, pre-releases. What? When you now look into the future here, what is what's been the biggest surprise for you lately?

So I think the biggest surprise for a lot of us has been this idea of reasoner models that you kind of see here, right? So I showed you a little bit of this as an example earlier. But it turns out that models that sort of think out loud outperform those that don't. This and this very kind of simple trick has increased the ability of AI by a tremendous amount. So I think the capability curve is coming faster than I expected it to, um, and that was a that's been a big surprise.

And then the other side of it, that's been a big surprise, is how fast adoption has occurred. So, um, this is a very fast-adopting technology. According to any historical precedent, we're probably up to a billion people using, uh, you know, ChatGPT at this point. The last numbers they released were somewhere between 500 million and a billion people. There's another few 100 million using other models. Like, this is an insanely high adoption rate for technology that sometimes doesn't work, or, you know, is weird to use, and where we don't quite know what it's good or bad for yet.

And so I I think that the speed of adoption and the speed of capability gain are both faster than I thought.

By the end of the year, what is it that it can do that it can't do now?

So I think the the arbitrary, you know, end of year deadline comes right in the middle of where agentic systems may actually be useful for thing, you know? So we're starting to see these narrow purpose agents work quite well. So if you use AI for, um, you know, for coding work, uh, for example, you're starting to see that be very useful. But the agents where you just say, fix the code for me, still kind of flawed.

Um, a narrow agent, like Deep Research, is a very good research agent already and replaces a lot of what you'd have an analyst do. And it increasingly will go into territory that was lawyer territory and other places of doing research and analysis and pulling stuff together. We're not quite there yet, but it's getting there. So I think the question is how quickly you could just assign an agent, do this job for me. And it does a reasonably good pass that you still need a human interaction with.

And I think we might be there for many jobs, uh, before the end of the year.

When can I say, Uh, hey, I'm going to be in New York first week of November. Please look up the ten best best Mexican restaurants and book a table seven thirty.

I mean, I think we're there already. That's that's not a that's the that's not a hard problem. Uh, the only question is the book a restaurant, and if you use Manus, it would probably be able to email that person or do that. That that's an easy one. Um, and and I think your current AI will do everything. Book the table for you, and that's an example of a connector that we just need to connect. It does voice, uh, in fact, um, Google has been, even before generative AI took off, has a model where you you.

If you try and book a table at a restaurant that, uh, that doesn't take reservations, it will call and have a voice call them, and an AI try and book the reservation for you. So that's already there. But one of the interesting things about AI is right, it's already there, almost right, like you can't just ask ChatGPT to do it. There's a little bit of hoops to jump through. So it's not a capability problem, it's a user experience problem, it's a UI problem, it's a communication problem, and that's, I think, AI ability far outstrips our ability to use it.

now you wrote about, um, cybernetic teammate, um, when? when have you seen teammates most strikingly or perform a human colleague?

So this is again a case of of co-intelligence, right, of working with the AI. So the Cybernetic Teammate paper we, uh, again, with, uh, with my colleagues at Harvard and MIT and University of Warwick and, uh, and at, uh, Penn. We went out and, uh, we did an experiment. Procter and Gamble, where they gave 776 of their employees, and we did real work that they actually do. So, you know, um, whether that's product analysis work or product development or marketing, and we had them either work alone or in cross-functional teams of two, with one marketing person, one business person, and one, uh, technical person.

And what we found was that individuals working alone, um, with AI, had the same performance statistically as teams of two. And they also produced more diverse ideas, uh, than than if they were working alone, and they were happier. So that's the sort of cybernetic teammate idea, right, of working with the AI to do things. I think a lot of people are already doing cybernetic teamwork, right? I I s I know you are. You consult AI about all sorts of things.

Now you have a small team working for you, you know, a panel of experts. So I think that people who use this, this is the natural way to use AI, actually, is as a teammate where it's helping you and you're filling in goals on it. So I think we're we're seeing pretty big gains across the board from that approach.

How can we best develop, um, the people in, uh, in the usage of this? What's the best way to really drive? I mean, we talked a bit about the institution and how you drive the institution forward. How do you? What's the best way to drive the individuals?

So I think that, um, we are still learning about how to do that. I it it. This is not like traditional tr- training in that we give you four or five rules for using AI. I mean, I do think there's some things to learn, but a lot of training focuses on like prompting techniques, and prompting techniques turn out to be less and less important as time goes on. The AI models are fairly resistant.

Why, why? Why is that becoming less important?

Well, two things I think are happening. One is the models are getting better, and larger, better models get your intent better, and the second is it turns out we just don't know a lot about prompting. It's very contingent. We have actually a little study that we did at the Generative AI Labs at Wharton, where we measured, um, the accuracy of AI on answering questions. If you are polite to it versus if you are mean to it, um, and it turns out that politeness matters a huge amount on certain answers.

So if you do a hundred tests on a particular math problem, it turns out in that math problem, if you say please, you get, uh, more accurate answers than if you don't say please, right? But it turns out on a separate math problem, saying please makes it do worse, and yelling at it does better. And we don't know why it works in one case but not the other. So the average effect is zero, basically, uh, but the individual question effect is quite large.

So if we told people, Oh, always be polite or always use this approach, it's sometimes gonna work, sometimes it's gonna backfire. And the best

So I have, so I know I normally say please, so I should just not do it.

I. It doesn't seem to help, but, you know, I, but then again, it's I. I think there's value in the mental model of treating AI like a person, and if please helps you do that, even though it's not a person, then I think it's no problem. And it's just kind of hard otherwise, right? It feels weird to just order your computer around. And then if you, if you talk to people who really believe super intelligence is coming soon, you better be polite now, cause they'll they'll know.

um, how do you? How do you test for, uh, AI literacy when you interview people, uh, in a job interview?

I I wanna avoid over-indexing on today's literacy because we don't have a definition of AI literacy. I mean, I teach AI, and I we don't have a definition of being AI literate. It's tends to be you're an AI user, and then the question is, are you a sophisticated user or not? Does that matter to you? Because it might turn out that they're not a sophisticated user, but a small amount of experience with a sophisticated user will make them good.

It's a if we're judging AI literacy today, you're really judging people's independent ability to go out and figure out ways to use AI in their job. And that can be valuable, but that doesn't always tie in with subject matter expertise and other issues. So I would just ask people at this point, show me, you know, I? When we were hiring people for the Generative AI lab, we both ask them how they used AI. We would give them a task and say, use AI to accomplish this impossible task and get as far as you can in an hour, right?

Um, and we would also be, uh, asking people about, you know, actually interviewing them about what's working, what isn't for their AI use, and having them show us some of that. So, I mean, I think you would judge AI, but that's kind of a example of usage and creativity with AI. And we're an AI lab, so we kind of need those people. I worry at this stage about companies picking AI literacy as the major issue. One of the things we found from the BCG study was that more junior people were actually worse at using AI in the organization than more senior people.

They may have gotten AI, but they didn't understand the organization. So they would produce a document and say, this is a great memo, and then someone with 20 years of experience would look at the memo and say, no, it needs to be better. In these following six ways, they would have been better using AI than the junior person. So I I worry it kind of goes back to that chief AI officer example you gave earlier. Companies are trying to hire their way out of this problem, and I don't think there's a way of hiring your way out of the problem without the executives also engaging deeply and making some decisions about what AI means in their organization.

What personality type do you think will adopt the best?

So the model I I we we talk about agency a lot. Uh, we don't fully know what it means. I I teach entrepreneurship, so it's, uh, you know, there's an entrepreneurial action or entrepreneurial, you know, a tendency to entrepreneurship that I think is the same thing as agency. Which is a feeling that you have control over your environment, uh, kind of locus of control, that you are in control of your own destiny. And you should seek out opportunities to do things that are different or better or improve.

And and that is not something everybody has that seems to play a role in AI, but we don't have measurements of it. So I'm a social scientist. I'm making stuff up here, which always makes me feel bad, cause we don't have a measure. I don't have an AI measurement that makes people that it can tell you. But there is this sort of agentic sense. On the other hand, some of the most entrepreneurial, interesting people I know are late adopters of AI.

For whatever reason, they hit a barrier when they were using it, and I'm not sure what that is. So I, I, you know, again, I. We don't have easy answers to almost any question about AI at this point.

Uh, two, uh, uh, slightly different ones. Uh, how do you see the regulation in the Europe versus the US?

So I'm not a regulation expert, but, um, I, I will tell you that. One of the interesting things that happened with the release of DeepSeek, which is a Chinese open source model, um, R1, DeepSeek R1 specifically, was it was the first time a Chinese model was on the frontier of capabilities. It not beat American models. They're better again at this point, but close. It's a typical closed source, open source. They're a few months behind, but a very good model.

It caused a bit of a bit of a panic in the US that I don't think was warranted necessarily, um, of, uh, even from a great powers competition standpoint, um, but that sort of, um, that. Plus the new administration, I think, has created this opportunity where there's very little regulation. I mean, in fact, one of the, um, one of the bills in front of Congress right now in the US. is that there is, um, would ban AI regulation at the state level for the next ten years.

I don't know whether that passes or not, but I think there is a desire to put the foot in the accelerator and less regulation and more sort of letting AI rip. Um, I think that my impression from talking to people in Europe has been there has been a move to less regulation from a very regulated viewpoint. I think part of this came down to the idea that a lot of the initial regulation was based around existential risk, which I think is important, um, but we don't know how to measure.

And, um, and now I think we have to start moving towards a place where we're regulating harm. So even though there was this, um, there's a bill saying you can't regulate AI, another bill went through Congress, um, banning deepfakes. So I think we're gonna have to regulate the outcomes of AI. But I think that Europe is still a much tighter regulatory environment, but is probably looser than it was a a couple years ago on the AI front.

You mentioned existential risk. Is there an existential risk?

I have no idea, but a lot of very smart people think there is. Uh, you know, for me, it's I. The idea that AI gets smart enough, that takes over the world, doesn't feel feel real to me, but that doesn't mean that it isn't right. And I a as as a as a as a good sort of, uh, you know, social scientist, right? I have to realize my own views are kind of secondary to what is what do the smart people say?

If we have a forecasting contest, and there's a lot of the founders of AI think that it it's an existential danger to humanity, a lot of very smart people think it that way. So we have to view it as an existential risk as well. The problem is we don't know what form that takes, and we don't know what the level is. Like originally, a GPT four or GPT, you know, uh, o3 class model was considered an existential risk level when before we had these things.

Now we have them, we're like, meh, I. These aren't gonna let anyone take over the world or spontaneously go out and and destroy the economy. So is there a magical point of where that happens? I, I just don't know.

Well, Ethan, this has been, uh, uh, forty, uh, incredible minutes. You are at the at the very forefront of the very forefront. Uh, it's been, uh, so interesting talking to you. So, uh, uh, keep up the good work. Look forward to to staying in touch and see what, uh, what the future brings.

Thank you. It was a pleasure being here. And I will say it's it's exciting to talk to people who get the AI thing, because I think it is important and spreading that news is important. We can't go with eyes closed. We have to make decisions about the future, and you can only do that if you're using these things.

Absolutely fantastic.