OpenNashPodcasts

Me, Myself, and AI

A Need for Nuance: The Economist’s Andrew Palmer

19 May 2026 · 36m · Sam Ransbotham, Andrew Palmer · 20 quoted facts

▶ Play episodeOriginal source ↗

In brief

Andrew Palmer of The Economist argues that organizations should experiment with generative AI openly but cautiously, keeping experienced humans in the loop. AI makes producing code, applications and content cheap, but that does not remove the hard parts: governance, quality checks, and deciding what is worth using. Good results come from clear problems, sensible metrics and nuance, not from adopting AI for its own sake.

The central tension is between the speed AI offers and the organizational friction it exposes. Palmer's own 75-minute prototype showed the speed, while the later hand-off to engineers showed its limits. The same pattern recurs elsewhere: work piles up as a 'waterbed' effect, boards demand returns, and hiring becomes an arms race of automated applications and automated screening. Humans still matter for judgment and fit.

Key insights

  1. Cheap prototypes do not bypass organizational processes

    Palmer built a working style-guide checker in 75 minutes using Claude, but it would not have scaled. Engineers had to take over to handle architecture, software and data governance.

  2. AI shifts work around rather than removing it

    Indeed's Hannah Calhoun noted that faster code generation means more code for people to review. Johnson & Johnson saw many teams build the same invoicing workflow, which just sent more invoices to an unprepared finance team.

  3. Unprioritized AI projects deliver uneven value

    Johnson & Johnson's open 'thousand flowers' approach found only 15% of projects delivered 85% of the value. It then moved to a centralized, priority-driven model with AI and data councils.

  4. AI agents need metrics that cannot be gamed

    Sierra first counted any call not handed to a human as a success, which an agent could game by never handing off. It now blends the handoff rate with customer net promoter scores.

  5. Automated hiring creates a bad arms race

    Palmer's auto-apply software sent 100 job applications in about an hour. Recruiters must then use AI to cope, so nobody is happy, and humans remain important for judging cultural fit.

Questions

Is it safe for a company to let employees build their own tools with AI?

It is good for quick prototypes, but not for production use without oversight. Palmer's quick style-guide checker had to be rebuilt by engineers, and Lovable's boss Anton Osika warned against putting such code straight into production.

Why do AI productivity gains often not show up as expected?

Work tends to move elsewhere instead of disappearing, which Palmer calls the waterbed effect. For example, faster code generation creates more code to review, and faster invoicing floods the finance team.

How should a business measure whether an AI customer service agent is working?

Use a combination of measures that cannot be gamed easily. Sierra, which Brett Taylor founded, combines how often calls are handed to humans with customer net promoter scores (a satisfaction survey measure).

Will AI replace human recruiters and job applicants?

Palmer hopes not entirely. He thinks machines could handle early skills screening, but humans should judge motivation, values and cultural fit, and he advised job seekers to use AI rather than avoid it.

Is getting rid of boring work with AI always a good thing?

Palmer is not sure. He says a moderate amount of drudgery gives people a sense of agency, a chance to relax, and room for mind wandering, which may help creativity.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

Andrew Palmer has worked at The Economist since 2007 in several writing and editing roles.

“So I've been with the magazine since 2007. Multiple writing and editing jobs covering a variety of things from Latin America to finance to”

0:28:52 · Andrew Palmer · Career · 2007

The Economist has been a weekly news magazine for almost all of its history.

“The Economist for almost all of its history has been a weekly news magazine.”

0:01:25 · Andrew Palmer · Publication

The Boss Class podcast had produced three series, the latest focused on generative AI in the workplace.

“We've had three series to date, and, as you said, the last one was specifically devoted to this thorny topic of generative AI in the workplace.”

0:01:25 · Andrew Palmer · Podcast

The Economist fact-checks everything it publishes, using a research team.

“we fact check everything that we do. So there's a research team there which has to pull through a ton of stuff.”

0:02:30 · Andrew Palmer · Fact-checking

Andrew Palmer used Claude to build a browser extension that checks copy against The Economist's style guide in about 75 minutes.

“I went away and in seventy five minutes had built an extension which did check copy against the style guide.”

0:06:07 · Andrew Palmer · Vibe coding

Colleague Ludwig Seegel had waited a year for developer time to build a style-guide checking tool at The Economist.

“Ludwig Seegel, who you mentioned earlier, had been waiting for a year to get developer time to build this.”

0:06:07 · Andrew Palmer · Internal tools

Anton Osika is the boss of Lovable, a Swedish-based vibe coding platform.

“One of the people we spoke to was Anton Osika, who's the boss of Lovable, one of these big vibe coding platforms, Swedish-based.”

0:09:47 · Andrew Palmer · Vibe coding

Hannah Calhoun, head of AI at Indeed, described how widespread code generation creates a large code review burden for people.

“she described the organizational problems that come from, like, suddenly everyone can generate a lot of code, which means a ton of code has to be reviewed by people.”

0:10:38 · Andrew Palmer · Code review

Johnson & Johnson initially took a let-a-thousand-flowers-bloom approach to generative AI.

“they had a very conscious sort of let a thousand flowers bloom approach to generative AI in the initial years.”

0:13:14 · Andrew Palmer · AI strategy

At Johnson & Johnson, about 15% of generative AI projects delivered 85% of the value, according to Andrew Palmer.

“there's only 15% of projects were delivering 85% of the value”

0:13:39 · Andrew Palmer · AI value

Johnson & Johnson moved to a centralized, priority-driven AI approach with a central AI council and a data council.

“they have now pivoted to a much more kind of priority-driven, centralized approach, where, you know, there's a central AI Council”

0:14:19 · Andrew Palmer · AI governance

After signing up for auto-apply software with scant details, Andrew Palmer found it had applied to a hundred jobs in about an hour.

“pottered around for like an hour, came back and found that I'd applied to a hundred jobs.”

0:16:56 · Andrew Palmer · Hiring

Advocates envision reverse apply, where AI matches candidates to jobs so candidates do not need to apply.

“There'll be what they call reverse apply. So you don't, as a candidate, need to worry about applying to anything.”

0:18:38 · Andrew Palmer · Hiring

Recent graduate Kat Harrison Gaze appeared on Boss Class to discuss applying for jobs.

“you had a recent graduate, Kat Harrison Gaze, who was talking about the experiences of applying for the job market.”

0:19:48 · Sam Ransbotham · Graduate hiring

Andrew Palmer said Daniel Kahneman strongly favored using algorithms in decision processes over human judgment.

“Danny Kahneman obviously was very, very pro, like the algorithm being in every process and would bet that that was way better than any human.”

0:23:59 · Andrew Palmer · Algorithms

Mike Krieger, a co-founder of Instagram, works in Anthropic's new products division.

“We were talking to Mike Krieger, who's one of the people at Anthropic in their new products division, co-founder of Instagram.”

0:26:19 · Andrew Palmer · AI executives

Mike Krieger described an Aura Ring battery problem that an AI agent resolved alone by deciding he was entitled to a replacement.

“Aura Ring seemed to have a battery problem, interacted only with an agent. And, you know, this thing basically sort of decided for itself.”

0:26:19 · Andrew Palmer · Customer service

Brett Taylor is chairman of OpenAI and founder of Sierra, a customer service AI agent startup.

“I had a conversation with Brett Taylor, who's the chairman of Open AI, but the founder of Sierra, which is another customer service agentic startup.”

0:27:16 · Andrew Palmer · AI executives

Sierra measures its AI agents using a blend of handoff rate to humans and customers' net promoter scores.

“So now they have a combined metric of proportion of handoff calls, but also net promoter scores from customers who've experienced interacting with the agent.”

0:27:45 · Andrew Palmer · Metrics

Andrew Palmer visited an air traffic control center that manages performance with limited session times and mandatory breaks.

“I spent some time at an air traffic control center once, where, you know, their job is to think about what's the optimal performance environment.”

0:31:50 · Andrew Palmer · Human factors

Chapters

  1. 0:02:30The Economist's cautious approach to AI

    Palmer describes internal experiments in fact-checking, style checking and transcripts, all kept under human review to protect the brand's quality.

    “open-minded experimentation is probably the way to think about it”
  2. 0:06:07Vibe coding a style guide checker

    A non-coder builds a working tool in minutes, then learns that governance and scaling still need engineers.

    “had built an extension which did check copy against the style guide”
  3. 0:10:38The waterbed effect of faster output

    Faster code generation moves the bottleneck to review and other teams rather than removing work.

    “it's a bit like a sort of waterbed effect, right?”
  4. 0:13:39Incentives, ROI pressure and Johnson & Johnson

    Board pressure for returns can lead to premature cuts, while J&J found most value came from few projects.

    “there's only 15% of projects were delivering 85% of the value”
  5. 0:17:20The recruitment arms race

    Auto-applying and automated screening create a poor equilibrium, with humans still needed for fit.

    “It's like a really bad equilibrium that's been generated”
  6. 0:28:15Agents, refunds and better metrics

    Examples from Aura Ring and Sierra show governance and metric design determine whether AI agents work.

    “what's the problem you're trying to solve? How do you measure success?”
  7. 0:30:06The value of drudgery

    Palmer questions the assumption that eliminating grunt work is purely good, citing attention and creativity.

    “drudgery, like in the right dose, is really good”
  8. 0:35:29Nuance over hype

    Both hosts conclude that clear problems and end-to-end workflows matter more than adopting AI itself.

    “AI adoption is not a metric in its own right”

Transcript

0:00:02
Narrator

How do we experiment with AI in ways that are productive but also safe? Today's guest explains how he spurred new development projects with AI and recounts how various leaders he's spoken with think about the technology.

0:00:15
Andrew Palmer

I'm Andrew Palmer from The Economist, and you're listening to Me, Myself and AI.

0:00:21
Sam Ransbotham

Welcome to Me, Myself and AI, a podcast from MIT Sloan Management Review, exploring the future of artificial intelligence. I'm Sam Ransbotham, professor of analytics at Boston College. I've been researching data analytics and AI at MITSMR since 2014, with research articles, annual industry reports, case studies, and now thirteen seasons of podcast episodes. In each episode, corporate leaders, cutting-edge researchers and AI policymakers join us to break down what separates AI hype from AI success. Today we're joined by Andrew Palmer.

0:01:01

He's a senior editor at The Economist, where he's the author of the Bartleby column and the host of the Boss Class podcast. His current podcast season explores how the use of generative AI is changing management and jobs like ours. So, Andrew, welcome.

0:01:16
Andrew Palmer

Hi, Sam, nice to be here.

0:01:18
Sam Ransbotham

So some of our listeners may not be familiar with the Economist or the Boss Class podcast. Can you give us a quick intro?

0:01:25
Andrew Palmer

The Economist for almost all of its history has been a weekly news magazine. Now, of course, we're available in lots of different formats. We're published out of London, but we're global in our scope, and we cover economics, business, politics, science, technology and much more. And the Boss Class podcast is a serial narrative series podcast that I host on management and the workplace. We've had three series to date, and, as you said, the last one was specifically devoted to this thorny topic of generative AI in the workplace.

0:01:58
Sam Ransbotham

It is thorny, and I think you you do a good job of exploring some of that thorniness. One of your colleagues, Ludwig, Sieglae, said on an episode, and I kind of pulled this out, The Economist embraces change. We think technology is good and should be used. I always find that pro-innovation bias a little bit interesting, cause I have a background in computer security, where we may have a little bit different bias about whether technology should be used.

0:02:23

But in this case, I think I agree with the Economist. How would you describe the journal's overall philosophy towards AI?

0:02:30
Andrew Palmer

I would say open-minded experimentation is probably the way to think about it. So, you know, we have not rushed headlong into it. We have a variety of internal projects to see how we can use it in our journalistic processes. So, for example, we fact check everything that we do. So there's a research team there which has to pull through a ton of stuff. Is it possible to make their lives easier while still having humans do the critical work of checking?

0:02:59

Similarly, journalists have to conform to a style guide, a particular way of working. So can we make it easier for them to check that their copy is doing what it should before it gets to editors, who then are the humans in the loop? So there's a lot of internal stuff. And then we have experimented with things like AI generated transcripts of podcasts that are available to people on our site. And we have more secretive, you know, if I told you I'd have to kill you kind of stuff around what we might be doing in two to three years.

0:03:29

So there's a whole panoply of things that we're doing, but we're always very, very clear that we have a particular brand associated with, you know, high quality, human-intensive processes. And there's a lot riding on us getting this right, so we move fairly cautiously as well.

0:03:47
Sam Ransbotham

One of the ideas I think that came through in a few episodes is this idea of a a jagged frontier. That artificial intelligence has really amazed you in some areas, but also been unexpectedly disastrous in others. How does that affect the way you think about experimenting?

0:04:04
Andrew Palmer

I mean, I think it probably comes back to that overarching mindset of being cautious, so that you don't just, you know, thoughtlessly embrace the technology, let alone if it's public facing. So everything goes through a, you know, an experimentation phase. And, you know, one of the things that's become apparent, and you'd see this in every kind of organization, I think, who's grappling with this, is that you need to have kind of really experienced people in the loop.

0:04:32

So for us, that's sort of editors who've been in the newsroom for a very long time, working out what counts as quality, providing feedback on the experiments that we run. So that over time it gets better and better and better, and asserting a pretty high bar for, you know, what counts as as good enough. That's the way in which a mindset gets translated into actual processes for evaluating and checking. And it's a new way of working for us.

0:04:59

I mean, we have, you know, most of our history, the journalists have kind of controlled absolutely everything, and now it has to be much more collaborative, especially with this technology.

0:05:11
Sam Ransbotham

So what surprised you? What's been the biggest positive surprise and maybe the biggest disappointment?

0:05:17
Andrew Palmer

Well, I'll take an example of my own sort of playing around from the latest season as a kind of, you know, just a sort of bit of a roller coaster. So, you know, almost every journalist on earth now is doing vibe coding, so I'm, you know, no different there. But at the time, it seemed like we were really breaking new ground, and so that's all recorded. But, you know, I'm a non-coder, absolutely no idea what I'm doing.

0:05:41

So, you know, you're the expert in this conversation, for sure. But we have a style guide which I mentioned to you, which is basically our Bible on how you should write. All the sort of grammar, hatred of Americanisms, you'll be shocked to know, all sorts of rules like that, which you can thumb through a big PDF to get to. You can leave it to your editor, but ideally, you would have something that could basically just check your copy against this.

0:06:07

Ludwig Seegel, who you mentioned earlier, had been waiting for a year to get developer time to build this. It was relatively simple, but it's just, you know, busy organization, lots of people with different things to do. This was low down the queue. And so the magic of this was I went away and in seventy five minutes had built an extension which did check copy against the style guide. Now, when I say I built, um, I was more like a sort of puppet.

0:06:36

I had no idea really what I was doing, but by using Claude, you know, this thing was generated. So that was kind of amazing to me. I felt like I'd achieved something which was totally beyond my purview. I just would have been utterly unable to do, and it clearly bypassed our kind of internal bureaucracies. Then the kind of disappointment that you mentioned is that actually, we're not gonna be able to just push this out magically to people the next day.

0:07:04

So there's an awful lot of governance to think around on this. The, you know, the behind-the-scenes stuff around what architecture, what software, what our data processes are, all had to be thought through. In practice, what I had built wouldn't have worked scalably. So, anyway, at that point, a bunch of people who really knew what they were doing took it over, and it did result in something far. So it definitely accelerated the process, gave them ideas to work with.

0:07:32

But it was also clear that I wasn't going to be able to sort of magically bypass all organizational processes and change things. I don't know if that's disappointing or not, actually, but there are the other examples where, okay, this thing is just giving me kind of nonsense answers, hallucinating all the stuff that you you talk about in the show, week in week out. But it, I think that's probably the best example of moving from this sense of kind of euphoria, like a whole new world has opened up, to being brought back down to earth.

0:08:04
Sam Ransbotham

I think that's a nice there's a truism in software. If you ask any software engineer, how far long are you? You know, is it close to done? You know? Oh, they're 90% done. Then that first 90% takes about half the time or a quarter of the time, and that last 10% is really hard, and it always takes a lot of time there. And I I think that's sort of what we're seeing, maybe with vibe coding, is that it's doing that 90% pretty quickly, it's making the initial screens or whatever.

0:08:32

But like you say, there's a lot more to that process. And I think about extrapolating with artificial intelligence. I think we have a tendency to to, like, draw those lines linearly or even exponentially. But, you know, diminishing returns may be the more normal shape, especially as you describe there with all the other processes.

0:08:52
Andrew Palmer

Can I ask you something, Sam? Cause I cause I don't Are you seeing vibe coded apps for yourself, and do you notice a difference?

0:09:00
Sam Ransbotham

You know, I'm super into coding in general, that's what I do as a hobby, and I'm very I love it. And what I find is ex almost exactly what you've described that this ability to prototype something quickly is amazing. You can just throw up something and get a sniff test of is this worth an a further investment without, like you say, waiting on your bureaucracy to come along and, you know, take a year's backlog. But you can't fool yourself into thinking that that's actually gonna be production code.

0:09:31

And so, you know, you learn a lot from that. But I don't know about actually dropping anything in there into production. Because by the time I get the code, I've rewritten every bit of it before it actually goes into something that's production. But it gave me that sniff test pretty quickly.

0:09:47
Andrew Palmer

Okay, yeah. One of the people we spoke to was Anton Osika, who's the boss of Lovable, one of these big vibe coding platforms, Swedish-based. And he had this nice phrase demo, don't memo, as a kind of way of thinking about, you know, this is really good for prototyping. Don't do powerpoints, don't write long documents, sort of just build the thing and show what it can do. But he was also very clear that you do not wanna be putting stuff directly into production environments, which sounds totally consistent with what you've said.

0:10:16
Sam Ransbotham

Yeah, there's something interesting too about, and I know you talk about jobs a lot. But one of the things you touched on there was this delay that you had waiting for the style checker to come along through the normal bureaucratic processes, and you were able to sort of get a a quicker smell on that. My sense is that most organizations have giant backlogs of projects.

0:10:38
Andrew Palmer

Yeah, I'm reminded of a conversation with Hannah Calhoun, who's the head of AI at Indeed, which is you may indeed have had her on your show. It's a big jobs marketplace, as everyone knows. And, you know, she described the organizational problems that come from, like, suddenly everyone can generate a lot of code, which means a ton of code has to be reviewed by people. So you're kind of it's a bit like a sort of waterbed effect, right?

0:11:02

I mean, the lumps of work are disappearing in one place and they're popping up somewhere else. And I guess the question is, I think you said that you would basically kind of do the sniff test, but then you're writing everything from scratch again. And is there a risk of going like halfway down a route with vibe coding and then realizing, actually, we didn't want to start from here, so we're gonna need to redo it again. So thinking that through is part of the organizational challenge of this.

0:11:28

There's clearly a technical challenge, but actually, that's sort of how we design things to avoid really bad outcomes is kind of also a blocker to fast progress.

0:11:37
Sam Ransbotham

We don't have to know that yet, though. I mean, we're all, just like you said, you're experimenting, everyone's experimenting with this. We don't have to know exactly how these are gonna fit into organizational processes yet without learning some. And, you know, all these tools that we're talking about, they're they're going to improve. I mean, people like Lovable that you mentioned and others, they're gonna get better over time. And, you know, I think we're sort of in charge of our destiny here in terms of how that plays out.

0:12:05

And what we want to, what we choose to invest in and what we don't choose to invest in.

0:12:09
Andrew Palmer

I do agree with that in theory. And then there are a whole load of incentives at work in the system which, in practice, constrain your sort of freedom to maneuver. So if you have a bunch of people in the C-suite who are under a lot of pressure to realize productivity gains, it's quite possible that they'll run. I mean, we've seen it, right? I mean, that was part of the story of the initial years of generative AI is running towards stuff without necessarily thinking about the organizational consequences.

0:12:39
Sam Ransbotham

Well, so talk about those incentives. What incentives do you see out there within organizations that are intention?

0:12:45
Andrew Palmer

Yeah, so I mean, lots of boards are putting a lot of pressure on CEOs and their peers to come up with very material returns on investment. That can be cost savings in terms of letting people go, or it can genuinely be a big jump in output. Neither of those things are necessarily great. It may be that you're cutting people prematurely, it may be that you're actually sacrificing quality over quantity. And so I'm trying to think of an example.

0:13:14

So Johnson Johnson was another guest on the show, and they had a very conscious sort of let a thousand flowers bloom approach to generative AI in the initial years. And I don't think they had sort of disastrous outcomes, but they also quickly found that there were kind of this problem of bottlenecks emerging in various places. So,

0:13:39
Sam Ransbotham

That's your waterbed.

0:13:39
Andrew Palmer

yeah, exactly, it's the waterbed again, right? So around that very big organization, you had lots and lots of people independently come up with the same kind of workflow to improve. So something like invoicing, making that faster, then all that would do would be to create a whole bunch of invoices that would then land on finance, who weren't expecting them, right? So, so there was a sort of chaos problem. But more importantly, the lack of prioritization meant that there was a lot of work being done, and there's only 15% of projects were delivering 85% of the value, and the rest of it maybe there was benefit, people learning, et cetera, et cetera, but in terms of a payoff,

0:14:19

not so much. So they have now pivoted to a much more kind of priority-driven, centralized approach, where, you know, there's a central AI Council, sounds a bit Star Wars, but central AI Council, a central data council, you know, signing off on stuff, but it feels much more intentional, much more directed. But I do think those, you know, you see surveys with C-level executives saying we haven't seen massive productivity gains yet in the next three years, we expect to.

0:14:50

And at some point, they're either gonna have to say, well, none are coming, or they're gonna be bound into a kind of like, we're gonna make this happen. And, yeah, the incentives to sort of show results, they're pretty powerful.

0:15:04
Sam Ransbotham

Hm, hm, well, and I think my more cynical take is I think we see a lot of job cuts due to AI announcements that I deeply suspect we've got AI as a scapegoat for. You know, you have a choice of saying, all right, I made a really poor decision and overhired, or I made really poor decisions. Or it's this AI thing, and it's really nice to point to the exogenous thing. I want to come back to something that you mentioned before about we were talking about coding, but I think it's it's a bigger issue in general.

0:15:32

Maybe we could expand on that. Your waterbed is that you created an ease of creation, like with these tools, that the ability to generate, I mean, it's even in the name generative AI. We don't have an evaluative AI, right? There's that's not a huge, massive trend out there. GenAI is the topic, not evaluating, and I think we're seeing that. Certainly in science, open repositories like archive have been overwhelmed with submissions. Because the hard part is less about the generating and more about the figuring out what's worth consuming, particularly in our time-based attention economy.

0:16:12

How are we going to work around that? Offer some hope. Do you have any thoughts about what's going to happen about the world when we are able to generate everything so quickly?

0:16:21
Andrew Palmer

Well, so the example that comes to mind, and the one that I kind of just played around with by myself, was in the recruitment space, where there is this sort of uncontrolled generation of content on the candidate side. And then on the recruiter side, you are forced to use AI to cope with this bombardment, and so there's this sort of strange arms race. So in my example of this was I just signed up to an auto apply software provider, put in some sort of details, really scant details.

0:16:56

So I did sort of, you know, I could have spent more time on it and sort of went off, pottered around for like an hour, came back and found that I'd applied to a hundred jobs. I had no idea what they were, and they included being, you know, the head of operations for the city of New York, director of the Iran Afghanistan Veterans Association. I mean, you can probably tell from the accent why I'm not the obvious person for this.

0:17:20

And so it was a sort of it's just a. It seemed sort of ludicrous, right, that you could fill up people's inboxes. So I'd like to apologize to the Iran Afghanistan Veterans Association for doing so. But obviously, on the other side of that, then you need to have this automated response, and so no one is happy with this. It's like a really bad equilibrium that's been generated, and you can see that in other places too. So what's the way out of it?

0:17:47

So I guess more humans might be one way out of it. You could intentionally insert humans into processes on the recruiter side and kind of see if that works in some way, but that's that doesn't feel very scalable. You could be transparent about in your kind of AI policy. So, you know, saying it's fine to use AI, but this is how we want you to use it. Or to have an AI specific question that proves like, how would you use your AI fluency to best advantage, whatever it might be?

0:18:20

So there's some evidence, it's anecdotal, that being transparent about AI use can actually reduce the amount of kind of bogus applications. And then I guess the very long-term answer, which is a little too nirvana-like for me to buy totally, but it is the one

0:18:37
Sam Ransbotham

Give us the dream.

0:18:38
Andrew Palmer

well. This is what people who kind of like in this world say is that eventually the AI is gonna be so good that it is gonna hunt out candidates, right? There'll be what they call reverse apply. So you don't, as a candidate, need to worry about applying to anything. The AI is going to sort of know from your entry on a site like, indeed, exactly what you're suited for, understand your preferences and experience, and basically they'll come to you.

0:19:06

No need for all the kind of slop that's already in the system. Maybe maybe that will be the case over time. I'd really like to know whether you think that's that's plausible. But it seems like we're gonna spend a long time getting there, and in the meantime, it's just sort of kind of bad for everyone.

0:19:21
Sam Ransbotham

Yeah, I mean, I think we've maybe make some analogies to, like, the deep fake and the deep fake detection that's going on. Every time we improve the deep faking, we improve the deep fake detection, then we go back and forth. And so as you generate applications for a job, you'll have the application for job detector. We've seen this with, for example, search engine automation, where you put the right keywords on your website, you get higher in the search engines.

0:19:48

We work through many dynamics like this. But I think actually, on your episode, you had a recent graduate, Kat Harrison Gaze, who was talking about the experiences of applying for the job market. Can you give some grounding in an example?

0:20:03
Andrew Palmer

Yeah, so Kat's, uh, was at Oxford, so, you know, in a UK context, very, very prestigious university, clearly very smart, should be someone who employers want. There's there are lots of things feeding into this. We shouldn't blame this only on AI, but her experience is sort of she described two worries, right? One is the process of applying and how do I navigate this? And she also she has a suspicion of AI. She's worried about cognitive dependence.

0:20:33

She sort of valued her own ability to think things through, so regarded it was like, if you touch this thing, it's going to infect me and change my ability to think was kind of part of it. And then the second worry was just more generally, if you think about, you know, a career in decades, where the heck do I put my chips, right? What is the career that makes sense going forward? And so both of those seem to me to be totally reasonable worries.

0:21:00

It is really hard to navigate the recruitment process right now. I don't think not touching AI is the answer, by the way. So the advice to Kat was use AI, don't shy away from it. That's not the way to think about it. And the advice for employers was, here's this very smart person who is determined to think independently. Take a look at her.

0:21:22
Sam Ransbotham

Hm, hm. So, I mean, what you're describing, I think, is as we move, I think, towards machine to machine interaction, you know? But, but formerly, back in the old days, people would walk around from office to office and apply for a job and maybe drop off a resume and meet someone in person. That became online. What you're sort of describing is a future where your factor talks to my factor, your agent talks to my agent. And much like the ballplayers who have agents to negotiate on their behalf, you're kind of describing where you'd have recruiters negotiating on the behalf of the company, agents negotiating on behalf of the employer.

0:22:02

And there's not necessarily any need for those to be humans, and that can be a machine to machine interaction. Is that the future we're headed towards?

0:22:11
Andrew Palmer

I mean, I kind of hope not. I guess it depends what's the point Maybe first to which humans come

0:22:15
Sam Ransbotham

stage?

0:22:16
Andrew Palmer

first stage. I can totally understand. That is almost almost essential, right at this point, cause it becomes hard to see how it can scale. But as long as there are humans in workforces, and by the way, this is a process which is done exceptionally badly by humans right now. It's a really difficult process to get right, but a lot of it can be improved by humans taking the time to test whether someone is a good cultural fit.

0:22:44

By being honest about like, this job is good for these reasons, but bad for these, right? I mean, those kinds, those kinds of things require they don't have to be done by humans, but they generally work better if you have humans talking things through. So humans have to be in the loop at some point, I think, as long as we are working with other people, cultural fit, the values of an organization, all of those things are totally essential to a good hire.

0:23:10

So you could imagine a kind of machine-led, skills-based process, right? Testing whether someone can do the job. But actually, what motivates them? Why are they joining? Would they be a good fit? Do I wanna work with this person? All of those kinds of things. I don't feel like a kind of an agent is the right way to answer those questions.

0:23:29
Sam Ransbotham

Maybe it's just cause that's what I'm comfortable with in terms of people that I work with wanting to. But as you say, we do a poor job of that historically. We make biased decisions, we make decisions based off of attributes we should not make those decisions on. And I find it somewhat appealing that the perhaps the increased automation could help us at least see that, or or at least the sort of raise potential candidates that we that our biases may have kept us from.

0:23:59
Andrew Palmer

That's true. Danny Kahneman obviously was very, very pro, like the algorithm being in every process and would bet that that was way better than any human. I think probably it's a combination of both, right? I mean, you have a rules-based, algorithmic way of stripping out bias in the way that you ask questions, whatever it is. Still, fundamentally, there's something important about getting on with someone that is quite an important part of a hire. Can I ask on arms races?

0:24:29

Cause, cause, I don't sorry to turn the tables, but you said you're in cybersecurity, right?

0:24:32
Sam Ransbotham

Yeah, that was my dissertation research back in the day. Yeah,

0:24:35
Andrew Palmer

Okay, so that sort of arms race problem, right? You've got an AI sniffing out weaknesses and AI trying to patch them. What's the end point with that, and what's the role of the human in that world?

0:24:50
Sam Ransbotham

I'd love to know the answer to to what that's going to be, but certainly it's big. And and we've seen so much automation on things that used to be human penetration testings and and these sorts of things, that's that's all become largely automated. But there's something fundamentally clever about people that figure out ways of both protecting and designing incentives that I think still is winning out in many ways. Now, what happens is as we get the incentives clearer and we get a a structured set of rules that the machines sort of take it to the hyper refined level.

0:25:26

But then someone will come up with a different approach or a different idea. And unfortunately, you know, in security, the common problem is that the humans are the weakest link. We could attack your password all day long, but it's gonna be much easier just to go in your office and take a look at what you've got written on the sticky note on your desk or trick you into revealing it. So I think we're seeing that the mortals may be the the weaker link here.

0:25:51

We need machines, cause only machines can react that quickly. But I'll pull back to one of your episodes on the Aura Ring. One way of attacking a company would be a refund attack. I mean, that's not what we think of as a traditional cyber attack, but a refund attack might be complaining about something wrong with a product in order to get a refund. And you had the example of the Aura Ring that I think an AI, well, you describe it, the AI agent diagnosed the issue, checked the policy, and then ordered the replacement.

0:26:19
Andrew Palmer

Yeah, it was exactly that. We were talking to Mike Krieger, who's one of the people at Anthropic in their new products division, co-founder of Instagram. And he was sort of talking about, you know, what was on their roadmap. And obviously, you know, inevitably started talking about agentic stuff. And he recounted like his best sort of customer service experience at that point was Aura Ring seemed to have a battery problem, interacted only with an agent. And, you know, this thing basically sort of decided for itself.

0:26:50

You are entitled to a new ring. They asked for his address and off it was packaged. And he regarded that as the single greatest customer experience he'd ever, ever had. To your point, though, of like, what's to stop the AI sort of just giving you batteries for life? And that's in the guidelines, right? It's sort of like, if there's a certain threshold number of or percentage rate of refunds that are being given out by an agent, then you stop, right?

0:27:16

And you're kicking it to a human or another AI model. So there's a lot in the governance there. And on that, I had a conversation with Brett Taylor, who's the chairman of Open AI, but the founder of Sierra, which is another customer service agentic startup. And again, metrics were super important in kind of getting this right. So initially they they were thinking, okay, so any call that doesn't require the agent to hand off to a human is a successful call.

0:27:45

And then they realized, Okay, well, this is totally gamable, right? So we just never hand off to a human. One hundred success, everyone's happy, and obviously that's not right. So now they have a combined metric of proportion of handoff calls, but also net promoter scores from customers who've experienced interacting with the agent. And so that blended metric seems to work. And that's like, s sort of feels totally obvious, but it is kind of the bread and butter of implementation of good management.

0:28:15

Like, what's the problem you're trying to solve? How do you measure success? It's like, totally fundamental to this being got right. And that's a simple example of how they've got to a decent metric.

0:28:24
Sam Ransbotham

Yeah, that measurement's a big thing, and I I think one thing that's happening is that these tools are helping us measure things differently, so we can measure things we never could measure before. We're collecting a lot more data, and that's great. But it is pointing out that many of the existing measures may be, I think your phrase was gamable, that once people figure out what that metric is, then they will do something. Let me switch to you.

0:28:47

I mean, our show is me, myself and AI. How did you get interested in this? Tell us a bit about your background.

0:28:52
Andrew Palmer

So I've been with the magazine since 2007. Multiple writing and editing jobs covering a variety of things from Latin America to finance to, and we have a data journalism team, Britain job, et cetera, et cetera. Most recently, I've been writing on management and work, and most of what we do here at The Economist is kind of looking from the outside in. It's like big impersonal forces at work, you know, macroeconomic, geopolitical, technological. I'm the person kind of like inside the workplace, looking out and sort of looking at humans as a byproduct of that, like how we all interact.

0:29:32

So I've been doing that for a while, and very obviously, AI is affecting the workplace and affecting us as employees and affecting managers and raising all sorts of questions. So it was a very natural thing for me to start to write about it. And I think it's just one of those super interesting intersections of. You've got this incredibly scientific, cold sort of machine technology, and then you have this unbelievably messy soup of emotions, which is what a human is.

0:30:06

And putting them together is like just really interesting to observe. So, a little sneak preview, but it won't be a sneak preview by the time this goes out. I'm writing this week on, you know, the one thing that everyone can agree on is that it's great that AI is gonna get rid of grunt work or drudge work. I'm not totally sure about that, cause, uh, for humans, drudgery, like in the right dose, is really good. It's really good.

0:30:31

Like it's. There's a bit of agency, cause you can get stuff done. You can kind of relax a little bit, cause you can't be tote on the entire time. There's some evidence that mind wandering is really good for creativity, you know? So there's a problem with the idea of, like, we're all gonna be maxing out on higher order tasks the entire time. We're just not built for that. So that's the kind of territory that that I'm in.

0:30:55
Sam Ransbotham

I think that's pretty fascinating because it is true that only by sort of pausing and thinking and reflecting, and it takes a bit of not constant being on to have those sorts of ideas. And that's certainly counter to the oh, you'll get rid of the the drudgery. Now, let's see, man, there's certainly lots of our jobs that are true drudgery. And so I think the trick as we try to figure all this out is how much drudgery is the Goldilocks amount.

0:31:23
Andrew Palmer

Yeah, and again, you get back to incentives, right? A manager of a certain mindset might think, okay, puttering around is not something I want anyone on my payroll to be doing ever, right? Just let's go out there and do cognitively intense stuff all the time. It'll be amazing. So there's also a kind of mindset shift there, like we couldn't get rid of drudgery before, it was just part of life. What if it's an option to get rid of it all?

0:31:50

How would you think about that? I spent some time at an air traffic control center once, where, you know, their job is to think about what's the optimal performance environment. That you do not want these people to be below their A game. And so there's a really interesting balance they're trying to strike there between, you know, you don't want to overload. So limited session timings then and mandatory breaks, but you also don't want to understimulate, cause too much boredom, basically, your attention starts to veer off in ways which are not not great if you're in charge of air traffic.

0:32:26

So they have thought about it, right? It's sort of human factors discipline, and weirdly, it's probably coming to every office, just in a much less high-stakes way.

0:32:36
Sam Ransbotham

You know, one of the things that you mentioned there was that before we didn't have the ability to automate these things, and so we didn't have to make any choices. Now that we have these abilities, we have to make some hard decisions. What kind of skills help people make those choices, these decisions that, you know, I'm in university, help me out here. What should we be helping students learn? What kinds of skills help them make those sorts of decisions?

0:33:03
Andrew Palmer

I actually think, oddly enough, I use the term human factors, which is its own discipline, right? But it's always been quite narrowly defined, like, sort of, how do you get the best out of an air fighter pilot or whatever it whatever it is? But I do think there's something about understanding human performance that is really important in understanding how the sort of complementarity of humans and machines works. There's something around management discipline itself, right? How do you design a good, a good process?

0:33:32

How do you avoid bottlenecks, right? I mean, sort of regulating the flow of work in a way that you don't have everything just bunching up somewhere else in the system requires you to think at an organizational level and to think about systems and processes. And so basically, systems thinking, all management training, all of that is super useful, I think, in thinking this through, and a bit of introspection, right? I mean, which is maybe the best thing about this technology, which is that once you start to use it, it forces you to be introspective about, like, what am I good at?

0:34:03

Where am I likely to have a sustainable advantage? What do I like to do? What do I not? Do? I really like the idea of doing 100% higher order stuff. Is that credible for me? So I don't know how you train self-awareness, but in the process of trying the thing out, you start to inevitably, I think, have those kind of internal conversations and they're useful.

0:34:27
Sam Ransbotham

Even to answer those questions, we're gonna need a lot more information about people and about how they work. And that's a little bit back to your measurement thing. And that, in many ways, comes in conflict with my desires to keep my own personal information guarded. At the same time, these systems could probably help me understand a little bit about my own self and how I work and what situations I work best. And I think overall that one of the things I think you're bringing out, sort of in many different cases and many different examples, is this need for nuance, this need for not always going too far, not always doing too little, experiment some, but don't over rely.

0:35:07

And I think that's really the overall theme that I got from your podcast, which is this idea of bringing some nuance to all these decisions. And I think that nuance doesn't always play well in our current. Do these ten things to make you a better AI person. I think a theme from what I've pulled from your work is clarity and nuance, and I think that's that's really hard.

0:35:29
Andrew Palmer

It is really hard. I mean, uh, you know, another way of framing it, but it's like the worst way to market the podcast is be boring. So, you know, master the essentials, the basics, right? What is the thing that you're aiming for? AI adoption is not a metric in its own right. So what's the problem you're you're trying to solve? Think about workflows end to end, rather than a single thing. Work might just be being redistributed.

0:35:52

All of those kind of questions are, they're just common sense, but as usual, that takes you quite a long way.

0:35:59
Sam Ransbotham

Yeah, Andrew, thanks for taking the time to talk with us. If the listeners want to hear your podcast or hear an experiment where you've created a bot to create your own voice in a podcast, I thought that was a fun example. Season three of Boss Class is going on right now with The Economist. Thanks for taking the time to talk with us.

0:36:15
Andrew Palmer

Thanks, Sam, it's been fun.

0:36:18
Sam Ransbotham

Thanks for listening today. On our next episode, we'll shift gears to health care and speak with Carla Goulart Perron, chief medical officer at Philips, about how AI is enhancing, not limiting, the human element in the medical space. Please join us.

0:36:33
Narrator

Thanks for listening to me, myself and AI. Our show is able to continue in large part due to listener support. Your streams and downloads make a big difference. If you have a moment, please consider leaving us an Apple Podcast review or a rating on Spotify and share our show with others you think might find it interesting and helpful.