OpenNashPodcasts

Analyse Asia with Bernard Leong

If AI Models Have No Moat, What Are Investors Buying? with Benedict Evans

24 Jun 2026 · 57m · Benedict Evans, Bernard Leong · 21 quoted facts

▶ Play episodeOriginal source ↗

In brief

Benedict Evans argues that large language models (the AI systems behind chatbots) currently show no winner-takes-all effect. Several firms produce similar models from similar infrastructure, so models look more like commodity infrastructure than a protected business. Durable value is more likely to sit in the apps and products built on top. He says that is why investors in AI model companies must explain where pricing power would come from.

The main tension is between how quickly AI is advancing and how little anyone can predict about where it settles. Evans compares today to the PC in the 1980s and mobile in 2005: exciting, but nobody yet knows the right questions. Chatbots remain a poor interface, models give 'the average', and most people use them weekly rather than daily. He concludes that better products, not better models, are what would show AI has eaten the world.

Key insights

  1. Models lack winner-takes-all effects so far

    Evans says Google's search position or Windows' dominance came from structural advantages that competitors could not buy their way past. He sees no equivalent in language models, where three to six leaders are followed by a dozen firms only months behind.

  2. Telecoms show big revenue can mean poor returns

    Evans notes that mobile carriers earn about $1 trillion a year and spend about $200 billion on capex, yet their stocks have gone nowhere in 20 years. He uses this to show how value can move to whoever builds on top of the infrastructure.

  3. Software survives because of the blank screen problem

    Evans says buyers of software are really buying someone else's worked-out workflow, since most people cannot design their own from scratch. Writing code is the easy part; spotting the problem, distribution and go-to-market are what make a software business.

  4. The jagged frontier makes AI unpredictable for normal users

    Evans says models excel at some tasks and fail at others in ways users cannot predict from the interface. Software companies earn their place by wrapping the model with context, checks and tooling that tell you what will work.

  5. Language models give you the average by design

    Evans says a model produces what most people would probably say. That is fine for a risotto recipe or a standard NDA, but not for something creative or out-of-pattern, such as spotting that listeners want a new sound. Buyers of professional services often want more than the average.

Questions

If AI models have no moat, why are investors still valuing AI labs so highly?

Evans says he doesn't see a clear answer yet. A 'moat' is a lasting protection from competitors, and he sees none in models today. He asks why companies selling similar models would have pricing power, and he points to telecoms, cloud, chips and fiber as outcomes that could apply.

Will ChatGPT-style chatbots become the main way we use computers?

Evans calls the chatbot a poor interface, and the host compares it to DOS, an early text-based operating system. Evans says mass adoption probably needs AI wrapped in apps built for specific jobs.

Will AI take most people's jobs?

Evans says automation has always removed visible jobs while creating ones nobody predicted, and he notes the podcast industry as an example. He sees no clear decline in hiring after three years, and says anyone claiming this time is different needs to explain why.

What does it mean that AI 'gives you the average'?

Evans says a language model tells you what most people would probably say. That works for standard needs like a risotto recipe or a plain NDA (a confidentiality agreement), but it won't come up with something new that people want.

Why can't companies just ask an AI to build their own software instead of buying it?

Evans says most people can't specify the workflow they need. The hard parts of software are spotting the problem, finding customers and building the product around it, not writing the code.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

Benedict Evans says the mobile telecoms industry earns about $1 trillion a year in revenue, spends about $200 billion a year on capital expenditure, and has seen its stocks go nowhere in 20 years.

“it's 1 trillion dollar annual revenue industry pays $200 billion a year on CapEx and the stocks have gone nowhere in 20 years”

0:07:51 · Benedict Evans · Telecoms returns

Benedict Evans says Rock's Law observes that semiconductor manufacturing gets more expensive with every generation.

“What Rock's Law observes, it gets more expensive every generation.”

0:08:41 · Benedict Evans · Rock's Law

Benedict Evans says leading-edge chip manufacturing has narrowed to about three companies or fewer, compared with dozens 30 or 40 years earlier.

“And so now there's basically three chip companies. Whereas, or two, or one, or however you want to count it, as opposed to 30 or 40 years ago when there were dozens.”

0:09:06 · Benedict Evans · Chip manufacturing

Benedict Evans contrasts the fiber bubble, built ahead of demand, with AI infrastructure, which he says is built behind demand because every token produced can be sold.

“Whereas this, of course, is being built out behind demand, like you can sell every token you can make.”

0:09:34 · Benedict Evans · Fiber bubble

Benedict Evans says the fiber market became disaggregated into separate layers such as ducts, dark fiber, light fiber, wavelengths and IP streams, with others trading bandwidth on top.

“So what happened with fiber was that you got split up. So you had ducts and dark fiber and light fiber and wavelengths and IP streams and probably some other stuff.”

0:09:34 · Benedict Evans · Fiber disaggregation

Benedict Evans recalls that as a junior technology analyst in 1999 he visited a company selling computer parts and components online.

“there was a company that sold computer parts, computer components, chips and memory and computers and printers, all this stuff.”

0:11:24 · Benedict Evans · 1999 reseller · 1999

Benedict Evans says the online computer-parts reseller had revenue of about 100 million pounds a year after launching its website.

“So they make a website, they're doing 100 million quid a year in revenue. We all go up to see the company.”

0:12:21 · Benedict Evans · 1999 reseller · 1999

Benedict Evans says Anthropic got coding working first and reached product-market fit with it, with no inherent reason Google or OpenAI cannot catch up.

“so what we see now is Antwapic got code working and got coding working and got product market fit.”

0:03:53 · Benedict Evans · Anthropic coding

Benedict Evans says that late last year OpenAI tried many products at once, including a social video app, app stores and an e-commerce checkout flow, to find what worked.

“we'll try a social video app, we'll try an app store, we'll try another app store, we'll try an e-commerce checkout flow”

0:21:27 · Benedict Evans · OpenAI strategy · 2025

Benedict Evans says a late-1980s program called Sideways sold for about $150 and only printed spreadsheets in landscape, a function later reduced to an operating-system checkbox.

“There was a program called Sideways that you could buy in the late 80s. And it was like $150 in late 80s dollars”

0:23:09 · Benedict Evans · Software integration · 1980s

Benedict Evans says spellchecking was once a standalone software category, with computer magazines running group tests of ten programs priced at $200 to $300.

“they would have a group test of 10 different spellchecking programs that were all $200 and $300”

0:23:09 · Benedict Evans · Spellchecking software

Benedict Evans says US census data show the number of accountants and bookkeepers rising in a near-straight line through the 20th century despite decades of automation.

“You go look at the US census data, the number of people who are called an accountant and a bookkeeper basically goes up in a straight line from the 20th century.”

0:35:16 · Benedict Evans · Accounting jobs · 20th century

Benedict Evans says the internet hurt journalism not by changing the editor's job but by removing the local advertising monopoly of the printing and trucking business that paid journalists.

“journalists were being paid through a light manufacturing and trucking business that had a monopoly on local advertising.”

0:36:28 · Benedict Evans · Newspapers

Benedict Evans says about 200 years ago roughly 90 percent of people were peasants, and automating farming over the following centuries did not reduce the number of jobs.

“200 years ago, most of us were peasants. And, you know, we worried that the crops would fail. And, you know, 90 % of us were peasants.”

0:38:09 · Benedict Evans · Automation history

Benedict Evans recounts that a BlackBerry co-founder, on seeing the iPhone, said that if it worked BlackBerry would be competing with a Mac.

“this is a great quote from one of the co-founders of BlackBerry. He said, you know, he looked at the iPhone, if this thing works, we're competing with a Mac.”

0:18:29 · Benedict Evans · BlackBerry and iPhone · 2007

Benedict Evans says the original iPhone had four or eight gigabytes of storage and ran a PC operating system, which is why BlackBerry and Nokia were badly placed against it.

“And it had like four or eight gig of storage, which was just a ludicrously large amount of storage. And it was running a PC operating system.”

0:18:56 · Benedict Evans · iPhone hardware · 2007

Benedict Evans says that around 1978 an Apple II with enough memory and disk drives to run a spreadsheet cost roughly $10,000 to $15,000.

“to buy an Apple II with enough memory and the disk drives and stuff with like $10 ,000 or $15 ,000”

0:54:07 · Benedict Evans · Early spreadsheets · 1978

Benedict Evans says most people use large language model products weekly or monthly rather than daily, even among 13 to 19-year-olds.

“most people are weekly active users and monthly active users and not daily active users.”

0:54:34 · Benedict Evans · AI usage · 2026

Benedict Evans says that around 2010 many clever people in tech expected mobile to repeat Windows versus Apple, with Apple obliterated, and that this did not happen.

“this was going to play out like Windows versus Apple, Apple was closed, Apple would get obliterated and that was just not what happened”

0:15:54 · Benedict Evans · Mobile predictions · 2010

Benedict Evans describes the AI model market as three to six companies spending heavily plus up to a dozen others willing to run three to six months behind.

“you've got pick a number of sort of three to six companies spending a lot of money and then maybe anything up to a dozen companies that are willing to be three to six months behind”

0:03:53 · Benedict Evans · AI model market · 2026

Benedict Evans says coding is the only AI use case that has clearly reached product-market fit, a market he puts at tens of millions of people rather than hundreds of millions.

“The only thing that's actually got product market fit, like really, really got product market fit is coding.”

0:07:07 · Benedict Evans · AI product-market fit · 2026

Chapters

  1. 0:02:41No winner-takes-all effect in AI models

    Evans answers what investors are buying in OpenAI and Anthropic. He explains how models differ from past platforms that had structural advantages.

    “there's no apparent winner-takes-all effect in models”
  2. 0:06:12Why models may become commodity infrastructure

    He sets out his preliminary thesis: chatbots are a weak interface and the model companies cannot build every app. That points to models as infrastructure.

    “Models are commodities. There needs to be loads of apps.”
  3. 0:07:51Telecoms, cloud, chips and fiber as analogies

    Evans tests four past infrastructure industries to see where AI models might land. He shows how large revenue can coexist with weak investor returns.

    “all the value went out of the stack”
  4. 0:12:21The 1999 reseller story and pricing power

    A banker's one-line verdict on a 1999 online parts seller frames his core question about pricing power for model makers.

    “it's a low margin reseller, one time sales”
  5. 0:15:32The S-curve and not yet knowing the questions

    Evans uses cars, PCs and mobile to argue that early in a technology's life, opinions are premature. He applies that to AI today.

    “everything's amazing and nothing works”
  6. 0:25:55The blank screen and jagged frontier problems

    These two problems explain why software companies still matter. Users don't know what they need, and models fail unpredictably.

    “the blank screen problem and the jagged frontier problem”
  7. 0:37:42AI, jobs and what work is really for

    Evans draws on accounting, newspapers and past automation to argue that new jobs appear that nobody predicted. He says the real question is what customers are hiring for.

    “you don't know what the new jobs will be, but you know there will be new jobs”
  8. 0:41:03Language models inherently give you the average

    Evans asks when buyers want the typical answer and when they pay for something different. This shapes where AI substitutes for human services.

    “absolutely inherently is they give you the average”
  9. 0:54:34Better products, not better models

    Using a lawyer and a 1978 spreadsheet as an analogy, Evans says most people still use AI weekly. Fixing that needs products built around real use cases.

    “most people are still the lawyer looking at spreadsheet”

Transcript

0:00:00
Benedict Evans

When you automate away work, you can always see the jobs that are going to go away because they're right there. And you don't know what the new jobs are going to be. Human needs are infinite. How many people are earning a living from making podcasts now? Imagine predicting that 10 years ago. There's a stage in the evolution of the market where, like, if you're still arguing about that, you're an idiot. But there's a stage at the beginning where you might have opinions about some of these questions.

0:00:22

You're probably not even asking the right questions. That, I think, is where we are with this stuff today.

0:00:31
Bernard Leong

Welcome to Analyse Asia, the premier podcast dedicated to dissecting the pulse of business, technology, and media globally. I'm Bernard Leong, and the AI cycle is no longer about just better models. It is becoming a capital markets test of infrastructure, distribution, and energy. Entropic and OpenAI are moving towards public markets. SpaceX is preparing one of the most consequential listings in technology. And Entropic's launch this morning of Claude Faber 5 shows how frontier capability now becomes bundled with safety, access, and governance questions.

0:01:05

These headlines point to the same deeper story, massive capex, increasing commoditized models, shallow but enormous user reach, and a new battle over who controls the curation layer over infinite intelligence. Today, we test whether the model layer becomes a utility, where durable value actually accrues, and how AI reshapes work and competition globally. With me today, recurring guest, Benedict Evans, independent technology analyst behind AI Eats the World. Benedict, welcome back. Thanks for having me. So since our last conversation, what have you been up to recently?

0:01:42
Benedict Evans

Been on lots of airplanes. People want to know about this AI thing you may have heard about.

0:01:47
Bernard Leong

Yeah, I guess when we talk about AI, right, I just want to go straight to the question. I mean, I guess early this year, you have started updating your presentation every six months now, right? And AI is eating the world probably now in the 2026 edition. So I think I want to start off first with the biggest question, which is OpenAI, Entropic, and the public market test. So one article that I really enjoy reading, since I'm a subscriber of your newsletter, is how OpenAI compete.

0:02:17

Is that the frontier models are pretty much near what we call commoditization with no network effects. And now everybody is trying to buy compute time. And this is not a moat. So both OpenAI and Entropic are now heading towards a public near $1 1 trillion capital market cap. So what is the public investor actually buying if not a moat?

0:02:41
Benedict Evans

Well, so there's several different questions embedded in that. The first of them is, at the moment, there's no apparent winner-takes-all effect in models. And this is kind of a subtle but important distinction. There's a difference. There are things that you're doing that no one else can do, no matter how hard they try, that they won't be able to do because of the nature of their business, that they won't be able to catch up no matter how much money you spend.

0:03:14

because that's where Google is in search. It doesn't matter how much money Microsoft spends, they can't catch up. And that's what happened with YouTube and with iOS and Windows and Instagram, that there are kind of inherent structural reasons why it's really difficult for them to lose their position. And we don't see equivalence of those in large language models yet. Now, the big challenge, of course, is we don't know how this market is going to evolve. and some people have a problem with that statement too but at the very early stage of this market you don't know how it's going to evolve we also don't know how the science works and we don't know how the science will change so stuff

0:03:53

may happen that means there are network effects but right now there aren't and so you're in the situation where you've got pick a number of sort of three to six companies spending a lot of money and then maybe anything up to a dozen companies that are willing to be three to six months behind but there's not some mechanic that means that if one of them gets ahead they'll get further ahead and so what we see now is Antwapic got code working and got coding working and got product market fit.

0:04:20

There's not some inherent reason why it's just impossible for Google and OpenAI to catch up with that and to get their own things working. They may do, they may fail to execute but you can't predict that. And so that's a base observation. The models are all kind of the same with the same kind of benchmark scores the same kinds of evals. Because they're using the same infrastructure and the same training data and the same algorithm. So that's what you'd expect.

0:04:46

So then you get to a question which is, firstly, would there be mechanisms, for example, use of capital that would allow one of them to pull ahead? Secondly, how far up the stack do they go? Is the model the whole experience? Is the model what we use? Does the chatbot become the universal user interface? Which is a question we've been asking for three years. or does mainstream mass market adoption and adoption that's not using it once a week or a couple of times a month but using it all the time every day, does that need this to be wrapped in apps and use cases and tooling and go-to-market and data and everything else that we think of when

0:05:24

we see software? Which is, you know, your phone has 50, we all use 50 different databases every day. That's right. But they're all different. They will run on SQL or whatever it is but that's kind of not the point. That's not what the product is. And so how far up the stack can the models go? Does Agenic go up the stack? Or do the models have to be APIs that are used by stuff built by other people?

0:05:46

And in the end, if there's 500 of these things, they can't all be built by Anthropic and OpenAI. Not because you can't get the model to write the code, but because you've got to build the business and you've got to work out what it would be and you've got to build Salesforce and you've got to do go-to-market and you've got all the other stuff that happens after you've written the code. And so if they can't, and so like, I don't know why they would be able to do that any more than Microsoft could or Apple could.

0:06:12

And so that kind of gets you to like a kind of a sort of preliminary like thesis, which is, okay, models are kind of a crappy UX. Chats are a crappy UX. Models are commodities. There needs to be loads of apps. The model company can't build all of those. And so the models kind of look like they're going to end up as commodity infrastructure. Now, then you get a bunch of questions. So to your point about pricing, clearly right now we're in this moment of extreme pricing crunch.

0:06:40

But we should just presume, like all the multipliers in that are going to get flung all over the place in the next year. So you've got $1 1 trillion of capex coming down the pipe. The models get 50 times, 100 times, 200 times more efficient. The chips keep getting better. You don't know what the next model will be. The next model may be way more efficient. the next models on past history, I know they use way more tokens.

0:07:07

The only thing that's actually got product market fit, like really, really got product market fit is coding. So imagine if we had something that had product market fit that would actually have lots of people who wanted to do it rather than coding, which is in the end like tens of millions of people, not hundreds of millions of people. So all those levers are going to get swung all over the place. So where might it settle out in the end?

0:07:26

And so I was thinking about this, that you can kind of throw analogies out there. So you can say, well, you know, So what happened in telecoms is in mobile, mobile has marginal cost. There's a bunch of people in tech that don't get this. Mobile networks have marginal cost. They have more users using it more. You have to build more network and that costs money. And so what happened is that basically we went to paying $50 or $100 a month every month for 20 years.

0:07:51

Yes, not quite as simple as that. But we paid roughly the same amount of money. We now use 1 ,000 to 2 ,000 times more data. it's 1 trillion dollar annual revenue industry pays $200 billion a year on CapEx and the stocks have gone nowhere in 20 years. There's this very big industry that spends a lot of money on CapEx and earns a lot of money but there's not really any profit there's not very much profit and there's not a great investment as an investor because all the value went out of the stack.

0:08:23

Correct. All the cool stuff was built by other people. Now there's other comparisons you can make so you can also point to cloud where there is a pretty good return and there is differentiation between Google and Microsoft and Amazon. But there again, there's no network effect.

0:08:38
Bernard Leong

They're already there as well. All the businesses are using these hypherscalers, right?

0:08:41
Benedict Evans

Yeah, you run on AWS, but AWS doesn't get a percentage of your UberTrip. So they don't actually get that much value or leverage further up the stack. They're just an essential component. You could point to chips and the value of the chip comparison is that it gets more expensive with energy duration. No, I don't mean NVIDIA, I mean like TSMC. in the internal systems. Semiconductor manufacturing, because of Rock's Law, gets more expensive with it. What Rock's Law observes, it gets more expensive every generation.

0:09:06

And so now there's basically three chip companies. Whereas, or two, or one, or however you want to count it, as opposed to 30 or 40 years ago when there were dozens. Because with each generation, it got progressively harder. So it might look like telecoms, it might look like cloud, it might look like semis. The fiber comparison is interesting because at first sight the fiber comparison is dumb because the fiber bubble 25 years ago was built out.

0:09:34

But nobody was using it then. It was built out way ahead of demand. Yeah. So there was all this empty fiber. Whereas this, of course, is being built out behind demand, like you can sell every token you can make. However, the interesting part of the fiber comparison is with some massive disaggregation. So what happened with fiber was that you got split up. So you had ducts and dark fiber and light fiber and wavelengths and IP streams and probably some other stuff.

0:10:01

And then you have people trading bandwidth on top. And then there's always somebody who's got slightly newer equipment with slightly lower stuff. And so everyone is selling it at marginal cost or less. Feels like tokens to fiber. Well, so this is the question is like the more the market gets disaggregated, so you've got like Mark Zuckerberg saying, well, you know, if I've got spare capacity, I'll sell that. And people trying to trade it and people building ETFs on top of it and stuff.

0:10:28

The more that you get, the more the market gets fragmented. And then in parallel, there's this sort of other question of how long do you need the newest model? I'm being slightly unsystematic in how I'm explaining this. But how long do you have to have the newest full-fat model? And at what point and how many use cases can you use the six-month-old model or the three-month-old model or the open-source model or run it much more slowly and run it on your own equipment or whatever?

0:10:53

how many use cases have to have the full intelligence, the full model running in the cloud at the massive price with using lots of tokens. So all of which is to say, you know the bell curve IQ meme? Yeah, that's right. And it's like the dumb guy, the person with 50 IQ and the person with 150 IQ, they say, well, commodities tend not to be high margin products and compute tends to get cheaper at a time.

0:11:24

And then you can kind of sit in the middle trying to analyze, you know, how many TPUs does Google have this year? Which is, you know, I'm being slightly unfair because that's a hard thing to do. But what does this actually tell you? There's a story that occurred to me. I'm sorry, I'm monologuing. No, it's okay. The story that occurred to me when I was a baby and a baby technology analyst back in 1999, there was a company that sold computer parts, computer components, chips and memory and computers and printers, all this stuff.

0:11:58

Sold all this online. And it had been converted from, originally it had been one of those companies that had 20 pages in the back of a computer magazine, if you remember. They kept a lot with their parts. Yeah, it was one of those companies. You know, the back of a computer magazine in the 90s, it was like a 300-page magazine and the back 150 pages was one insert after another of 20, 30, 40 pages from different resellers.

0:12:21

And they were basically selling the same stuff and you kind of page back and forth and say, well, they've got the printer I want for $260 and they're selling it for $250. So they make a website, they're doing 100 million quid a year in revenue. We all go up to see the company. On the way back, and I think another investment banker got a mandate to float it at like 700 or 800 million. And on the way back, we're discussing it and this veteran banker says, it's a low margin reseller, one time sales.

0:12:54

So nobody calls it up. You can spend a lot of time talking about Amazon and internet and consumer adoption and broadband and HTML and returns and pricing and the management team. So one time, it's a low margin reseller, one time sales. And that's, now that's a thesis. and you know I suppose the question I'm asking is like explain to me why people who are basically using similar infrastructure similar models, similar tokens to deliver a similar product and there's going to be three to six of them and it's going to get fragmented with better and faster and newer and older and cheaper and more expensive.

0:13:38

Why is there pricing power here? That's the point.

0:13:41
Bernard Leong

And yeah, I really come to your point on that. And then I think you are also very clear about the fact that the only UI that even with Agenda AI happening this year, we are still using the chatbot interface. Yeah, it's still DOS. There's not much change, right? It's still DOS. Yes.

0:13:57
Benedict Evans

I mean, I think there's another strand I think that's interesting to pull on here, which is that if you think about the kind of classic S-curve framing, how technology evolves. So if you think about the way that worked with mobile or the way that worked with the internet, There's a period at the beginning where it's kind of flat and it's not working and no one cares and it's dumb and stupid. And then there's a period where everything kind of starts working and the curve starts shooting up and it's really exciting.

0:14:21

That's right. And then there's a period where it's all kind of happened and it's been done and it flattens out and it's kind of boring. Correct. Which is where smartphones are now. It's where airliners are now. It's where cars were until electric happened. That's right. There's not very much difference between this year and the model from five years ago. It's all the kind of stuff that you wanted to do. Everything's kind of been done and it's become a kind of mature boring product.

0:14:46

The reason I mention this is when you're at the beginning of that phase, it's very unclear how this is going to work. And there are many potential paths because most of the stuff that people are going to, that need people, how can I put this? You've got a car in 1905, like you need to choose the spark plugs and you drive along with a bundle of 50 tires on the back of the car because you get a puncture every three miles.

0:15:12

And you generally have a mechanic with you. Like if you're rich enough to own a car, you're also rich enough to have a mechanic on staff. And you need it because the car breaks down every like 20 minutes. And nothing works. It's incredibly exciting, but nothing works. Same thing with PCs in the late 70s, early 80s. Incredibly exciting. And if you were using it then, you thought this is great. What do you mean it doesn't work?

0:15:32

No, like the normal experience of when we were kids was you'd be tapping away on your computer and you look up at the screen and it'd have frozen and you'd crawl under the desk and unplug it and plug it in again and hope that you hadn't lost more than 10 minutes of stuff. And so everything's amazing and nothing works. And the stuff that has to happen for it to work doesn't exist and you don't quite know how that's going to evolve.

0:15:54

And so that was where mobile was in 2005. Do you feel that this cycle is going much faster? But my point is, yes, the cycle's faster. but that's where we are in the sense of you know are you completely confident that in 2005 you would have predicted how mobile was going to work or even in 2010 there were all these people who were convinced that a lot of really clever people in tech were convinced that this was going to play out like Windows versus Apple, Apple was closed, Apple would get obliterated and that was just not what happened and we could explain why and I didn't think that was going to happen and I was right but I've been wrong

0:16:33

about plenty of other things. I thought Windows Phone might have a chance and I can explain why as well. The reason I'm saying this is there's a stage in the evolution of the market where you can have strong, clear opinions about how this is going to work. And there's a stage in the market where if you're still arguing about that, you're an idiot. If you're still arguing about why would anyone use an iPhone or you should be able to install any software you want on it.

0:16:58

like that was an argument from 10 years ago pay attention but there's a stage at the beginning where you might have opinions about some of these questions you're probably not even asking the right questions and that I think is where we are with this stuff today where the PC was in the 80s where the internet was in the mid 90s I think I used like 1997 picking out that data random where mobile was in either 2006 or 2010.

0:17:32

Where there's a lot where you just don't know how this is going to work and you don't know what the levers are going to be. The funny thing about mobile was like we thought it was working before the iPhone. It just wasn't growing very fast. But it seemed like it was kind of working. And we thought Nokia was going to dominate the world. Yeah, Nokia and Windows and BlackBerry. And it wasn't clear that something was going to come along and just blow the roof off.

0:17:55

and we all thought mobile was basically a PC accessory and it was, well, what do you do on the mobile internet or mobile use cases? And of course what actually happened is now you say desktop internet and the mobile made the PC a smartphone accessory. It's a smartphone that became the center of tech and the center of people's usage. And no one in 2000 was saying that or 2005.

0:18:21
Bernard Leong

Because they couldn't imagine that the mobile phone is actually the personal computer that actually can grow to that large market size, right?

0:18:29
Benedict Evans

Almost literally. I mean, this is a great quote from one of the co-founders of BlackBerry. He said, you know, he looked at the iPhone, if this thing works, we're competing with a Mac. Because this is the thing, it's quite hard if you're not in tech to understand. An iPhone is a Mac. And it's a PC, a PC in the general sense. And whereas it was competing with phones that had some compute added. And then suddenly, no, this is just a whole PC.

0:18:56

And it had like four or eight gig of storage, which was just a ludicrously large amount of storage. And it was running a PC operating system. So that was why, you know, BlackBerry and Nokia were completely screwed. The analogy I used to give was it was kind of like you're in the ocean liner business and then you look at jet airliners and you think, you just don't have any relevant skills at all for making that thing.

0:19:17
Bernard Leong

Let me point a question in another way. and I take this from my observation being a person living between the East and the West. I find that this batch of US companies, your Entropic and OpenAI are behaving more like Chinese companies. And I'll give you some features of Chinese companies. They work 996 and according to RAM this last two years, I think all the companies are expensing on their Saturdays. Only if people are working like crazy, I know some of the AI researchers are.

0:19:46

In China, you see Alibaba is not just eBay. It is also the PayPal. It is also all the other services. Much more horizontal competition. Yeah, horizontal. And they try to vertically integrate the Wi-Fi.

0:19:57
Benedict Evans

You've got a much wider matrix with many ways and columns and lots of companies have got lots of the cells filled in.

0:20:02
Bernard Leong

And you see this batch of AI companies, they are not just supplying to like, say, clock code, entropic supplies to Cursor. But now it's also coming in to compete with Cursor as well.

0:20:13
Benedict Evans

Yeah, I don't think that's true. I'm not sure about that. So I mean, I think. So do you find it? So I think the 996 thing, I think that the contrast there is just, well, what was the valley like in 2015 or 2020 when Google had one and had a fire hose of cash? And there were a lot of people there not working. Free spas and pets. Yeah, a lot of people were not working very hard because like, hey, the money's just there anyway.

0:20:37

So I'm not sure that was, I think that's just the nature of that S-curve I was talking about. Like when is everything amazing and exciting and you have to build everything yesterday. day um i think the and also like this is this um you know when it's early you don't know what it's going to be correct and you know there's kind of an interesting contrast here but contrast point out between like put this very very simplistically google's responses will make it a feature incumbents always make it a feature that's right the research is a feature yeah this is also apple But we'll add it to search, we'll add it to Gmail, it will empower new capabilities and make the

0:21:16

existing stuff better. This is also what Apple is doing now. That's right. Whereas if you're not the incumbent...

0:21:25
Bernard Leong

Entropi uses core work?

0:21:27
Benedict Evans

Yeah, so then you don't have places to add it as a feature. That's right. But then meanwhile for Entropi and OpenAI, the question is, okay, well what are the use cases for this? What do you do with it? How do you communicate to people what they would do with it? and so clearly the position at OpenAI late last year was the design was everything everywhere yesterday. So we'll try a social video app, we'll try an app store, we'll try another app store, we'll try an e-commerce checkout flow, we'll try, I don't know, there were like half a dozen different things at ad business, all sorts of stuff because we don't know what it is and so we'll try all

0:22:09

of them and double down on the one that works. It turned out that the one that works is software development. At least that's the first one that's working and Anthropic got that working first. And so OpenAI is sort of pivoting in to go into that one.

0:22:22
Bernard Leong

But Anthropic also indirectly starts to kill off cursor usage. So you're getting into this place where it's like a cannibalization because this is very different from, if I say the US internet, it's always you are in this lane like Google is always going in search but it never goes into someone's lane

0:22:42
Benedict Evans

That's total horseshit Google has spent the last 20 years people have spent the last 20 years saying Google's killing this startup or Apple's killing that startup or Meta's killing that startup If you're making a platform I mean this was always the whole five years of people arguing about competition like big tech companies killing startups if you go back and look at again go back and look at the way Microsoft and Windows evolved I used to have a slide of this.

0:23:09

There was a program called Sideways that you could buy in the late 80s. And it was like $150 in late 80s dollars, so like $300 now, whatever the inflation is. And what Sideways does is it lets you print out in landscape. That's it. It lets you print your spreadsheet in landscape. Yeah, I can remember that. Sideways. And that's $150. and today that's a checkbox in the print dialogue box in Windows or in Mac OS and so the point is the same thing if you bought spellcheck better another example spellchecking was a whole software category yes you buy PC magazines and they would have a group test of 10 different spellchecking programs that were all $200 and $300 and the way

0:24:00

it worked was that you would write your document and then you would save it and then you would open the spell checking document and point it at your spell checking program and point it at your document. It would open that and it would run spell check on that. And so my point is there's a whole bunch of class of stuff that just naturally gets integrated. And so this is, of course, the question is how far does that integration go for an LLM?

0:24:24

How far does that do you go? Is agentic coding, is that inherent to the thing? and so you could take the kind of maximalist view here and say that the way that these models will be able to do quote unquote everything is because they'll be able to write their own code to do stuff. So you won't have the model, so to speak, doing everything itself from first principles in tokens. Instead it'll make a Perl script to do it.

0:24:51

So the model will write a piece of conventional traditional software to do that thing. And so everything will be being done through, the models will generate their own intermediate layers of code to do this.

0:25:01
Bernard Leong

Yeah, so I come back to your favorite analogy. When you say that in the 1980s, people were given a spreadsheet, but they don't know what to do with it. But essentially, the spreadsheet became Tableau, all the different kinds of software that people built on top of because they discovered that they know how to use a spreadsheet, right? To me, code work is the equivalent of a spreadsheet today, except that it could create a Word document, create a coding script to help me to do exactly what you were saying.

0:25:25

maybe is that maybe the change in ui or maybe is it because of the fact that you have to still orchestrate using by telling it what you want rather than that makes it difficult to i think

0:25:35
Benedict Evans

this is realize what's going on so i mean co-work in a sense is what is kind of what i'm describing which is that you want the model to do a thing for you it may be that the way it does that thing is using Git or writing an Excel macro or something.

0:25:53
Bernard Leong

And it's not doing everything out there for me.

0:25:55
Benedict Evans

It's not doing everything in the LLM as LLM. The LLM is to using a tool. However, I don't think that solves the underlying problem of a chatbot as UX. And I think there are two fundamental problems here that drive software creation, company creation. and that's the blank screen problem and the jagged frontier problem. The blank screen problem is how is it that you know what you need? You don't sit down, you know, how do you know what buttons there should be and what the questions are and what the flows are?

0:26:30

Most people couldn't sit down and draw a flow chart of precisely how they're going to do this job. And when you buy software, what you're buying is a whole bunch of institutional knowledge and a whole bunch of thought as to what the problem is, what the workflow should be, how it should work, what options I should give you in each step, that you're not working out from first principles and you're sitting down yourself, shutting your eyes and thinking, hmm, okay, how am I going to explain what I want?

0:27:00

When you buy software, that's what you're getting. You're getting, they've worked that out for you. And most people are not tool creators in that sense. The person who's a really good graphic designer is not also the person to design in design. That's right. So I think that's one kind of broad set of problems. I think the second is the kind of the jagged frontier, which I think is a better way of phrasing than hallucination, in that these models do some things very well and other things not very well.

0:27:35

Even now, each new model is the same. or no, you've got a new model that maybe has the jagged bits from different places. And so there are some people who say, like, hallucinations are solved and you think, well, for you they are. But not for my use cases. Actually, no. Let me finish your point. I mean, clearly we do not have AGI. Therefore, the stuff these can do and stuff they can't. But that comes in unpredictable ways.

0:28:04

It's ways that the things that can and can't be done, we don't necessarily know why it does this and not that. You can't tell from the command line which is going to work and which isn't. As a normal user, if you're not reading AI papers every day, you're not going to know, oh, well, of course that task will work brilliantly. This task you've got to ask in a completely different way and it may still not be right.

0:28:27

So you've got this jagged frontier of what the models are good at. Then there's like a jagged frontier of what we can intuitively understand it's going to be good at or not. then there's like a jagged frontier of what actual use cases i have that are important and valuable and with software development those all kind of line up but there's a bunch of others where they don't necessarily line up never mind of course what are the tokens cost as well which is like this use case might use way more tokens than that use case and you might not realize why or we might not even know why and the one that uses loads of tokens might not be worth very

0:29:01

much either so there's all these sort of there's all this sort of jaggedness in how all these bits fit together and that's why you hire a software developer you know you hire a software company who sat down and worked out okay this is how it's going to work it will work if you do it this if you give it this data if you frame it like that if you preload it with this you give it this context and we'll make sure that we'll tell you if you ask it that that won't work or we'll tell you no this bit probably you probably want to check that number you wrap it into both all of that is to say you need

0:29:28

to kind of wrap it and tooling and use case and go to market and deep thought about what the actual problem is and how this is going to work. I mean, this is, you know, the kind of, this is the problem with the SaaS apocalypse or, you know, the nuance to the SaaS apocalypse is, you know, the hard part of building a software company is not writing the code. It's a distribution and network. It's everything else.

0:29:51

It's realizing that that problem exists. Because very often the people who have the problem don't see it. The people who have the problem that that software is solving do not realize that that's a problem. And then the way that you're solving it, maybe the third or fourth, maybe three or four other people have tried to solve it before and failed. And you've worked out the right way of solving that problem by turning it into a different problem that didn't look like that problem.

0:30:18

And you've worked out the right go-to-market and the right insertion point and the right way to get the data to make the whole thing work. And if you're sitting and doing accounts payable or doing color grading in a video editing shop, those aren't your skills. You're great at color grading. you're not great at trying to work out the right dissociation point or even seeing that that problem exists. So that to me is the challenge of the whole thesis that the model will go all the way up to the top.

0:30:43

Unless you believe that the models will just kind of scale to be super general, I don't know what terminology we're using now, if you believe that the models will be able to understand everything that I've just said and know all of it and do it all unprompted, then fine but then we've got bigger problems and wondering about the future of enterprise software

0:31:06
Bernard Leong

i i think the bigger problem and and this is speaking from being a practitioner and doing this uh every day every day is that one thing i always find is that like what you said this jagged p i like that analogy and i actually couldn't articulate it articulate it differently i feel is that the The AI software is very good at deceiving you with the output you think you want. With the demo, it's great. But once you get into a production situation, you start to see where the flaws came out with.

0:31:37
Benedict Evans

It's very good. I mean, there's all sorts of different ways you can describe this. It's very good at giving you what a good answer would probably look like. That's right. But it's not the good answer. But that may or may not be what you want. it may and it depends on the use case it depends on who you are it depends on why you asked so i mean a trivial example a couple of years ago when this just started working i went and spoke at an event and they asked me for a long biography and i didn't have one and they made a long biography in chat gpt and it was full of mistakes but the thing that occurred to

0:32:10

me is if i'd use chat gpt to do that i could have fixed it in like 30 seconds because i know i'm an expert on myself so So I can read the biography and say, no, not that company, not this company, change this, delete that paragraph, great. If you don't know anything about me, it wouldn't be very useful for you. And so this is kind of my point is, what is a good answer? There are some questions that don't need a precisely correct answer or where there is no precisely correct answer.

0:32:41

Like, give me, here's a picture of a new CPG product, suggest 10 advertising slogans for it. I mean, that doesn't have a right answer. It's got better and worse answers. There's no precisely correct answer that you require for that. And so this is sort of the question. It's like, how do you turn things into generative AI questions? If you think about the last 10, 15 years of machine learning, there was a bunch of stuff that was obviously a machine learning question.

0:33:15

There are also a whole bunch of companies where they said, well, we realized that we could turn this into pattern recognition. Machine learning is pattern recognition. So we realized we could turn this into pattern recognition and this is how. Or we realized we could turn it into image recognition. It didn't look like image recognition. We worked out a way to turn it into image recognition. And then you can automate it with machine learning models. And so there's this whole question of like, the stuff that's obviously an LLM use case, the stuff that right now looks like it's really not an LLM use case, like law.

0:33:46

or maybe that's the wrong way of putting it like it looks like it should be except that it's really hard because you need to get it right and then there's a lot of kind of questions of like how can you move how do you manage those all those problems those questions right can i can i ask

0:33:59
Bernard Leong

you this question and and this is like when people start giving the anything about um job displacement and i always argue with them in the following manner i said look the problem is that the lawyer's job that requires a lot of manual work, repeatable tasks, that is taken away by the AI. But the creative aspects of being a lawyer, as in putting the correct clause, knowing how to defend the client as a solicitor, that part still stays in the system because the AI is not going to give you the best answer.

0:34:32

It's giving you an answer that looks deceptively like the real answer, but not the correct answer. Am I right to bring it this way?

0:34:40
Benedict Evans

So two things here. I think firstly, well, there are different ways of talking about this. Firstly, I just wrote something about this a while ago, which is it's very hard to predict this. Correct. And you see this if you try and backtest it. So if you go back and look at, you could say basically half of the purpose of computing in the 20th century was to automate accounting. So we have adding machines and punch cards and mainframes and databases and data processing and spreadsheets and ERPs.

0:35:16

You're trying to automate accounting and bookkeeping. You go look at the US census data, the number of people who are called an accountant and a bookkeeper basically goes up in a straight line from the 20th century. And so this is where people go and pull up the Wikipedia entry for the German paradox, which is basically price elasticity. If you make it cheaper to do something, do you do the same work for less money or do you do more work for the same money?

0:35:35

or maybe you do more work for more money because you've got a new ROI. Now that's part of that. But actually if you stop and think about it for a minute, that's not what happened. What actually happened, because an accountant today doesn't do the work that they would have been doing 50 years, but way more of it. They're doing a whole bunch of other stuff that they wouldn't have done then because it would have been impossible.

0:35:54
Bernard Leong

It makes them able to do more work.

0:35:56
Benedict Evans

But also enables different work, different characters of work. And frankly, if you were an investment banker 40 years ago and you wanted to do a DCF, well that would take all week and now you can you know type in a different whack and you've got a new dcf like it takes you know half a second cool work now yeah exactly so my point is so but the point is like the stuff that the accountants are doing now is not what they were doing 50 years ago and so the fact that you could automate what they were doing 50 years ago unlocked all sorts of other stuff the inverse of this is what happened to most obviously newspapers where the internet

0:36:28

didn't really change what it was to be a journalist but journalists were being paid through a light manufacturing and trucking business that had a monopoly on local advertising.

0:36:36
Bernard Leong

But that's the kind of stick on the way, right?

0:36:39
Benedict Evans

So if you'd sat and looked at the job of a copy editor and said, does the internet change the job of a copy editor? The answer is no. Zero exposure to internet. Except that the salary was being paid by something completely different. And so that wouldn't be a new analysis of is this job exposed here. Same thing with Uber. Like, you know, if you've been saying what jobs were exposed to smartphones, no one was talking, everyone was talking about GPS, no one was talking about taxis.

0:37:08

So, like, you know, in a sense what I'm saying here is like, it's like, why don't we just only buy the stocks that go up? We should, like, you can't predict this stuff. You don't, you know, we don't have perfect knowledge of the future. I think the second answer here, though, I think would be like, yes, obviously, you know, it's even simpler, like Jevons Paradox, lump of labor fallacy. love of labor fallacy is that like the new thing gets you know when you automate away work you can always see the jobs that are going to go away because they're right there and you don't know what the new jobs are going to be but the fact that you've automated away

0:37:42

that work has unlocked new economic demand you know new economics that allow the consumption of new things and human needs are infinite how many people are earning a living from making podcasts now imagine predicting that 10 years ago so So you don't know what the new jobs will be, but you know there will be new jobs. And so, yes, it will be painful and there will be jobs that go away. But that's what's happened continuously over the last 200 years.

0:38:09

200 years ago, most of us were peasants. And, you know, we worried that the crops would fail. And, you know, 90 % of us were peasants. And we spent the last 200 years, 250 years automating that away, and yet there's just as many jobs. So you kind of need to have a theory for why this is different. And at that point, there's just kind of a lot of hand-waving because you can say, well, this is happening way quicker.

0:38:31

I think, is it? We're three years in and nothing's happened yet.

0:38:35
Bernard Leong

But you see less hiring. Oh, but no, if you look at the job state, it's not true, right? No, no, no, I agree with you on this one.

0:38:42
Benedict Evans

At best, you'll get a lot of argument amongst economists as to what's really good. I don't think there's any sense amongst economists that we're clearly seeing a decline in hiring yet. But I think the broader kind of job question here is that I want to come back to something else you said, where you started, which is what is the actual job? What are you actually hiring them for? Why did you hire the law firm? Did you hire them to make the contract?

0:39:09

And sometimes the answer is yes. Sometimes the answer is I hire them to make a plain vanilla contract that does what all the contracts will say. And somehow or other when you actually hire a lawyer, there's always a reason why it's not that and like but maybe that's what what you want like sometimes no you really just want the thing and the analogy i was thinking about here is to think about amazon and retailers because amazon can get you the scoop i mean set aside that there's a bunch of stuff that isn't on amazon like you know luxury goods but for the sake of argument amazon if you think about like books would be the best example here if you

0:39:45

know exactly what book you want amazon can get you that book. But how do you know you want that book?

0:39:52
Bernard Leong

You have to search for it, right?

0:39:53
Benedict Evans

And when you go to a bookshop, generally you walk out with books you didn't know existed.

0:39:59
Bernard Leong

Yeah. I still go to bookshops, yes.

0:40:02
Benedict Evans

And so the role, the question here is, is the purpose of the retailer to be the most efficient endpoint to a logistics chain? In which case, Amazon is probably more efficient, not necessarily for groceries, say. Like it's probably more efficient to go to the shop on the corner or the shop in the ground floor of your apartment building and buy milk than to order it from Amazon. But if you don't know that that thing exists, what is it that the shop is doing something else?

0:40:32

Book buying is a leisure activity. It's about service and experience and suggestion and curation and recommendation. And Amazon has 800 or 900 million schools. You can't go to Amazon and say, I kind of feel like buying something nice for my house. That's not a SQL query. So the bookstore has been, the job has changed. My point is the job is separated. Is the function just to get me the thing in the most efficient way possible, or is it something else?

0:41:03

And Amazon can get you the thing, but it can't really do the something else. And the LLM can make you the thing, but can it do the something else? Another way to think about this is what LLMs do. absolutely inherently is they give you the average. They tell you this is what most people would probably say. Is that what you want? If you want to be creative, you have to take the entry in, right? You're going to your lawyer, I want an NDA for my company.

0:41:35

Okay. Do you want a different NDA from what everybody else has or do you kind of just want the same NDA as everybody else? It's probably the same NDA, right? Maybe you want a lawyer to check it. but you probably won't be paying the lawyer to write a document that's exactly the same as the document that everybody else uses. You probably shouldn't be now for that. But certainly with AR you won't be doing that. But then the question is, but is that what you were getting from them?

0:42:00

Were you getting something else? And if you want, and this is my point is, do you want the average? Do you want the mean of what everybody would probably do, what everybody would probably say? And the answer very often is yes. like how do I make a risotto how long should I cook the rice for I'm not looking for an original answer here I'm not looking for the answer but then there's and you can extend that here's a picture of my fridge what should I cook for supper that will work now try it it will probably work better than you would expect that will work I believe you It won't give you some radical weird new thing you'd never thought

0:42:45

of. It would say, well, I can see some spinach. I can see some ricotta. This is what you make. I know this weather's going. But do you not want the average? And that's why, you know, I mean, it's the point, why do you go to the lawyer? Why do you go to Bain or BCG or McKinsey? Is it you just want the deck? There's like the dumb analogy, and there's two parts to this, because LinkedIn is full of idiots who think that they'll say, hey, look, Claude made me this deck.

0:43:11

And the narrow criticism is the deck is terrible. If McKinsey gave you that, you'd throw them out. But McKinsey is using that too, right? Yes. But that's the thing. You have to distinguish between the things that will get fixed and will improve and the things that are different, the different question. But that's not why you hired them. Sometimes it is. Maybe if you're doing a plain vanilla private equity due diligence and you just want a deck that says this is a market structure and these are the competitors, well then yes Claude can do that for you or at any rate it can do it vastly quicker and so you won't pay them as much for them to do it for

0:43:47

you because they've got it set up right but maybe that's not what you are paying them for I was talking to a friend about it the other day who was at one of the North Bain CBCG McKinsey one of the big four accounting firms and he said the deal is that the CIO at the client wants to migrate from one version of Oracle to another and has paid one of the big four to do a study to say, should we do this?

0:44:12

And the answer was yes. The CEO says, this is a lot of money. Is this really the best way to spend it? CEO hires another of the big four to do a second opinion because what the first firm has done is said, yes, you should do this and you should hire 300 of us to come in and do the project. It's a two-year project. CEO says, interesting. So I'm going to hire another firm on the basis that they won't get the work to do the conversion and I would like them to give me an opinion on whether we should actually do this.

0:44:40

And the answer is yes, maybe if you double the normal depreciation life and also you do realize that number one, the partner at that firm is best friends with the CIO. Number two, the CEO really wants to have a big shiny project. Number three, the CEO is about to retire and doesn't have much political capital. So what the CEO actually wants is a document to tell the board what he and the board already knows. it's never about the problem now this is kind of an extreme case what is it that you're hiring them for sometimes you're just hiring them to get the debt you're just hiring them to get the code you're just hiring them to get the school you

0:45:21

just want what everybody would probably make I mean at the extreme this is you know what you can people who are most upset about AI well yeah I mean, it's like the thing of using AI to write romance fiction. Yeah, because it follows some very straightforward, well-understood conventions.

0:45:45
Bernard Leong

Or follow some good writer's form of writing, style of writing, for example.

0:45:51
Benedict Evans

The challenge is, do you want something that isn't the average? Because, I mean, this is an interesting kind of theoretical question in general. like an LLM an AI can make you more prog rock or more punk or more new wave or more stuff that sounds like not very good nirvana but it won't know that everyone is really bored of disco and prog rock and the economy's terrible and we're worried about this and that and so we want a completely different kind of sound and punk will do that i mean maybe that's not why people wanted but it won't know that people want something different like that and this is what it would be.

0:46:35

Or if it did, that would be a very different, maybe that's like a different amplitude. But it's easy to see that you can use AI to make more of the stuff we've got now. It's a different problem for it to know that we would want something different and what we would want. How will it know? Because by default that would be something that's outside the training data. Yes. And so it's a long way of kind of circling around your question, but how often is it that you hire something because you want the average of the training data?

0:47:08

And how is it that you want something that's out of the training data? You want something that's not what would be the normal suggestion.

0:47:16
Bernard Leong

Correct. If you're a hedge fund manager, you'll be looking for the alpha. So you won't be looking for what everybody agrees on. You'll be looking on what everybody disagrees on.

0:47:24
Benedict Evans

Yeah, but you can't just go to chat to EBT and say, give me some really stupid investment recommendations. Of course not, right? You can, but good luck with that.

0:47:30
Bernard Leong

But if I take your thesis in the sense, what you are saying is that you're saying that actually the LM is solving a lot of tasks that actually most people could agree this is the right way to do it. Except we do not know what the new tasks are.

0:47:46
Benedict Evans

Well, this comes back to automation, doesn't it? Yes. So the point of automation, right back to the 19th century, is take this one task and do it exactly the same way over and over and over again. Yeah. And so now with AI you can take a bunch of tasks that previously you needed people for and say do the same task, maybe not exactly the same way because it's a probabilistic system, do the same thing over and over again.

0:48:09
Bernard Leong

And redesign it in a different way so that it's efficient.

0:48:12
Benedict Evans

Yes, well it's like the thing of McDonald's making their hamburgers look artificially irregular.

0:48:17
Bernard Leong

That's right.

0:48:19
Benedict Evans

Do you want that? Do you want it done exactly the same way every single time? And the answer is sometimes yes, but not necessarily that's not necessarily why you're going to somebody yeah but if i take your

0:48:32
Bernard Leong

point to this right then i think creators will still have a job whether we have ai tools or not because there's always a need human need to create new things building new things and coming up with angles that i mean i'll use the example of people say uh go is dead because ai plays it but people we've been seeing a lot of gold masters now playing things that previously were taboos i mean i think that's it i'm not alluding to the age cases but i think that's i think it's a bad

0:49:01
Benedict Evans

analogy because that's like saying yeah it's like saying no one will go running anymore because we

0:49:05
Bernard Leong

have cars so well right of course not right people still go for a run right but that's what i'm saying is does that mean that actually nothing really changes but it's only the reconfiguration of what we think

0:49:18
Benedict Evans

I'll argue against everything I've just said there's a view here that says that what we've actually been doing for the last 200 years is automating higher and higher level human brain functions human functions so you start by automating human beings as beasts of burden so like human beings like pulling hauling things and carrying things and then you automate legs first and then you automate arms and you automate fingers

0:49:45
Bernard Leong

And now you're going towards the brain.

0:49:47
Benedict Evans

And now you're going towards the brain. Exactly. And so you'll reach the top and there'll be nothing left. Now, it's a neat framing. I'm not sure it's very convincing. It's like, well, how would you know that that's what it is? And robots don't work very well yet. Yeah, I mean, because how is it that you know that the things, that it's automating all the things that people want and all the things that people need?

0:50:09
Bernard Leong

And that is the underlying question, I suppose.

0:50:12
Benedict Evans

It is, and it's this sort of unanswerable thing because unlike every other platform shift, we don't understand the science of this. So, you know, I said, you know, you didn't know what was going to happen, but like at a deep level, you did. You didn't know how the internet was going to evolve, but you knew that PCs cost $2 ,000 or $3 ,000 and there's only 100 million of them in the world and there weren't going to be 5 billion people with a PC next year.

0:50:33

And that telcos weren't going to give everybody in the world fiber by the end of 1998. Like you kind of knew some basic constraints of what could happen. Same with mobile. You didn't know what the iPhone 4 would look like, but you knew it wouldn't have a retinal implant. And, you know, so you knew the basic physical constraints of where this could go. You knew roughly what it cost, you knew roughly how many people were getting sold every year and so on.

0:50:56

Whereas with RLMs, because we don't have a good theoretical description of why they work so well, this is the scaling thing, we know empirically that they have scaled. We don't have a theoretical explanation, a good theoretical explanation for whether that continues and how long. so we can't predict what it is that we'll be able to do with this stuff the same thing about cost maybe we have a paper tomorrow that says you can get the same results for 5 % of the compute maybe it's unlikely but we don't have a way to state that definitively

0:51:29
Bernard Leong

the theoretical physicist way of answering that question you say about why L works so well it just happens that you put a billion tokens you get the best gradient descent of everything that's probably the way I would look at it but I'll just have one final question, right? I think from listening to you and going through all your works over the last 10 years, I always, I think asking you about predictions is the wrong question. I have a different question this time, right?

0:51:55

What are the indicators that would indicate to you where AI has actually eaten the world or still eating the world? As in which part of the, how do we think about the S-curve? Is it, do we have to see something very unpredictable that hasn't shown up yet? Or is it that these things are always incrementally changing?

0:52:14
Benedict Evans

Well, so I mean, I think you could get a bunch of people agreeing, disagreeing on whether we've had a step change in capabilities or whether we've got sort of continuous improvement in kind of the same thing of principle. Have we had some radical acceleration? Or is it just that we got coding to work? Yes. I think it's certainly the case that people who only test this every couple of months and haven't looked at it since the beginning of the year do not really understand what we have now.

0:52:50

But I don't know. It's tough to make these kind of statements of magnitude. It's like saying, was mobile a bigger deal than the internet?

0:53:01
Bernard Leong

That's a meaningless question, right?

0:53:03
Benedict Evans

I don't really know how one would answer that question.

0:53:06
Bernard Leong

But your point of view is that you're just going to look at maybe what you do now and maybe six months down the line and see if there's really a dramatic improvement towards the task. Maybe that's a good indicator just to see whether this cycle has played out all.

0:53:23
Benedict Evans

So I'm trying to think how to put this. You know, I'm very much the lawyer, you know, the analogy I gave you, right, you know, imagine you're an accountant seeing the first spreadsheets in the late 70s. It's life changing. it does a week of work in an hour and that's what it's like to be a software developer looking at clawed code now it completely changes what it is to do the job however, now imagine you're a lawyer looking at a spreadsheet in 1978 okay, this is very clever my accountant should see this but that's not what I do now, word processors exist at the same time but if you're a lawyer looking at a spreadsheet you think well this is

0:54:07

very clever I might use it next week to do my timesheet I mean set aside that you needed like 15 grand worth of computer to run a spreadsheet in 1978 but I mean literally it was like to buy an Apple II with enough memory and the disk drives and stuff with like $10 ,000 or $15 ,000 but now imagine you set that aside you're a lawyer looking at this like that's great I could use that to do my timesheet next week but I'm not going to use it every day that's not what I do all day And it's kind of the same now with LRM.

0:54:34

So some people who are the accountant looking at spreadsheet, most people are weekly active users and monthly active users and not daily active users. Even like you look right in at like 13 to 19-year-olds, even there more people are weekly or monthly active users and daily active users. And so most people are still the lawyer looking at spreadsheet and they're saying this is very clever, it's quite useful, maybe next week. And I'm not sure that the solution to that is a better model.

0:55:08

I think the solution to that is no, you have to wrap that in different products and you have to think about how you create a use case where it's set up, no, I've realized that you have this thing that you didn't notice you were doing every day and I've made a thing that solves it for you.

0:55:25
Bernard Leong

and because the chat interface is not that I do interface for everything.

0:55:30
Benedict Evans

It isn't, no, and making it work is still fiddly. I mean, take an example, so I travel a lot, so I need to put my expenses together. So what's the right way of doing that? Well, I can take the email, I can look at the flight and the taxi and the hotel and type them into a spreadsheet, option one. because I don't need to put all the receipts together. Option two, I could get Gmail. Gmail could look at all those receipts as they come into email and automatically drop them into a Google Sheet.

0:56:05

Am I going to set that up? Isn't that Google's job? Option three, I could use a fintech or a corporate card that will automatically do that for me. So I could sign up to XYZ new card and that's one of their features and they've built that. and they're using machine learning or just as Google will be using machine learning. And they help you to do the expenses. The one thing I'm not going to do is load up Claude code and say, hey, can you log into my bank and my Gmail and here's my Uber account and can you calculate which of them are like, that would be insane.

0:56:45

That would just be a really, really stupid way of doing it. I mean, you could do that. Why on earth would you do that? Good point.

0:56:54
Bernard Leong

And that's a good place to wrap. Benedict, many thanks for coming on the show. Just a very simple question. Where do my audience find you? I'm a subscriber of your newsletter, so I'm just going to recommend it straight.

0:57:06
Benedict Evans

I need to check if you're a paying subscriber. I paid. I paid.

0:57:09
Bernard Leong

I can send you all the receipts over the last 10 years.

0:57:13
Benedict Evans

So my parents had good SEO. So if you Google Benedict Evans, then there's my website, and I do a presentation, or publish a presentation twice a year. I write stuff about what I'm thinking about and then there's a weekly newsletter which is my notes for the week of what's going on and what was interesting and what it meant.

0:57:30
Bernard Leong

Thank you for the spreadsheet analogy. I always think about it every time I think about a product.

0:57:34
Benedict Evans

Thank you.