# Are 95% of Businesses Really Getting No Return on AI Investment? — With Aaron Levie

- **Show:** Big Technology Podcast
- **Published:** 17 Sep 2025
- **Source:** https://pdst.fm/e/tracking.swap.fm/track/t7yC0rGPUqahTF4et8YD/pscrb.fm/rss/p/traffic.megaphone.fm/AMPP6580911203.mp3?updated=1758109571
- **Transcript:** https://podcasts.opennash.com/shows/big-technology-podcast/are-95-of-businesses-really-getting-no-return-on-ai-investment-with-aaron-levie-fe662c/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Box CEO Aaron Levie argues that the headline claim that 95% of businesses get no return on AI is a misleading read of an early-stage market. Most of what companies have run so far are pilots, and many are do-it-yourself builds that fail. He says real gains come from purpose-built tools, good data context and redesigned workflows. Staff are already adopting AI on their own, and he expects AI agents to spread over years.

The central tension is speed versus pragmatism. Levie says change will be gradual and uneven: early adopters such as coders and startups are pulling ahead, while large incumbents can afford to wait. The conversation moves from the MIT study to what an AI agent is, Box's new agent products, GPT-5's reception, and whether OpenAI's huge cash burn is a rational bet on automating knowledge work.

## Key insights

1. **Most failed AI projects are early pilots** Levie says the survey mostly captures centralized pilots, because the word 'agents' only appeared less than a year earlier. Pilots are expected to fail, and a single survey taker cannot see the unsanctioned ChatGPT use spread across a company.
2. **Building your own AI stack tends to backfire** The study found internal builds fail at double the rate of external partnerships. Levie says a company that builds itself may need 10 or 15 pieces of software, such as vector databases and permission systems, before one user can try the AI.
3. **Workflows must change to capture AI gains** Dropping AI into an existing process will not make it 3x better, according to Levie. In coding, engineers now act as managers who write specs, run background agents and review the output, and he expects other fields to follow.
4. **Employees adopting AI on their own signals value** The study found official LLM purchases at only 40% of firms but personal AI use by 90% of employees. Levie calls this revealed preference: people use it daily because it saves them hours of research.
5. **Levie sees OpenAI's losses as a strategic bet** Levie says the roughly $115 billion of cash burn is a choice, made to subsidize free usage and build capacity. He thinks it is small against the economy-wide prize of automating knowledge work, though individual players may still fail.

## Questions this episode answers

### Is it true that 95% of companies get no return from AI?

That figure comes from an MIT study cited by Axios, which found 95% of organizations saw zero return on AI. Aaron Levie of Box says it mostly reflects early pilots and do-it-yourself builds, and that his customers see strong returns on focused uses.

### Why do so many AI pilots fail?

Levie says pilots are experiments, so many are expected to fail, and companies that build their own AI stacks fail at double the rate of those buying ready-made solutions. He also says gains need changes to how work is done, not just adding AI to an old process.

### What exactly is an AI agent?

Levie describes it as an AI system that does work for you by looping through the model many times, tracking what it's doing and updating its memory. Examples are coding tools like Claude Code, Cursor or Replit that can do weeks of human work in about 10 minutes.

### Can I trust AI output enough to use it for real work?

Levie says that if the AI is given trustworthy source data, good prompts and a strong reasoning model, errors become rare. He compares reviewing AI work to reviewing a colleague's work, which takes minutes and saves hours.

### How can OpenAI lose so much money and still make sense?

Levie says the losses are a deliberate choice to win a huge market, by funding research and data centers and subsidizing free usage. He argues about $100 billion is small compared with the value of automating knowledge work, but says individual companies may still fail.

## Facts

- **Alex Kantrowitz** [0:02:31]: "They studied 300 public AI initiatives trying to suss out the no hype reality on AI's impact on business."
- **Alex Kantrowitz** [0:02:31]: "95 % of organizations said they found zero return despite enterprise investment of $30 billion to $40 billion into generative AI."
- **Alex Kantrowitz** [0:13:35]: "official LLM purchases cover only 40 % of firms, yet 90 % of employees use personal AI daily, at least those surveyed"
- **Alex Kantrowitz** [0:07:50]: "internal builds fail at double the rate of external partnerships"
- **Aaron Levie** [0:04:49]: "a company that wanted to deploy AI in a particular workflow in their enterprise, they might have 10 or 15 different pieces of software that they have to run and manage just before a single user could actually interact with AI"
- **Aaron Levie** [0:08:41]: "open source is actually extremely valuable for a company like Box because, you know, we're powering technology for 120 ,000 customers."
- **Aaron Levie** [0:15:49]: "Claude this week announced a new capability that will generate files for you."
- **Aaron Levie** [0:21:46]: "I was talking to a nine-person startup the other day that estimates that they're at a minimum executing at the size of about a 100-person company."
- **Aaron Levie** [0:30:34]: "Jeffrey Moore came up with this idea of the technology adoption curve or at least popularized it."
- **Aaron Levie** [0:34:33]: "it wasn't really realistic until 2006 and 2007 when you had the iPhone."
- **Aaron Levie** [0:34:58]: "Obviously, Waymo got kicked off, I don't know, a decade, decade and a half ago. And only this year is it accessible in suburban Silicon Valley."
- **Aaron Levie** [0:37:01]: "What we announced at BoxWorks was a new capability called Box Automate."
- **Aaron Levie** [0:36:32]: "we have an AI agent right now that any customer can go and use, which is a data extraction agent."
- **Aaron Levie** [0:40:47]: "And we saw meaningful improvements from GPT-5 versus GPT-4 -1, as an example, on our eval."
- **Alex Kantrowitz** [0:42:00]: "OpenAI's losses are now going to total $115 billion through 2029. Oh, sorry, it's cash burn."
- **Alex Kantrowitz** [0:42:00]: "It's expected to make like $10 billion this year, but it just signed a $300 billion deal with Oracle"
- **Alex Kantrowitz** [0:42:00]: "turned Oracle into a nearly $1 1 trillion company almost overnight and made Larry Ellison the richest person in the world above Elon Musk."
- **Aaron Levie** [0:45:15]: "They're choosing to lose that money. They're doing it for a strategic reason."

## Transcript

**[0:00:00] Alex Kantrowitz:** Why are the headlines telling us that businesses are getting no return on AI investment? And are AI agents finally ready to get to work? We'll cover it all with BAC CEO Aaron Levie right after this. When it's time to scale your business, it's time for Shopify. Get everything you need to grow the way you want. Like, all the way. Stack more sales with the best converting checkout on the planet. Track your cha-chings from every channel, right in one spot.

**[0:00:30] Alex Kantrowitz:** and turn real-time reporting into big-time opportunities. Take your business to a whole new level. Switch to Shopify. Start your free trial today. How do you price intelligence as a service? How do you charge for delivering end-to-end work, not just access? Those questions remain front and center for AI founders and operators, and getting to the right fit business model, whether that's selling usage credits, outcomes, seats, or a hybrid combination requires constant iteration. Chargebee is the billing and monetization infrastructure that powers just that for ambitious, fast-scaling AI businesses.

**[0:01:06] Alex Kantrowitz:** Visit chargebee.com slash Alex to learn more. That's C-H -A-R -G-E -B-E -E dot com slash Alex. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. Well, today we're going to talk about AI and its application in business, whether it's actually making a difference and whether AI agents are a real thing. We have the perfect guest to do it today because we have Aaron Levie back with us fresh off the BoxWorks AI event.

**[0:01:41] Alex Kantrowitz:** And Aaron, it's great to see you as always.

**[0:01:44] Aaron Levie:** Thank you. Thank you, Alex. Good to be here.

**[0:01:47] Alex Kantrowitz:** Sure. So did I add BoxWorks AI event or is it just called BoxWorks?

**[0:01:52] Aaron Levie:** I actually like you calling it an AI event. It is just called BoxWorks, but anytime you want to jam in AI in there, we're good.

**[0:02:00] Alex Kantrowitz:** Okay, sounds good. You had a lot of AI news. We'll get into that in a moment. But since you are talking with a lot of folks about AI applications in business, I want to run this MIT study by you and get your perspective on what's real and what's not. So this is from Axios a couple of weeks ago. MIT study on AI profits rattles tech investors. Wall Street's biggest fear was validated by a recent MIT study indicating that 95 % of organizations studied get zero return on their AI investment.

**[0:02:31] Alex Kantrowitz:** They studied 300 public AI initiatives trying to suss out the no hype reality on AI's impact on business. 95 % of organizations said they found zero return despite enterprise investment of $30 billion to $40 billion into generative AI. This has been a study that everybody in the business world is talking about. Do you think there's any validity to it? You're already shaking your head.

**[0:02:57] Aaron Levie:** I'm shaking my head on actually like seven dimensions. We can parse each one. Let's do it. So, I mean, actually, maybe the first one that is maybe the most funny is the kind of Wall Street element. Actually, Wall Street is completely schizophrenic on this dimension. You know, obviously, a report like that, you know, scares them on one dimension. But actually, there's an equal amount of kind of Wall Street, maybe, you know, kind of frenetic energy around the idea that AI will be so good that all of software is dead.

**[0:03:30] Aaron Levie:** So it's this very kind of bipolar state of, you know, where are we in AI adoption versus AI is going to be so powerful that there's not even going to be software business models because everything will just be delivered by AI. And, you know, as with most things that have these kind of, you know, extreme polarization elements, I think the reality is just, you know, way more nuanced. We are still early in the adoption curve of AI.

**[0:03:57] Aaron Levie:** In the early curve of all of these types of technologies, you have lots and lots of proof of concepts. You have lots of trials of different technologies. People are trying to figure out which tool works for which use case. So by definition, you're kind of in the Wild West where there's lots of attempts at trying these technologies with various vendors and technology stacks. And many of those projects and pilots will absolutely fail because by definition, they're pilots.

**[0:04:21] Aaron Levie:** and we're still in the early phases. One interesting thing about this study was they saw a significant delta between companies that tried to effectively DIY their AI stack versus going with really kind of applied solutions and use cases. And this is what we tend to find in our customer base. So I think there was maybe an initial theory of, well, AI will be relatively easy to kind of get our arms around. We could build our own AI application.

**[0:04:49] Aaron Levie:** We'll do all of the vector embeddings of our data ourselves. We'll put it into a vector database. We'll manage the security and permissions of data access ourselves. And before you know it, a company that wanted to deploy AI in a particular workflow in their enterprise, they might have 10 or 15 different pieces of software that they have to run and manage just before a single user could actually interact with AI within that organization. So that's probably an architecture that's not going to work.

**[0:05:17] Aaron Levie:** You need to have purpose-built solutions that solve sort of tailored use cases. Those can be very big use cases like all of AI coding, but you probably don't want to be in a position where you have to kind of bootstrap this or build it all out yourselves. And that was one of the kind of recognitions in the survey. But I obviously wholeheartedly disagree with any of the maybe conclusions other than just you have to get your use cases right.

**[0:05:42] Aaron Levie:** You have to target the most effective areas for AI, and you probably shouldn't be building this technology yourselves. And so, but it's sort of empirical on our end. We get to talk to customers every single day that are seeing the immediate gains. We've talked to customers where they have had colleagues that can't actually, they can't present the actual ROI savings to their board. The actual kind of expected ROI savings to the board because the board won't believe how, they won't believe the numbers based on how good they are.

**[0:06:23] Aaron Levie:** So they actually have to water them down. So it's actually more pragmatic and believable based on what they're seeing. So -

**[0:06:30] Alex Kantrowitz:** Isn't that a terrible board? I mean, if the board can't hear the truth, probably need a new board.

**[0:06:34] Aaron Levie:** But the truth is so good that it doesn't sound incredible. So that is the, like, when the ROI is so good that you actually don't, you aren't going to be believed when you actually explain how this thing's going to work. So we're seeing examples all across the board, at least for our customers. You know, we have the benefit of a very applied use case, which is we take documents and unstructured data. And then we have AI agents that can operate on that data to do things like extract structured data from your documents.

**[0:07:04] Aaron Levie:** So give us, you know, 100 ,000 contracts and we'll pull out the structured data fields in those contracts. Or give us invoices and we'll pull out the key details in an invoice so we can help automate a workflow. Those use cases tend to be very high ROI because either you weren't getting that data before or it used to be very expensive to do so. And AI is getting increasingly good at being able to execute that kind of task.

**[0:07:26] Aaron Levie:** And so there's immediate benefit to customers. You can automate workflows much more easily as a result. You can lower the cost of operations in some areas. So we tend to see a different set of outcomes based on the AI adoption within our customer base. But if you zoom out and you kind of think about all projects across the past couple of years, I do think you're going to get a mixed bag just as a reality of how early we are in the space.

**[0:07:50] Alex Kantrowitz:** Yeah, and it says internal builds fail at double the rate of external partnerships. So spot on there. People trying to parse this together on their own versus doing it externally or having a tough time, which sort of flies in the face of like some of the conventional wisdom. I think the conventional wisdom was you wanted to be able to build internally, maybe with open source so you could customize to your use case. But it turns out some of the off the shelf stuff is actually working quite well.

**[0:08:13] Aaron Levie:** Yeah, I think you have to, you know, a lot of the challenge with either these types of surveys or even talking about architectures is you have to kind of separate the tech industry from the non-tech industry. The non-tech industry being the kind of consumers of these types of technologies and the tech industry being the builders. So open source is insanely valuable, but not in the sense where a law firm should go off and build their own AI project using an open source model.

**[0:08:41] Aaron Levie:** Like, that is just a recipe for disaster if, you know, we think that every single company on the planet is going to go build their own technology to go automate their workflows. And that has been actually the case for a lot of pilots because we've been early in the technology and you haven't had applied solutions that you could go deploy. But open source is actually extremely valuable for a company like Box because, you know, we're powering technology for 120 ,000 customers.

**[0:09:08] Aaron Levie:** And so we actually do have the expertise internally to leverage those kinds of capabilities. And so I would say the conclusion from the dimension of open source, as an example, is just you probably shouldn't expect that every company on the planet is going to DIY their own AI strategy. And that's a recipe for not getting the returns and gains from an AI adoption standpoint. And then maybe the other final thing I'd kind of point out is just there really is a decent amount of change management required to getting real gains from AI.

**[0:09:38] Aaron Levie:** This is not a panacea type of solution where you could take an existing workflow, drop AI directly into it, and then all of a sudden that workflow will be 3x better. You usually do have to re-engineer the work to take advantage of AI. And the conclusion I've recently come to more and more is, you know, I think we had this feeling maybe two or three years ago where AI was going to learn everything about how we work.

**[0:10:04] Aaron Levie:** It would be able to adapt to our workflows and then bring automation to our workflows. And I think realistically, increasingly, we probably will have to modify our work, hopefully incrementally, but in some cases meaningfully, to fully take advantage of AI. And that sounds maybe hard on one hand, but for the companies that do that, the ROI is going to be fairly massive. So if you think about AI coding as maybe the most obvious example right now where you're seeing productivity gains, the way that AI kind of first engineers tend to work is pretty different than how you engineered two or three years ago.

**[0:10:39] Aaron Levie:** The engineer really becomes more of a manager. You're deploying agents to go off and work on large parts of the code base, and then it's coming back with a bunch of work that you go and review. So if you don't change your workflow as an engineer to take advantage of background agents and how you give them the right kinds of prompts to actually execute on their task and the new ways you should effectively think about your code base and handling the specifications and rules of what the AI agent should do, if you don't do all of that work, you're probably not going to get a 2x or 5x gain from AI.

**[0:11:12] Aaron Levie:** And so we will actually have to re-engineer some of our business processes to make agents effective, as opposed to thinking agents will just drop into our processes and automate everything that we're doing.

**[0:11:23] Alex Kantrowitz:** By the way, you've brought up pilots a couple of times. And I think it's important to talk about because this study was not just pilots. It was 95% of organizations get zero return on AI investment. So I think the pilot thing is interesting because it's natural that pilots are going to fail. And in fact, we've had some listeners who've given me some feedback that said, because I talk often about how like only 20 % of AI pilots or 10 to 20 % of AI pilots get out the door into production.

**[0:11:50] Alex Kantrowitz:** And that might be a good number because you're going to obviously have some trial and error in the early days.

**[0:11:56] Aaron Levie:** Yeah. And to be clear, I'm using pilots colloquially in the sense that we're just so early in the technology that when we talk to customers, what a lot of times they have so far deployed is the equivalent of a pilot. just because of literally how organization-wide. Yes. Well, organization-wide is, you know, it's hard for one centralized survey taker to represent an organization-wide. It's like, that's why, again, that's why I don't want to like, the survey is great.

**[0:12:24] Aaron Levie:** It's an interesting, you know, kind of conversation starter. But like, if you actually tried to go assess, how is the answer, you know, answering this question and what is their way of measuring that productivity and have they actually surveyed all of the end users that are just using ChatGPT in an unsanctioned way and what they're doing. It's like, it's not possible to capture all of that. So it tends to more represent the kind of the centralized, heavily sort of, again, kind of, I think more likely pilot oriented type projects because of just, again, how early we are.

**[0:12:56] Aaron Levie:** The word agents just came onto the scene less than a year ago. So we're just early in a lot of these spaces. But again, I think it's a fantastic survey because it gets a conversation going. But I think if the takeaway was to slow down, you know, using AI or to do anything other than kind of realize what you should mitigate from a risk standpoint, then actually the failure would just be, or the problem with that would just be, all it's going to do is cause some companies to move even more slowly, and then you'll have other companies just outrun them.

**[0:13:26] Aaron Levie:** So it's kind of, you know, at the risk of, you know, you know, the risk is now on the listener to decide what

**[0:13:35] Alex Kantrowitz:** they want to do about that survey. Yeah. And I can tell you one more thing that I found super interesting about this study, which has sort of been underappreciated. So it says official LLM purchases cover only 40 % of firms, yet 90 % of employees use personal AI daily, at least those surveyed, which just is so interesting, because it means that, yeah, there's there's more personal use and more interest among individuals than companies to get this stuff uh into production yeah you obviously have a reaction here so let's hear it yeah well i know i just think that's that's

**[0:14:07] Aaron Levie:** like empirical revealed preference so so like you don't have to like you don't even have to survey once you know that why are people you know going off and and using ai in in a personal productivity sense um at that rate it's because they're they're getting value from it so you almost like that that That is sort of now in the baseline of how people are working. It's unquestionable that if you just sort of eliminated AI just today, let's just say, you would just notice, wow, okay, I actually have to go and do that three hours of research that I used to be able to go and kick off as a deep research project and go and check back in

**[0:14:41] Aaron Levie:** on it after five minutes. And so it's empirical that we're choosing to use these technologies on a daily basis because they're adding that productivity. And I would argue that what we've seen with AI thus far is barely scratching the surface of what is going to start to happen as you start to deploy these technologies.

**[0:15:00] Alex Kantrowitz:** But do you think the use in business, could it potentially be just individuals using, let's say, ChatGPT on their own versus scaled enterprise use of large language models? Or do you think it will be some blend?

**[0:15:14] Aaron Levie:** In the future?

**[0:15:15] Alex Kantrowitz:** You're obviously watching in the future because you're obviously watching this happen on the other side of things.

**[0:15:19] Aaron Levie:** No, the future is, I think that we are in the earliest phases of just even the diffusion of the technology itself, of the basic use cases of, hey, when you're going to go research a customer, you know, why don't you get a full account plan, you know, instead of just saying, okay, this person works at this company and they're interested in these things and these are the trends of that industry. Why not ask an AI system to generate the full plan?

**[0:15:49] Aaron Levie:** That's super powerful, but also relatively basic if you think about how people work and the full scope of workflows that people do. One really interesting example of, again, how early we are. Claude this week announced a new capability that will generate files for you. and even though we're two and a half years, nearly three years into the ChatGPT moment, it's the first time where an AI system can, I believe, generate reliably a kind of high quality document in the form of a Word document or a PowerPoint presentation.

**[0:16:24] Aaron Levie:** So we're nearly three years in and it's the first time ever that you could generate something that you would sort of look at and say, oh, that looks like a good presentation. So we are only at the very, very beginning stages. Now imagine, it'll still take a couple years, now imagine a technology like that begins to ripple through corporations. And in the future, before you go and present whatever product you're selling to a customer, instead of spending one or two hours of doing a bunch of research and making your PowerPoint file that's your presentation, you go to an AI agent, you say, I'm about to go sell to this customer, generate this presentation for me.

**[0:16:59] Aaron Levie:** You kick that off, and again, three minutes later, it's sort of done for you. this is going to just show up in all of our workflows every single day in almost everything that we're doing. So coders are getting the first lens into what the future looks like earliest because they're sort of wired to take advantage of these tools, and AI coding has been the kind of first breakout use case. But that same dynamic of you're going to go to an interface, you're going to talk to an agent, it's going to go and execute kind of multiple steps of work for you, that will start to emerge within all of knowledge work over the coming years.

**[0:17:33] Aaron Levie:** I actually am probably a pragmatist on this sense that it will not be like this instant overnight, you know, transformation of work. It will take years of change management. We just hosted our conference this week, as you noted, and it happens to be a crowd, obviously, by definition, that is sort of forward-leaning and kind of early adopters of technology, but that represents a small fraction of the total economy. It will take years before, again, all of the banks, all of the pharma companies, all of the law firms start to get wired up in this AI first way.

**[0:18:05] Aaron Levie:** But, I mean, unequivocally, it's going to happen. And there's nothing that will kind of slow that train down.

**[0:18:12] Alex Kantrowitz:** All right, let's talk a little bit more about this using Cloud to generate documents use case. I mean, I would imagine, so you talked, the example that you gave was using one of these to go in and sell into a client. Now, I would imagine most organizations, they have like their PowerPoint templates and the data baked in. So even if I were to go into Claude and upload my pricing spreadsheet, my inventory spreadsheet, a document about positioning and say make a PowerPoint based off of this, I'm sure it would do a good job.

**[0:18:40] Alex Kantrowitz:** But how practical is it to then say this is going to be a way that people do their work versus something that might look like a party trick where you're going to use the other documents that you have already when you actually are going to go out into market?

**[0:18:53] Aaron Levie:** Oh, yeah. No, the way that this will actually show up and I can't represent the exact date that this will happen, but Box, you'll just go to Box and you'll say, here's my sales presentation template. Here's the new client information. Please generate a PowerPoint presentation with that. And then you'll just do that with your existing data. This is not sort of some kind of one-off vibe-coded document. You will use your existing assets as the source material for the next document that you'll generate.

**[0:19:24] Aaron Levie:** And you'll go and review its work, and that'll take you three minutes. But it will have saved you an hour or two hours of all of the time that it took to do the customer research and move around all the graphics and put the relevant information in place. That will just be done for you. So, and that will, you know, multiply that over a million people that do that per day in, you know, some sector of the economy.

**[0:19:45] Aaron Levie:** And you'll just see, you know, that's how you'll get tens of millions of hours of productivity gained, you know, within the economy.

**[0:19:52] Alex Kantrowitz:** And how are you feeling about the trustworthiness of these models? Because you've talked a couple times now about how you could use deep research to prepare you for something. Or you could use these models to generate a PowerPoint and then spend a couple minutes checking them over. Are you at the point now where you think the outputs of these models are trustworthy enough that that's all it takes?

**[0:20:11] Aaron Levie:** I think as long as – and this is where I get very excited about now obviously what's in the zeitgeist is context engineering. As long as you are really good about what context you're giving the AI and how you are effectively grounding the AI in trustworthy data with the right kinds of prompts and a high enough quality model, you can nearly eradicate all of, if not the vast majority of hallucinations or accuracy issues. So in our case, everything that we do at Box is we think about your existing data as the source material for the AI agent.

**[0:20:47] Aaron Levie:** So it's the source context for the AI agent to be effective. And so if I take an existing PowerPoint document that's our sales presentation, and I say modify this for a new customer, and you do that with a frontier model that is a reasoning model with some degree of kind of thinking mode, I would posit that 99 % of the time it's going to make infinitesimally small kind of errors or failures on that. That's just like a solved problem at this point.

**[0:21:17] Aaron Levie:** And it is still easily worth the kind of five-minute tradeoff for the couple hours you save to go and review its work. And we actually have this incredible front row seat in watching what the future looks like with coding. So if you talk to the brand new startups, and I don't know if you do this, but I know that you get to spend your time with the dentists of the world and whatnot. But go talk to a five-person startup that's brand new.

**[0:21:46] Aaron Levie:** And what's exciting is they're working in the craziest ways that I've ever seen in my entire life. I was talking to a nine-person startup the other day that estimates that they're at a minimum executing at the size of about a 100-person company. And that was, again, kind of conservative probably when you do the underlying math. And it's because each of their engineers has the capacity output now of 5 or 10 or 20 engineers worth of work.

**[0:22:11] Aaron Levie:** But they are working in a completely different way. They are managers of AI agents. They spend their time on writing really good specs for what they want to build. They spend a really good time on the design architecture of their software. And then they spend a lot of time on reviewing the output of the agent. So not every area of knowledge work will look exactly like that. But if you imagine in sales, if you imagine in marketing, if you imagine in legal work, and your role is to manage agents that are doing a lot of the underlying data preparation, research, creation type of work, and then your job is to go review that work and put it together in a

**[0:22:52] Aaron Levie:** broader business process, that will actually be what a lot of work looks like in the future. And this idea of hallucinations or errors will be no different than the fact that I have to sometimes review other people's work and other people review my work. And I have errors in the presentations that I create that somebody catches and they see a misspelling or they see that I changed the name of a customer in the wrong way and they change that.

**[0:23:14] Aaron Levie:** We will be doing that for AI agents. So it's this flip of the model where we thought AI agents were going to review our work and kind of incrementally make us more productive. We will be the reviewers of the AI agents' work. We will be the editors. We will be the managers. We'll be the orchestrators. And that's actually how you then get the productivity gains. So I'd say watch the AI coding space. Watch what startups are doing to get leverage.

**[0:23:37] Aaron Levie:** And then think about that against the broader economy.

**[0:23:40] Alex Kantrowitz:** You know, it's really interesting, Aaron, because the last time we spoke, you told me about this person that you knew who was basically building a company on their own using AI coding tools. And so I was in the process of writing this profile of Dario at Anthropic, which you're quoted in. And I went out and found a developer doing something quite similar using Cloud Code to build on their own. So this is clearly, I mean, to the point where like Anthropic now has to put some rate limits on.

**[0:24:06] Alex Kantrowitz:** But this is clearly a thing that's happening.

**[0:24:07] Aaron Levie:** And this is the thing that, again, still, I love the MIT survey. I think it's great. It's a fun conversation topic. But the one travesty would be if people miss that what you just said is actually happening on the ground and then not starting to pay attention to what that's going to mean as that ripples through corporations and how people should probably start to think about reengineering workflows for a world of AI agents. And, you know, this happens in every single technology, you know, wave, which is actually why you have early adopters and early innovators and why you have laggards is because the early adopters and innovators are going to read, you know, your anthropic piece and see, oh, this

**[0:24:49] Aaron Levie:** actually is a real trend. And the laggards are going to read the MIT piece saying, oh, I've been vindicated. And some companies will then get those early returns at a much faster rate and other companies can wait. And, you know, sometimes that means that your company gets disrupted and sometimes it doesn't because you actually have, you know, some proprietary, you know, capability as an organization. Like if Pfizer or Eli Lilly took a little bit longer to adopt AI as a result of, you know, wanting to be more pragmatic, that'll be totally fine.

**[0:25:19] Aaron Levie:** They're not going to get disrupted. Like they have enough market position. They have enough distribution. They can afford to kind of wait for this technology to be more baked. But if I'm a startup right now, I'm probably going to use that as my advantage as much as possible to try and run circles around maybe a larger incumbent. And this is what kind of creates this nice tension in the market that creates creative destruction in every kind of wave of technological change.

**[0:25:44] Alex Kantrowitz:** Okay. I definitely want to speak a little bit more about what the definition of an agent is and how you're rolling them out at box and also get your reaction about GPT-5. So let's do that right after this. When you're at work, you never know when you'll be interrupted. But with the Dell Pro, powered by Intel Core Ultra with vPro, no matter what distracts you, your laptop won't. It's battery optimized for the way you work, with built-in intelligence that quiets distractions when you need to focus.

**[0:26:11] Alex Kantrowitz:** Your laptop will help keep you locked in, even when it's bring your dog to work day. Built for those who stay in the flow. That's Dell Pro, built for you. Dell.com slash Dell dash pro. Everyone's talking about AI, but there's a big difference between running a promising pilot and actually making AI work across an enterprise. That's why we're launching the Big Technology Guide To series presented by DXC Technology, available on YouTube. Throughout the series, we're tackling some of the biggest questions facing businesses today.

**[0:26:45] Alex Kantrowitz:** How do you take enterprise AI from pilot to production? How do you navigate an increasingly complex cybersecurity landscape? What's the real opportunity with AI agents? And how do you calculate the return on your AI investments? On my Alex Cantor's YouTube channel, you can hear from experts like DXC Technologies Chief Digital Information Officer, Russell Jukes. When you ask a question to somebody in your company, do you trust them? And what made you trust them? Do you trust but verify every single time?

**[0:27:11] Alex Kantrowitz:** Russell and I discuss the challenges and opportunities of building production-grade AI and what it really takes to move beyond the hype. Check out the Big Technology Guide To series presented by DXC Technology on my Alex Kantrowitz YouTube channel today. And we're back here on Big Technology Podcast with BAC CEO Aaron Levie. Aaron, let me start, before we get into agents and before we get into GPT-5, let me just start with a basic question, which is, if this is already happening in business, which is basically like you're finding ways to get the AI to do work on its own and pull information from different data sources and present it coherently.

**[0:27:51] Alex Kantrowitz:** Why do you think it's been so difficult for consumer companies like, let's say, Amazon with Alexa Plus and Apple with Apple Intelligence to put this together as something on device or a consumer product that does similar activities? Because they've all promised it, but it's not quite there yet.

**[0:28:10] Aaron Levie:** Yeah, I think the fact that the technology can exist is different from still the execution requirements to bring it to life. And so we get to all have a front row seat on what the frontier models can do. And you have companies that can package those up in a way for these applied use cases. But if you're a company with tens of millions or hundreds of millions of users of your product and consumers that have a certain expectation, And that is a lot of execution gap required to go from the frontier model to how do you deliver that to your end customer in a reliable way that is trustworthy, that is affordable.

**[0:28:48] Aaron Levie:** And so I think that the bigger companies are all going through their own version of that motion. I'd also imagine that given the space is moving so fast, I can sympathize for probably some degree of indecision maybe, where one day a model is on top and then the next day a different model is on top and another day another model kind of breaks through. And so you probably want to make sure that by the time that you land on a final architecture, you want that to be the sustainable long-term architecture.

**[0:29:16] Aaron Levie:** And so to some extent, time is on your side up to a point because you might want to wait to see kind of who falls out and who keeps going. But I think, you know, as an example, the companies you just mentioned, like I don't think those spaces have been so utterly disrupted that they can't catch up once they land on a final architecture. But, you know, we'll have to see kind of how they execute through this.

**[0:29:44] Alex Kantrowitz:** And so for business, it's more that there are more prescribed use cases. And I think with a phone, maybe, if you're trying to get these proactive notifications, then you're looking at a massive universe of data, whereas you're more concentrated in business. Or what's the difference?

**[0:29:58] Aaron Levie:** Well, actually, I wouldn't say there's a difference. I would say even in business, we're insanely early. Like we have to process how early we are. The breakouts so far have been ChatGPT for consumers. The breakouts have been coding agents for very, very wired in engineers that are very online. They're paying attention to everything going on. And then early adopters across the economy. Most of the agents that are being deployed in the enterprise are being done by, maybe you can flash it up or something.

**[0:30:34] Aaron Levie:** Jeffrey Moore came up with this idea of the technology adoption curve or at least popularized it. It has multiple categories of where a company or a group of individuals will be. You have these early innovators and early adopters. Then you have a chasm. Then you basically have kind of pragmatists and early majority. And then you have laggards. And we are in the early adopter, kind of the earliest phase of jumping over the chasm on some use cases.

**[0:31:00] Aaron Levie:** But we have to imagine there's this chasm where what happens is the early adopters, the people that we all hang out with and talk to all day, they're going to try everything. We're going to try these crazy goggles and we're going to put magnets on our head and we're going to do the craziest things. We're going to wear Google Glass. And that actually tells you almost nothing about whether the thing will jump over the chasm. you have to actually see like what makes it to the early majority or those pragmatists that really adopt things at scale.

**[0:31:31] Aaron Levie:** And so the kind of technologies that have clearly broken through our chat to BT, products like Cursor, products like, let's say, you know, a bunch of these kind of next-gen research agent type things, perplexities done well in that kind of early majority. But we're so early in terms of AI agents jumping over now the chasm. So some won't make it, some will. But I would say that business is not particularly moving faster than the examples you just gave.

**[0:31:57] Aaron Levie:** I just think we can see lots of examples of it, but they're usually in that kind of early adopter type category.

**[0:32:03] Alex Kantrowitz:** Right. And so the week we're talking, you at Box are releasing a number of different agents. Let me start this discussion by just asking you, what is an agent? Because it does seem like it's an overused term. And even myself, who I'm in this all the time, I don't fully have clarity on what that word actually means.

**[0:32:21] Aaron Levie:** I think we should anticipate that it's fully overused. It is now the new term of art for talking to an AI system that is doing work for you. So this will be the main term that we use going forward as an industry. And not because it's a buzzword, but actually it's a useful term. It's a definable object that is doing automated work for you. That could be, in some cases, as simple as answering a question. but I think most people in the tech industry would generally argue that it should be doing some degree of work and looping through the AI model multiple times to do that work.

**[0:32:58] Aaron Levie:** And so that could be everything from very clearly something like Claude Code or Cursor has an agent or Replit has an agent where you give it a task like build me a website that has these qualities and it will go off and do weeks worth of human work in 10 minutes. and that's an agent that is managing that whole process, looping through the model multiple times, keeping track of what it's doing, updating its memory in the process and that's effectively an agent.

**[0:33:27] Aaron Levie:** So that's an agent in coding and we're going to see that same kind of agent architecture emerge in law, in healthcare, in finance, in education where you can deploy agents to go off and do work for you. And there'll be a critical access which is how much work can the agent do before you have to intervene and modify and kind of repoint it in the right direction. And so a lot of that work right now can be maybe a couple minutes long, but we're seeing examples where agents could be running for tens of minutes or maybe even hours and effectively drive better and better and more high-quality output.

**[0:34:02] Aaron Levie:** So I think that's a way to think about agents, and these are going to be very pervasive in the coming years. But this is really the first year. 2025 is the first year where we could even really be talking about it seriously. And I think Andre, you know, Carpathia had a, you know, probably phrased it as we shouldn't think about this as the year of agents. We should think about it as the decade of agents. That's probably the right way to think about it.

**[0:34:28] Aaron Levie:** This is sort of - It's like the year of mobile.

**[0:34:29] Alex Kantrowitz:** It became the decade of mobile, but then eventually we started using mobile.

**[0:34:33] Aaron Levie:** Yeah, but the, and again, when you just said the year of mobile mattered, right? Did people say, you know, some people said that was in 2022, but probably the first time it could have been realistic was 2000, sorry, not 2022, 2002. Some people, but it wasn't really realistic until 2006 and 2007 when you had the iPhone. So, you know, I and I think fairly many other people are actually convinced we already have our iPhone for agents.

**[0:34:58] Aaron Levie:** We don't need any kind of new breakthrough architecture. We have an architecture that already kind of works as the core scaffolding for agents. So we can start the decade kind of clock now. But it will be a full self-driving type problem. Obviously, Waymo got kicked off, I don't know, a decade, decade and a half ago. And only this year is it accessible in suburban Silicon Valley. So what took a decade or a decade and a half?

**[0:35:27] Aaron Levie:** It was just lots of engineering work, lots of miles on the road, lots of improving every single dimension of the accuracy and the intelligence of the system. We are going to see the same thing for knowledge work. It's going to take years. The early adopters will get the early returns. The pragmatists will use it once it sort of works without a lot of hand-holding, and everybody will land somewhere in the middle of that spectrum.

**[0:35:54] Alex Kantrowitz:** Okay, and so I watched a chunk of your presentation this week, And some of the agents that you're talking about enabling companies to deploy will be things that will, for instance, take a look at a application to be involved to maybe take an apartment out or to look at some property records and then do tasks there or to create reports, looking at clinical tests and trying to pull out issues. So talk a little bit about how the process to create these works.

**[0:36:26] Alex Kantrowitz:** And is this still in the demo phase or is this actually real?

**[0:36:32] Aaron Levie:** So maybe second question first. So we made a number of big announcements this week. Some of the product and capabilities that we announced are fully GA right now. So customers can already start to use it. Some of it we kind of give a little bit of a crystal ball view into the next couple of quarters of the product that we're getting out there. As an example, we have an AI agent right now that any customer can go and use, which is a data extraction agent.

**[0:37:01] Aaron Levie:** So you can give us, again, contracts or invoices or medical data. And then we have an AI agent that works through that content, pulls out the critical data from those documents, and then lets you go and automate a workflow around that. What we announced at BoxWorks was a new capability called Box Automate. And what the idea of Box Automate is, is it's very, very powerful to have one-off agents that can help you review a document or generate a proposal or generate a sales plan for a client based on data.

**[0:37:32] Aaron Levie:** That's super powerful. But what's even more powerful is that I can drop many of those agents into a full business process. So what Box Automate lets you do is actually define your business process within Box. It could be a client onboarding workflow. It could be an M &A due diligence review process. It could be a healthcare patient review process. And you define that workflow with Inbox Automate. It's a drag and drop kind of workflow builder. And then at any point in the process, you can bring in an AI agent to do work within that process.

**[0:38:02] Aaron Levie:** And so one thing that is very important with AI agents is they need the right context to be effective. So our system allows you to get that context to agents from your enterprise content. So your marketing assets, your research data, your contracts, your invoices, that becomes very important context for agents. So Box Automate lets you basically build these agents on demand or on the fly in a workflow that leverages your existing content. And then we can start to help you automate a bunch of knowledge work tasks around the enterprise.

**[0:38:33] Alex Kantrowitz:** Now, a lot of the early reviews around GPT-5 was it was sort of built to do these type of things or like as a foundational layer for this type of work, right? The reviews we read early on was that it just does stuff. And there have been people that have noticed that like when you're in chat GPT using GPT-5, you like literally can't have an answer where it doesn't say, can I do something for you?

**[0:38:55] Alex Kantrowitz:** So I'm actually curious, Aaron, what your response has been. The last time we spoke was pre-GPT-5. what your feeling has been about this new set of models, really it's a set of models. And I'm curious like what you make of the fact that so many people were disappointed early on.

**[0:39:11] Aaron Levie:** Well, yeah, so we, on the disappointment or kind of online zeitgeist, which actually interestingly has already shifted, I think, you know, quite a bit where a lot of folks have kind of updated their views on GPT-5 and I think Codex has come out very strong recently on the coding agentic side. You know, I think we have gotten used to, and we've been hooked on these incredible kind of jumps and breakthroughs over the past year or so.

**[0:39:44] Aaron Levie:** We went from, if you think about it, we went from GPT-4 to GPT-4 -0 to 0-1, 0-3, and then GPT-4 -1, and each of those on a different axis was actually a pretty meaningful step function. So if you had just taken GPT-4 and then you jumped to GPT-5, it would have looked insanely exponential. But we got these points along the way that effectively kind of gave us an early preview into what GPT-5 would ultimately become, which is a thinking model, a chain of thought with a way higher quality of coding skills and a bunch of capabilities on critical dimensions of work.

**[0:40:24] Aaron Levie:** And so I think it was mostly just driven by the fact that we got lots of incremental steps or step function steps on the path to GPT-5. And then GPT-5 was just the culmination of a lot of those breakthroughs. So, again, I think it's probably more psychological than kind of empirical. Like I think if we had gone from, you know, three to four to five, it would be the most vertical axis we've ever seen.

**[0:40:47] Aaron Levie:** But it was really, again, those steps along the way that maybe caused a little bit of that kind of reaction. In our world, we test every single model on a number of evaluations where we give the model different types of enterprise data, contracts, financial documents, research materials, internal memos, those types of things. And we ask the model a series of questions about that document or data. And we saw meaningful improvements from GPT-5 versus GPT-4 -1, as an example, on our eval.

**[0:41:17] Aaron Levie:** So for us, it was multiple points of improvement on a number of our key tests. And those improvements then translate into real-life improvements for customers, where they all of a sudden will mean that when you're a healthcare provider using GPT-5 on unstructured healthcare data, you're going to get better results than you got before. Or when you're using it on your contracts, you're going to get better results. And so on a number of spaces where either it was kind of expert analysis required in health care or law or financial services, we saw improvements.

**[0:41:52] Aaron Levie:** Or in a general sense, if you needed logic or reasoning or math, it was also an improvement on those dimensions as well.

**[0:42:00] Alex Kantrowitz:** Can I get a quick gut check from you on the economics of the AI industry right now? I mean, we are talking at a moment where we just talked about this on the Friday show with Ranjan that OpenAI's losses are now going to total $115 billion through 2029. Oh, sorry, it's cash burn. $115 billion through 2029, $80 billion higher than it previously expected. It's expected to make like $10 billion this year, but it just signed a $300 billion deal with Oracle that like turned Oracle into a nearly $1 1 trillion company almost overnight and made Larry Ellison the richest person in the world above Elon Musk.

**[0:42:37] Alex Kantrowitz:** How does this make sense?

**[0:42:40] Aaron Levie:** Well, I think it makes sense if you believe like I do and certainly others, you know, Jensen, clearly Sam, even Elon, I think would believe that this is the single biggest technology that we've probably ever had access to. And so if you think about this as sort of a third industrial revolution, where for the first time ever, we can bring automation to knowledge work. Just think about that for a second. We bring automation to knowledge work.

**[0:43:10] Aaron Levie:** Everything about the world of knowledge work was always basically limited by how fast we as humans could work. We could type into a computer, put data into a system. Somebody else reads that data. It moves along in some kind of process. That was about the speed of knowledge work was how quickly we could type or read information and then do something in the real world with that data. That was the rate of pace. That was the pace that knowledge work could happen at.

**[0:43:35] Aaron Levie:** And so every field that we know of in kind of knowledge work, you know, healthcare experts reading, you know, medical diagnoses, life sciences experts that are doing research on clinical studies, lawyers that are trying to find facts about a case or, you know, working through intellectual property, an engineer trying to generate code and read product specifications. All of that work has always been constrained by how fast we as individuals can do that work individually ourselves. For the first time ever with AI, we can bring automation to effectively all of that work.

**[0:44:13] Aaron Levie:** And that automation can kind of be tuned based on just how much compute we throw at the problem. And then, of course, how good our data is and how effective our systems are at getting that data to the AI. But in a world where you can toggle compute and then get different levels of automation and effective output in work to get done at a way lower cost than what people can do, that is the biggest breakthrough we've ever had in the economy and in the kind of post-industrial world.

**[0:44:45] Aaron Levie:** And so, you know, $100 billion of loss, let's say, to get to that point of, you know, saturation where that technology is out there, that's a very, it's actually a very small number when you think about the economy and the size of the economy for all of healthcare, all of law, all of life sciences, all of financial services, all of engineering. So I think that's how these technology companies are underwriting this. And the losses are a choice, to be clear.

**[0:45:15] Aaron Levie:** That's very obvious. They're choosing to lose that money. They're doing it for a strategic reason. That's at least their decision. The strategic reason is that this is such a valuable market to own and to dominate in that they would rather build up capacity and, in many cases, subsidize usage, let's say in free consumer tiers of ChachiBT, then charge everything at today's rate of cost and then make sure everything is profitable. That's a choice. They could decide to charge for everything.

**[0:45:47] Aaron Levie:** They would get less adoption today. They would be instantly a more sustainable business. But enough people believe that the prize is big enough that it's worth actually doing all of the research expenses, all of the data center expenses and the subsidization where necessary to drive that adoption and demand. And it's a go big or go home type of bet. You know, clearly very, very smart, very economically rational firms, individuals, sovereign wealth funds believe that that bet is worth it.

**[0:46:17] Aaron Levie:** I'm probably on the side that the bet is worth it because of, again, how material of an economic impact this technology can have. And then we'll obviously see how it plays out with any kind of individual player in the space.

**[0:46:29] Alex Kantrowitz:** folks you can learn more about boxes offerings at box.com there's a video playing on the home page right now that talks a lot more about the things that aaron and i have discussed here today aaron so great to see you thanks again for coming on the show thanks alex all right everybody thank you so much for watching we'll see you next time on big technology podcast This episode is brought to you by Luva. You're here to listen to your podcast, not talk about car insurance.

**[0:47:07] Alex Kantrowitz:** Luckily, Lu, Luva's AI-powered guide, can do the talking. Lu helps you compare actual rates from top-rated carriers, walking you through each step and answering questions about your options until you're confident. It's free to use with no social security number or hard credit check required. Searching for car insurance just got clearer. Compare quotes at Luva.com. Build your front-end and back-end together on Base44. Describe the product logic with a managed database, authentication, and hosting included.

**[0:47:38] Alex Kantrowitz:** Spend less time assembling infrastructure and more time getting your product live. Go to Base44.com.
