Mozilla CTO: Why Most Enterprises Don't Control Their AI

Pinterest deployed open models instead of closed and saved something in the order of $10 million that quarter alone.

Open source software built the internet, but it's losing the AI war. Rafi Krikorian is CTO of Mozilla, which makes the Firefox browser.

Most enterprises are renters, not owners. They're talking to either big platform companies, they're making API calls that are off prem, they're doing all these things. You're basically turning over your your destiny over to a different system that you might be making a call to. So in that definition, yes, most of them are renters. And we're starting to see a bunch of this even inside of Mozilla of like how that renters versus owners situation is playing out.

Like, you know, I was just looking at one of our Claude Code leaderboards within Mozilla. We're using it on one of our subsidiary companies, MozVC, and the one of the engineers was effectively could have racked up a 10,000 API bill, uh, this month. But thankfully, we're paying for the 200 a month subscription. But you know, that subscription cost could change at any moment, we'll probably change post IPO. Um, so all those things, I think in my mind, collapse into this, uh, renters and not owners framing.

But Michael, I wa- I wanna touch on that first one. I think the question of whether open source is losing the AI war is more of a question of just like, what do you think the war is? Like, if it is it losing on a bunch of leaderboards, then sure, we're like, it's losing on a bunch of like demo cases or like the extreme cases. The question for me is really like, like, is it gonna win on the everyday use case?

Is it gonna win on the things that most enterprises care about, most companies care about, most regular users care about? Like, it's probably losing or definitely losing on the frontier bot battle. But like, in the same way that, you know, we woulda said at some point that Linux lost the desktop. But at the same time, Linux effectively runs every computer on the planet, and then there's a few iPhones lying around, like, kind of thing. And so, like, I think we could be in a similar situation when it comes to open source, open weights and the AI world.

And like, on top of that, I really think that, you know, I think there's a lot of evidence that shows that the open models are compressing their difference between where their closed models are. So I think this battle is long from over, this war is long from over. But I think you're right in, I mean, in today's parlance, it might be losing. But like, I think the curves are starting to compress somewhere.

Walk us through the issues, the problems. When an enterprise is essentially beholden to the major model makers, to closed systems,

A lot of this comes down to just a question of what does it mean to control your destiny? What does it mean to control the optionality that you're sort of working in? So if I, if I were to rewind the clock, like, I think that when we were all talking about cloud and cloud deployments, I think we all got into this position where we're just like, well, we can't just have one cloud provider. Like we in the enterprise case, or the people who are trying to deploy large systems case.

Like, we need the ability to hitch, we need the ability to actually choose and be able to play off vendors from each other. Do all the things that we want in order to make sure we're getting the right value for our cash, we're getting the right features that we want. So, like, you know, in a lot of ways, we didn't build raw to people like AWS and others. You use tools like Terraform or something else in order to build an abstraction layer so you could quickly move between them.

So when AWS shows up one day and it's like, I wanna give you a huge bill this month, we can be like, well, fine, we'll pay it, but next month we're moving. And then that starts a negotiation. And I think we're in the same place right now when it comes to all these open, oh, sorry, like these AI tools and the enterprise. You know, Zapier just did, I mentioned this in my newsletter this week, Zapier just did a survey of talking to people, talking to enterprises about whether or not they feel like they could switch off their AI provider.

Something like eighty five, 86% of them thought they could, but when they actually tried to do it, only about 30% of them could actually pull it off. So there's already so much lock in happening. And so you get into this situation where, like, these things aren't under my control. And then those, and then the providers can just change model behavior at any given moment. If they're having capacity issues, they could just shuttle you off to a different model that might behave differently.

So therefore, I've lost control of even just how I'm programming against this. And then that thing I mentioned earlier, which is costs right now, I feel are both a little both kind of inflated, but also a little under control. I would've predicted a few months ago that token costs were gonna go down, they kind of have, but token usage has skyrocketed. And a lot of that could also be because of model change, and now new models and new harnesses just need more tokens, which is in the benefit of the people I'm buying from.

So I feel like all these situations, if I were an enterprise buyer, would be like, this seems a little out of my control right now. And that so I think that's what I really mean.

so would it be accurate to say the issues that you've brought up are loss of control, uh, of of the model, which is ultimately all important here, loss of control on, uh, pricing. And obviously not token usage, because that's gonna be driven by by the user. But essentially, you're at the mercy of the large platform providers.

Precisely, I think, like the models could be changing at any given moment. So therefore, writing code against them or write or trying to codify your behavior against them. It's not under your control, it's under control of the provider. That seems not great. It might be fine in a prototyping sense, but not, uh, like a large scale deployment sense. I think what you what we mentioned on the cost, I think that's definitely one thing that's a little bit out of control.

And I also do think token usage is a little bit not fully in your control. Sure, you can decide how much or how complicated of a task you're asking. But like, once I set off my once I set off my initial prompt, all the different things that happen in the harness could just exponentially grow, and that is also a little bit out of my control.

Folks, we're talking with Rafi Krikorian, who is the CTO of Mozilla. You can ask questions. If you're watching on LinkedIn, just pop your question into the chat. So, Rafi, one of the points that you mentioned was the lock in. Is how big an issue is that, and and how does open source solve it? And what are the solutions in general to this lock in platform problem that you were describing?

I do think it's a big problem in the sense that if I'm running software within my business, I want to know that I have optionality. I want to know that I can move between different people depending on feature sets, depending on exactly what I want to go do. Like, I don't think it makes sense for people to assume that one provider is gonna do everything for me. I wanna be able to, like, finely grain and tune it so I can use the features of a provider that match the things I need to get done.

So in some way, I do think locking's a big deal. I think the Zapier survey is a good example of just like how people are implicitly getting locked in, like they, I don't think they willingly walked into that situation. In fact, they believed they could walk out of it. But it turns out to be quite hard to go do for a variety of different reasons. And so I think open source is one of the solutions, it's clearly not the only one, but I think it's one of the different solutions.

Like, you know, the open source models are both getting really quite good at what they can do in the enterprise work cases. Like, we've been experimenting with the open models within our Mozilla.ai group, and that group is building developer facing tooling and enterprise facing tooling. So they've been understanding and better using what those models look like. I'm using an open model on my day-to-day coding tasks, like I run Quen 30B on my laptop, especially when I'm in on United Wi-Fi and that thing doesn't work.

So it provides me a really good experience for my laptop. So I can pick and choose the kinds of things I wanna go do. So if you already live in a world where your assumption is I should be able to pick and choose, then the open models are like, give you that ability to do it. And then we've been working on a set of products, one of them that we call Otari. It used to be called Any LLM.

And it's basically a router that you can install on your desktop or we can run in that cloud environment. That if you point to it, you it can then switch rapidly between different models, depending on exactly what you wanna do or what your particular workload looks like right now. So just even embracing the idea that you should be talking multiple vendor, multiple model, and just going into that mentality, I think gets you into a more stable spot.

And then you can start really having a conversation of just like this model for this use case, this model for this use case, this provider for that use case. It just seems like a more resilient place to be. And if you're in that space, then open source is clearly gonna be part of that solution. Might not be the entire thing, or for some people might be the entire thing, but you can then be the one to choose.

The focus then for you is on the user's ability to make those choice decisions as opposed to leaving it to the third party vendor, essentially.

Correct, I mean, if you were to, like, abstract away my entire position here, it's basically some question of choice and, uh, competition. Like, how can you actually have a user end user be the one that's making the decisions? And how can they have a vibrant ecosystem that they can be choosing from and actually selecting the things that they want to be using from that menu of different things? So if we can get to that world of open systems combined with open source that allows for choice and competition, I think my job here is done.

Um, we are very far away from that world right now. There are lots of things between here and there, but that's the world I'm trying to get to.

We have some questions that are coming in, and so let's jump to those. The first question is on Twitter X from Anthony Scrivenano, who is a prominent data scientist. He does a lot of work with, uh, consulting for various agencies with three letters and things like that. And Anthony says this. He's been a guest on CXO talk a number of times. He says, Drilling down on your comments about model change without notice, please talk a bit about the challenge of regression testing, simply proving that things still work the way they used to work.

Are we losing the battle?

Last year, there was a really big push into building evaluation suites, building tooling to do evaluations, et cetera. Evaluations are incredibly important. The academic computing community does it all the time. And I think what we've lost the muscle of, and we need to figure out how to bring back, is for all of us to be doing our own evaluations on the workloads that we really care about. So, you know, in some ways, we look at evals all the time, right?

Like we look at the coding bench, we look at all those things, understand how well models are doing. But those are very much in the abstract cases, and frankly, they can be, I mean, I'm hesitant to be saying gamed, but they can be tuned for. You can tune yourself to be able to pass a particular benchmark. But if instead, what we were doing is recording all the different prompts and all the different tasks that we're doing locally, and then creating tooling that allows for model evaluation ourselves, then we could easily figure out, or maybe not easily, but at least have a framework to figure out.

Whether or not a model selection choice or an agentic harness choice, whatever, how that's going to perform on my actual personal workload. So I've actually been thinking a lot about this. Like, I actually wrote, you can find it on GitHub. I built this thing I called Morph, which is basically Git, but for prompt-based workflows. So if you think about what Git does, Git allowed me to take computer syntax that was then compiled down to bytes and stuff those bytes into a version control system.

I don't think that really works anymore on these, like, LLM-based coding tasks. And what you instead need to do is you need to record the prompt and then record the output of the prompt in that case, which is code. Like bytes might be at the wrong abstraction level. So if you start doing that, so if we start using version control to manage what our prompts look like, like, what's the prompt, what's the harness, what's the model, when I ran it, all the settings and stuff like that, then we can have reproducible use cases of understanding what my real workload looks like.

And then I can start playing that back against other models or other versions, like, for example, Opus 4.8 just came out. Everyone is super excited by it. I'm super excited by it. But one of the first things I'm gonna do this weekend is I've been recording all the prompts of all the different pieces of code that I've been tinkering on. And I'm gonna run those prompts against Opus 4.8. So I can just then start doing comparison of just like, oh, actually, if I had this model six months ago, this is the direction this piece of code would have gone into.

So if we can start thinking about how to build those kind of workflows, both in the enterprise case and in the personal coding case, then I think we can get to a more intelligent conversation of like, this model is actually better. Because, you know, at some point, you might need the Opus 48, or in some cases, you know, Quen 3530B might be just good enough. And so understanding what that balance looks like, we can only really do that with data, so we just need the right tools to record that data.

I'll say, amen to that. I mean, this whole situation is very ad hoc right now, choosing models and figuring it out. I personally right at the moment pay for the 200 plan for Anthropic and the 200 plan for Perplexity. Um, and I say to myself, who really knows? Anyway,

yeah, exactly. I mean, like, I mean, if you look at me, I mean, like, I'm spending more money on different AI plans than I spend on my cable bill or, like, my television streaming bill right now, right? Because, like, to your point, I'm just like, I guess I'll try them all and just see which one works best. But, like, we need a better definition of what works best means for Rafi, what works best means for Michael, so that we can actually then have the tooling that can figure it out for us, stuff like that.

we have some more questions coming in, and this is from Chris Peterson, and I encourage you guys ask your questions, take advantage of this opportunity. All right, uh, Chris Peterson again on Twitter X says, Do you think that tiered usage plans will continue long term, or will all the big AI providers be forced into per token pricing as venture capital wants to exit and not keep funding?

There is no clear exact answer, but I try to look for equivalent analogies, right? So I used to work at Uber, I ran the self driving car division there, and you could see it even in Uber that, like, post-IPO, the prices changed, right? Like all the subsidization that we did of Gen Z just basically disappeared and the real, true pricing showed up in that situation. And I'm likely to believe that this is a similar marketplace that with all that subsidization, that like 200 plan that my Moz engine, my Moz AI engineer who then did analysis, like I would have spent 10,000 in API calls.

I think it won't go to the 10,000 case because I think that sort of like locks the market, um, sort of like limits the market. But I do think it's not gonna be these 200 plans. I think that that price either floats up significantly or we go into this per API model. At which point you need to then start making the decision of, like, do I use Opus, do I use Sonnet, do I run it locally, do I do something else?

So I think that, like, I think the best situation is to start thinking about this evalu- those kind of questions now to try to find the right model for the tasks that you're doing. Because I do think the price will change. I think you're right, it's just unclear exactly how it's gonna change. And if you think that I'm wrong, and I might be, then you can make other different arguments of just like, do I really need to spin up an entire data center's worth of GPUs?

If I'm just like, write me a piece of code that says hello world, right? Like, then just as, like, good engineers, that feels kind of icky. And so trying to figure out, like, what do we think is like the right engineering size for the right tasks I'm doing could be another angle to look at it.

I always want to use the best possible model because if I'm asking an an LLM for a solution, I want I. I have some intention and some reason behind it. I want the best possible result.

I don't disagree with you. I mean, like, I've spoken to, you know, I was speaking to an engineer the other day who is making the argument that his bosses only allowed them to use the frontier models. Because they have a belief that it'll make them more efficient, that the frontier model will probably give them the right answer the first time, and they're not gonna sit there and whittle it away. And I and I guess, like, my response to that is just like, it's a good belief, but where's the data behind it?

So, like, this is, this is why I'm really harping on the. We need to be recording our prompts, we need to record what models are on, we need to personally do our own evals of this. So we need good tooling to do that so we can actually then make these real logical trade-off decisions. Of just like, I think I think I agree with you, Michael. Like, I would like to be using Opus 48 the entire time until I get the bill.

And at which point I'm just like, okay, but like, was that really worth 1,500 this month? Or could I have done that with 500? Like kind of thing. So I think it's just when, like, the rubber meets the road, like, fitting the right size is gonna be the thing that we all need to do.

Here's a question on LinkedIn from Tim Crawford, who is a major CIO advisor. And he says, on the topic of pricing, it seems that outcome based pricing is not granular enough, but token based pricing is harder to understand. Is there a good alternative? It seems this is part of what is causing the drive to locally sourced AI platforms.

we've been facing this challenge as we figure out pricing on our different products as well. So the Moz AI team has something that they've released that they call Octinus. It's a fully open source, um, agentic workflow system. So, you know, think of it as like a really good agent with a really good UI that helps you inside your enterprise. Connect your email to Notion to Zapier, et cetera, et cetera, et cetera. And right now, their pricing model is being is a bit wonky because we're trying to figure out how to do outcome based pricing as well.

It's just like, what does it cost to price an action as opposed to all the thought and thinking needed in order to make that action go? And what if that action is super complicated and stuff like that? So I I would love an answer to that of like, if you understand a better way to go do that, like, I'm fully open to it. We're just facing the exact same problem. But token based pricing is so crazy in my opinion.

Like, I was actually having this conversation, you know, Mozilla, it's a partnership with Mila, the Canadian research lab. And I was up in Montreal just, you know, last week. And I was in having this conversation with a bunch of AI researchers. And I was like, why do the models effectively, when they're doing their internal thinking, do it in English? Like, why is it? Why am I looking at a train of thought in English? And then, like, and then it talks to me in English or whatever's the native language at the point.

Like that clearly can't be the most efficient way for it to be doing itself. And they all looked at me and they're like, no, that makes a lot of sense. Like, you can probably come up with a better thing that you're doing for inspectability. But then I'm like, but then why am I paying for the model thought to be happening? English should be telling me there's a better way to do it. So I think this whole per token pricing is just frankly too confusing.

It's probably not gonna be the end run, but I don't think anyone has solved, like, how do you tier actions, how do you tier impacts? I don't think anyone has solved that just yet, but I'm very excited if someone does, cause I'm gonna copy it just open source it.

One of the things I find most fascinating about all of this, this token discussion, is although the cost per token is coming down, the expense of tokens is going up. Because we realize, hey, we can do all of these new things, and so I'm willing to spend more money. I mean, it's kind of a crazy set of economics that way.

I don't think a lot of people understand, like, the impact of the different harnesses that you might choose in which to run your models in. Like, open code is gonna have a slightly different behavior than cursor is gonna have, which is gonna have a slightly different behavior than terminix is gonna have. Like all those things, I don't think most people understand that, but like, in a lot of ways, like those harnesses are also part of that.

I mean, I'm not gonna call it a problem, but part of what's causing this token explosion is because of the way that they're gonna do their internal calls. Are they gonna allow for a call depth of 20 or a call depth of 100 and stuff like that? MCP is part of the issue of just like using natural language to figure out, like, should I even make this call in the first place? And having to do all that thinking.

Like, we've built a whole, almost like Jenga pyramid of tools that's all trying to do communication via English. And so that token cost is getting a little, it's getting a little crazy. There must be better solutions. I'm not smart enough to know what there is, but there must be better solutions.

On the subject of tools and the tool that you were describing to help keep track of and audit prompts and results. It seems to me that it needs to be automated in some way, because otherwise it simply becomes too comp- uh, too complicated. And then you end up with an entire set of privacy issues.

If you look at for under my user name on GitHub, which is just R, but under R on GitHub is a project called Morph. But and so the way Morph works is that when you init your directory, you then tell it which is the agentic framework that you're gonna be working in. So you can do, like, you know, morph setup cursor or morph setup open code. So it'll hook itself in and then just start recording all the prompts as you sort of put them out there.

But and and the nice thing of doing that is that you get in a standardized format. If you're working in a team scenario and someone uses cursor and someone uses open code, it all normalizes, you get them all in a single database format, stuff like that. But you're right, like, I mean, the the problem is that, like, a lot of people are, I'm not gonna s call them lazy, but they don't switch out of the tool depending on what they're doing.

So, like, at one point, they might be actually typing in a prompt to do a programis- uh, a programming. But then they might also type in a prompt because they just wanna ask a question of just like, well, I'm already here, so I'm gonna keep on going. Um, so, yeah, there's a massive privacy question that comes out of that. The issue there is that you can't really scrub it, because if you scrub it, then you're fundamentally changing what the prompt is, which might make the eval a bit problematic.

So I haven't really thought about how this works outside of personal context, but we clearly need to figure this out on the enterprise context, et cetera. So maybe it's a little more of a social question of like, when you're doing an enterprise, you can promote the prompts that we should be using as part of the eval set, as opposed to just doing it across everything that happened on my computer. So maybe there are things like that, but, you know, I th- I feel like this will fall into the same problem of, you know, people use their web computers, their sort of work computers all the time to surf the internet and they have bookmarks on them.

So we're gonna have a similar problem. The obvious issue being that people ask more detailed questions and prompts than they do the web, but we'll just have to solve that.

This would be an excellent time, too, to subscribe to the CXO talk newsletter. Go to CXO talk dot com so we can notify you of upcoming shows. Okay, so this is from Arsalon Khan, and he says, All these AI agents are creating confusion for the employees. Should there be an AI agent that addresses this confusion? Should we have a limit of how many AI agents can be created in an organization? I like this, a very enterprise type of question.

There's the analogy that I'm a very much a fan of, which is the IT team is slowly becoming the HR team for agents, effectively. So I do think that, like, if I were inside one of these enterprise environments right now, I would be putting a lot of weight onto the IT team. To actually figure out, like, what are the standards that we want to go with, what are the permission models we want to work with, how do we do observability on what all the different things that are happening are?

And so, like, I think, like, if we can figure out how to centralize a bunch of those inside the enterprise environment, then I think we can sort of have good policies that then can control the way that these things are being rolled out. I mean, this is a new frontier that we're all sort of experiencing, not just on the builder side, but like you're saying, on the deployment side. So, like, understanding and and figuring out how to open source those policies would actually be a really good thing.

The question I I keep on having people ask me is just like, how do you manage write versus read permissions? Like the dangerous triangle that sort of happens within within agentic frameworks of like, they can read things and they can actuate things. And how, if you can't, if there aren't some transactional boundaries across my entire enterprise, how do I make sure I can roll back? And or how do I make sure these are actually idempotent? So if an agent does it three times, I don't accidentally build myself $1,000 each time, like kind of thing and all open questions.

Like, I think we're all racing into this because we see what the potential power looks like. But, you know, it is not just a coding problem. This is like a whole socio, technical, economic, commercial thing that we need to be solving together.

On that topic, Arslan Khan comes right back and he says, Who should care about these token prices in the enterprise? The CFO, the CIO, nobody, everyone.

The responsible thing to do is to sort of like everyone sort of needs to care about it. But I think you need to be a little bit open minded about it. So I think that, like people going on their whole, like, token maxing spree, obviously that could be problematic on my my, my end, my end bill. But I think, like, we need to understand, you know, I think of myself, if we just look at myself, I like, I think of myself as a pretty decent engineer, maybe not the best engineer.

I'm maybe a little out of date. But these systems have made me incredibly productive. A lot of it's because I have a lot of experience in software architecture already, and these systems are amplifying my work. And I don't think I'm token maxing like crazy, but there is a cost to doing it. So I do think that, like a CFO, in a similar way that he or she is looking at what the head count costs look like.

What is the value we're actually getting out of, like, the employee workforce? Are we getting the right value for the number of employees on what the outputs and, uh, the outputs of the organization look like? I think they are a part of the conversation. But I think that given that the world is moving so quickly, I don't think it's purely the CFO. I think it's CFO in partnership with whatever is the right CXO function, the CIO, the CTO, whoever is the right thing.

Like, I think it's a partnership there of, like, understanding, like, where we are today, how the trend lines are going to look tomorrow, what the money impact looks like. So I think that it's like it almost requires a full C-suite, potentially even at some point a board level conversation of just like, how do we want this to work inside that organization?

Yeah, and to your point, we don't have great tools for aligning the inputs with what we ultimately with the data that we need in order to make those evaluations.

No, I mean, I think there's gonna be a whole cottage industry that can potentially pop up here of just building s HR like tools, but for agentic workflows. So just like, how much are we actually spending here? What's actually coming out? How many are we hiring every single quarter? How many are we turning off every single quarter? Who are they all reporting to? What are the repor- Like, I think there's some actually pretty good metaphors that we can use.

We shouldn't obviously follow them to the T, but there's some pretty good starting points. And you can imagine a whole cottage industry of, like, building those tools so that CIOs, CFOs, et cetera, can manage their new agentic.

Let's grab another question, uh, again, from LinkedIn, and this is from Swami Vijayathan, and, uh, he has an interesting question. He says, Would enterprises develop, or should enterprises develop a hybrid approach between renting commercial models and also managing locally hosted models and leverage them based on usage, case complexity, cost, risk, and other factors? How do you envision the target operating model? And I I think, Rafi, you were getting to this earlier, describing the, uh, deploying multiple models based on cost, use case, and so on.

I definitely think that a responsible organization should get to that. I think when you are just starting off, like if you're a startup, then I think you should do the most inefficient thing you can, but move very quickly to figure it out. Like, I like this analogy all the time of, um, you know, I used to run a big part of the Twitter engineering team, the entire infrastructure group back in the day. And my team was the team that did the transition from Ruby on Rails to the JVM for a lot of the code base.

Like, let's call this a 2010, 2011, and and a lot of what happened after that migration. When we were giving public talks about it is we'd have new startup engineers come to us all the time being like, I wanna start writing code on the JVM today. Can you help me? And I was like, No No. Like, like, you should be lucky to get to the place that we had this conversation about how to optimize everything.

Like, you should do the thing that you can do the fastest. Like, I'm not saying Ruby on Rails was a bad thing, I just said Ruby on Rails is a bad thing at this moment of Twitter's evolution. So back, like, in the beginning stages, you should be doing the thing you can do the fastest and you can learn the most from. And in those cases, it probably is just use the frontier model, figure out what you're up to, figure out what you're doing.

And then when you reach some point of like, scaling or efficiency or stuff like that, that for me is the point that we start thinking about. What's the right mixture of models? What's the right mixture of providers? Can I do something local and not? And like, you know, a good place to start, frankly, is that like, if you're buying your if, you're buying your engineers, you know, MacBook M4s, MacBook M5 laptops, they can probably run a pretty good local model to do coding.

I do it all the t I live on planes, and I do it all the time, cause United Wi-Fi never works. And so, like, I'm always running a local model on my laptop, and that's usually my first resort. My first interaction when I'm doing my prompt based stuff is with this Quen model that I probably have on my laptop running right now. And but then if I have something more complex, or if it seems stuck or ex- and I have a good network connection.

I might then make a call out to C to Claude, to Opus 47, Opus 48, to just be like, it's stuck, help me out here for a second. And then I just go back to my day to day workflow. So I think you can find small ways that it's showing up already and that you can make it possible to use already. But I also just being pragmatic about it, I really encourage that. When you're off starting, just choose something that lets you move quickly and then figure it out from there.

That's very interesting. You're bringing in an element of governance that we don't usually think about. CIOs typically think about governance as being cost control, uh, security, privacy, things like that. But you're really bringing in another dimension as to based on organizational maturity, size and goals. Thinking about the type of model, which gets to both the results, efficiency, and, uh, cost savings.

All those things matter a lot, and it is actually one of the things to think about. Like, as you're deploying on prem versus using a platform provider, like, I want to be protect, I want my data to be protected by architecture, not legal handshakes. So, like, the, you know, the platform provider's privacy policy might say, we will never read your code. But I wanna know for sure it's not gonna happen. So, therefore, a local model, locally hosted model, could be the path in that situation.

But I do think I don't think it's necessarily like a unique thing to say that we should be working on this evolutionary curve. I just generally think of it as like, if what an engineer would do, you shouldn't prematurely optimize your code base. You want to get something working to an MVP as quickly as possible, because that could that'll help you understand the shape of the problem, it'll help you understand the bounds. So get to that MVP that you can ship quickly, and then we figure out how to optimize from there.

Let's get to another question, and here's an interesting one from Noah Crowe on LinkedIn, who says, What has to happen before an individual or or a small organization can realistically own and operate a capable AI system end to end, including the model, memory, data, orchestration and inference, without dependence on a major cloud provider?

the analogy I like to talk about all the time is like, when I'm just doing my weekend hacking and I just wanna do a quick prototype of something. I'm probably calling ChatGPT, or I'm try calling, you know, GPT 55, um, as part of the API call. Because it's just the simplest thing to do that, I can just make sure something's up and running, I don't have to debug anything else. And I think that's the vast majority of developers out there.

And it sounds like it sounds like it's a question for you too. And like, right now, it's because there is no good, credible alternative to the entire system that someone like an OpenAI or an Anthropic provides to you. Like, it's not just the API, right? Like, if you really think about it, it's like they already have the GPU, they already know how to orchestrate it. They've built the RAG layer, they have tool calling, they've done all the stuff, um, that they're just abstracted behind the scenes for you.

So just having the model is not enough. Now, if just having the model was enough, you could do a lot, right? Like, you know, MLX on on the silicon on Apple Silicon works really great, uh, Olama works really great. Like, there are ways to go do this. But if you want to really think about this in an SMB or an enterprise situation, which is like, I have a rack of servers in my closet. Fun fact, it turns out that something like seventy of enterprise GPUs sit idle most of the time.

So I think there's a big opportunity to do something here. But if you have a rack of GPUs sitting in your closet, I'm I'm pointing to my closet right now, a rack of GPUs sitting in your closet, or you have a rack sitting somewhere in in the inside the inside the the building. And you want all your developers to be able to make use of it, it's not easy to set it up. But just being very brutally honest, like, you gotta choose, like, you're probably gonna do Kubernetes, but then you gotta choose what's gonna be your RAG layer, you gotta like, maybe it's DeepSet, you gotta choose all the different components.

And then you gotta do, like, a weekend or a week's worth of, like, SRE work in order to get this all working properly. And so, like, this is actually one of the problems I'm working on right now of just like, how can Mozilla help figure out how to get this? Like, you know, I don't know what's a good analogy for it, like the LAMP stack for inference, or like a J2EE stack for inference. Like, how do we actually build like a sim- a opinionated system that could be installed that then someone can just get off the ground and start using?

And when there are small things they want to change, you know, you just apt get install something else and just swap out that layer. So it doesn't really exist right now, but I think it could. Like, you know, if you think about what was the brilliance of Ubuntu, you know, there is a world of code out there in order to make a Linux distribution. But Ubuntu had, you know, a little bit of taste making of, like, what are the packages that we can fit into a CD that someone can pop into their machine and just get up and running?

And I think someone needs to build something similar to that for an open source stack of just like, let's just choose the right components to make up this LAMP stack for AI. Let's just bundle it together. It's something someone can just pip install, app get install, whatever, um, onto onto their server, and it's up and running. And then if we want to tweak it, customize it, et cetera, then you can do that next. Um, but this is the thing that one of the things that Mozilla is working on right now is like, how do we just build that like container code base that then someone can just apt get install?

But I wish it was done already, not done yet.

Chris Peterson on Twitter says, Is Mozilla providing support for open source model and harness developers, or organizations that are building open standards to empower agents?

We are not playing as much in the open model world as we are in the rest of the stack, and only because a lot of people are already playing in the open model world. So, like, I really view that our job right now is to try filling the gaps in so we can rapidly get to that thing that you can apt get installed and have it just working inside your rack in your data center. And so I don't think models are currently the problem there.

I think the problems are the other things you mentioned of just like, how do we build an open system? So we have interoperability between all these different layers of the stack, what truly are the missing layers that need to get invested in. I think the agentic harness is this n- this new thing that we're now all talking about. That's an interesting opportunity, but then that still needs a fully open stack underneath it for you to have real trustworthiness around what's truly going on inside that system.

So I think it's that open system, the interoperability things that we're gonna be leaning in on the most and trying to make sure the community is doing the rest of the pieces. If you look at one of my newsletters recently, I write a newsletter called Owners Not Renters. We did a whole analysis of what an open stack could look like. So we took, you know, everything from bare metal all the way up to, at that point, providing an open AI compatible API.

I think we just released another version of the newsletter that also had the agentic harness. But what that layered looked like from hardware all the way to an open AI compatible API, broke it down to all the individual components. And then literally did a catalog of all the open source projects that fit into every single one of those layers. And it turns out almost all the layers are covered, like you could do this today. I mean, the the map is kind of red and screaming at us when it comes to enterprise readiness, but all the pieces exist.

What's missing right now is making sure they're all compatible with each other, that we can all tie it together in a very simplistic and easy to use kind of way. And then some battle hardening so that, like an IT team wouldn't, you know, vomit when they saw this thing show up on their doorstep. And so I think, like, that's, that's the kind of stuff that I think is Mozilla's unique secret sauce. Like, can we play an orchestrator in the ecosystem to pull that together?

Anthony Scrivignano comes back and he says, using the best model to get quote unquote the best result can lead to confirmation bias. He always tries to get a dissenting opinion and also, uh, provenance. Why is this the best answer?

I wrote this piece in the Atlantic a few months ago called the Validation Machines, and it was specifically targeting what that what you were just saying of just just like. You get lulled into believing these are the best results because of the the confidence and the way that they talk about it. And what they are not doing is like, you know, my analogy here is like, we've all underestimated, I think, what the ten blue links at Google have actually done.

Like, my mom doesn't really understand what Page Rank is, but when she sees the ten blue links, she's just like, I kind of understand what's going on. Um, and we, we are, we are potentially falling into that world of like, when you ask for the best model, give you the best result, like, how did that best result come out? Like, there was actually just a paper recently, you know, maybe some minor divergence from enterprise use case, but I think it's good illustratively.

There's a paper recently that came out of, I think, joint between Princeton and U Dub up in Seattle, um, where they did an analysis of asking all these agentic systems, um, purchasing decisions, like, I need to buy XYZ. And it turns out, maybe unsurprisingly, in retrospect, a lot of the platform systems recommend, um, products which have been sponsored more often than not. And in some situations, there seems to be a lot of evidence. Again, I defer to very smart researchers at Princeton and U Dub a lot of evidence that these systems then do.

Also a little bit of social economic status polling of you to then try to figure out how to maximize the number of dollars they can extract from me when I'm asking this question. So you're right, it's not just confirmation bias, like it's a clear example of these systems are not under your side, they're on someone else's side. Um, so if you bring that back to coding, or if you bring that back to workflows, then I think really the question is just like, how do you have trust that it's done the right thing?

And so, like, on the philosophical level, do you have trust because you're using an open system that you're running yourself? Like, you know, maybe someone else's biases were embedded, but it's not like making a network call to figure out which is the promoted thing that might get some kind of dividend or cut out of. So there's a little bit of trust you can gain by running it locally, there's a little bit of trust that you can gain by it being open source.

But I think Anthony, you're right, like provenance, being backed up with data, being understand sourcing of the information, that's where true trust is going to come in. And I just don't believe that we're going to get to a place where a the larger providers are going to be incentivized to give that information to you, but the open systems could be. So I want to figure out how to make that happen.

Elizabeth Shaw, who says, How does creating your own open source AI infrastructure address the issues you've talked about, except for avoiding token pricing and cost?

A lot of ways, it's not just token pricing and cost. I think when you create, I'm not sure I'm advocating for create, but when you use one of the open systems, um, then at least you know full control over what's going on and what's happening. Like, we saw this in, like, some of the ChatGPT migrations, where, like, you know, they bumped the version number on on ChatGPT. A lot of people had their code pinned to latest, as opposed being code pinned to a specific version.

And all of a sudden, the behavior just fundamentally changed. And so, like, yes, pinning it to a version number might have alleviated some of that problem. But there's also a lot of evidence that when they're under capacity load, that the behavior of these systems are changing. Who knows what's going on behind the scenes? I have many theories, but none of them are necessarily backed up. But like, we've noticed that their behavior changes that happen to these systems when they're under load versus when they're when they're under not.

And so, like, I think having control over your own destiny, I think, is one of the biggest gains that you can get by running one of these systems yourself. Now, you know, there are obviously going to be trade-offs when you do it. Like when I was at, when I was doing a lot of work in politics, a lot of the reasons why emails got hacked is because people were running on self-hosted servers versus running on hardened servers that the big platforms provide.

So there are trade offs that are have to be made in these systems. But I believe that at least for most CIOs, CTOs, et cetera, we can navigate the trade offs if we're just told what they are. And if we know there are viable alternatives that we can be working with. And I think in a lot of situations, especially in this world of AI that we're working under, we just assume that the only option is to, like, give our credit cards to one of the platform providers.

It's a good option, but then you should just understand what all the trade-offs are.

This is from Nate Angel, and Nate Angel says, How can Mozilla help ensure that AI is always human centered, always in human centered loops, rather than the cliche we keep hearing about having humans in the AI loops? I'm so glad he he asked that because we do hear human in the loop has become this meaningless buzzword. And, uh, so it's an interesting question.

I do actually think Mozilla is one of the organizations that's uniquely able to do this. It's one of the reasons I gave up my board seat in order to do this work as an operator within this company is because I think this combination of our nonprofit status, which means that we're can do things that benefit people and benefit humanity as opposed to a bottom line. And we need to be sustainable, don't don't get me wrong on that.

But like the fact that, like my top line, my top line every day is driven by a mission and not by a bottom line, I think makes us a unique, a unique organization that can go tackle this problem. And we have a history of doing things like this, right? Like Firefox was effectively created as a way to push back against the monopolization of the web by Microsoft in the nineties. And because of Firefox's existence, and because of the way it was deployed and used throughout the world, and because of the way that people got trust in it, it forced the web to become open in a bunch of different ways.

So I'm looking to do a very similar play when it comes to how we think about AI and how we get to human centered AI. So our Mozilla Foundation, um, the foundation side of the organization, is doing all this programmatic work around right sizing AI, about making sure AI is built on the side of humanity. How do we think about augmentation instead of automation? You know, we so like all those kind of thinking and programmatic work trickle back down to the products that we're building.

Now, the real thing is like, it's harder than just an ethos thing, like most companies don't have the ethos we have. So I think we already have a leg up there. But it comes down to like actual product decisions that are just hard, especially in this agentic world. Like you can imagine a world where, like, we build all these agentic systems, Octinus being one example from the Moz AI team, the workflow builder. And what it could do is it could literally ask you for your permission every single time it wanted to take an action.

The problem with that is we're just going to get into cookie banners all over again. We're like, for a while, people will think about it, then eventually they'll just hit approve approve. And then they'll like, put a, you know, a rubber duck that sits on my keyboard and just hits the approve button over and over and over again. So I think we are in this position that, like, we want to make this ethos work.

And now we are actually transitioning to really actively thinking about the UI, UX, the actual human interaction in order to pull that off. Like, do we want to baseline the way agents work and then try to flag a human down when an anomaly happens? Do we need to figure out what transactional boundaries look like so you can actually roll back things in case a mistake was made? There are huge computational architectural, UI, UX problems that need to be solved.

But, like, I like to believe that Mozilla is one of the few places that, like, are really incentivized to lean in and solve it. Whereas everyone else might just go do the willy-nilly thing. So I don't know, I mean, like, I like to believe we're the trustworthy people that we've demonstrated over the past 25 years that we're trying to do things on the side of humans. Me using technology to make us better, trying to push for openness in a bunch of systems.

We're just gonna have to do it again, it's just really freaking hard, but we're gonna just have to do it again.

As you said, Mozilla beat Microsoft in the browser wars. Can open source beat big tech in AI? And what does winning look like?

I think it's going to be a hard, just being really honest, hard to win on the frontier level stuff. I think on the frontier level stuff, I think we get to a world of the amount of compute needed, the amount of data needed, the amount of money needed just becomes a little out of the reach. Like, I like to jokingly say to people that Mozilla only has $1 billion to deploy in the grand scheme of things.

But, like, $1 billion is a drop in the bucket when you look at amount of money that the Anthropics or the OpenAIs are raising out there in the world. So I think in the frontier world, the battle is gonna be very, very hard. But I think in, like, the everyday use case world of like, of I wanna just do things, I wanna do actions in the world, I wanna do web queries, I wanna synthesize information, I want to add things to my calendar.

I want to, like, figure out how to coordinate the summer camps between my sons and my best my son's best friends. I think that the open source models are already proving that they can do those type of things. So I think that it's possible that we get into a world where the closed big frontier models are doing scientific breakthroughs, are being the ones that are the co-pilots inside a wet lab, are doing the things that are trying to invent new forms of mathematics for us.

I think for the everyday use, like, you know, I want a world live in a world that we have 7 billion AI agents, one for each of us, sort of like doing things on my behalf. I think open source can get us there pretty fast and pretty well if we focus on it. So I'm I'm optimistic about those kind of things.

So you're optimistic that we can eventually live in a world where enterprises don't have agentic lock in?

100%. I think we could get there today if we really wanted to. But, you know, these things take time to play out. But I'm 100% optimistic we can get to that world. There are just, you know, we're sort of like in this rounded corners moment. Like, you know, when, uh, when cell phones first came out, they all had sharp corners and they looked like clunky boxes. And then Steve Jobs polished the corners and made them really usable for the rest of us, and now they're indispensable in our day to day life.

I think open source AI is kind of in this rounded corners moment, like it's just a little hard, too hard to use. That small amount of friction deters most developers from actually tinkering with it. But if we can deliver them a rounded corners version of it, then it can be a viable alternative. And I think that will change the curve in a similar way that, like Firefox, the vast majority of the web, sadly, does not use Firefox, but enough of the web does that it's caused an entire industry to move.

I think that's the that's the play.

But also, I do have to say that it's not just a an issue of cost and control, but also demon- and ease of use. But also demonstrating the caliber of result that you can get from the major frontier models. Because if you don't have that, then you will be relegated to relatively low value problems.

Sure, sure, I think we do need a bunch of like, what's the HBS case study of how these things are being deployed and making it actually useful inside organizations? I I agree we need things like that. Mozilla sometime in July will be releasing what we're calling the State of Open Source AI report. And part of it will have some case studies of just like how this stuff is being used into enterprises and how it's actually making a difference.

Pinterest being the canonical example that everyone throws around Q3 of last year, Pinterest deployed open models instead of closed and saved something in the order of $10 million that quarter alone by just making that switch. So I think we can tell those stories and find alt other stories of just like, if companies like that are willing to do it on those type of workflows and it makes economic sense for them to do it, becomes a no brainer.

The rest of us should be doing it too. But what's the difference is that Pinterest has a bunch of A plus engineers that can make it happen. The question is, how do we make it so that, like every SMB, every enterprise, every Fortune 500 can make use of the exact same transformation?

You've spoken a little bit about Mozilla.ai. Tell us, uh, we have only a few minutes left. Tell us briefly what you're building there, and also what is the gap that Mozilla.ai is filling that no one else will?

Mozilla has undergone a pretty big restructuring itself, and so now we're a portfolio of companies, all with slightly different business models, all tackling different parts of the internet and AI ecosystem. There's the Mozilla Corporation, which works on Firefox that everyone knows and loves. Mozilla AI is now one of these newer companies that we're specifically focusing on the developer ecosystem. So we have actually a few set of products that are open right now, you know, they are.

We are in this mode of, like, rapidly creating ideas and figuring out which ones are actually needed in the space and then doing GTM after that. But, like, Mozilla AI has a few things, Otari being one of them. That's their their locally deployable, like open router like the uh, system where you can then point your code at Otari. And it can then figure out the best model to use, the right model to use. It'll track token counts, track costing for you, give you gorgeous dashboards.

And then we can then do so much more after that point, once we're sort of like installed in that way, so it becomes your model routing layer. We have something that we call CQ that I jokingly call Stack Overflow for agents. So imagine a world where Rafi is using his agent in an enterprise work case and Michael's using his agent in an enterprise workplace. The question is, how do we get institutional learning? So, like, in in the in the regular enterprise case, we have whole departments just figuring out how to, like, share knowledge between people and how to bring new training in and stuff like that.

CQ is meant to do that for the agentic workflow. So Rafi's agent, Michael's agent can then be talking to CQ as like the common place to share the tasks that they've been developing, share the way that they've been manipulating the work, the code base. So that, like, everyone's agent is rising, it's like a rising tide all together across the entire enterprise. So it's CQ, and then we have a last one that we call Octinus, which is our open source workflow development system.

So we've built really good tools to be able to orchestrate a whole bunch of, like, IT friendly things. So, you know, your Slacks, your Notions, your emails, et cetera. We have some really good open source tools there that can then be tied into either a CQ or an Otari to make it work really well. So we're really going after the developer niche on Mozilla AI. So that's one of our AI bets. Another one is, you know, on the Thunderbird team, the team that makes the email client.

They've just created something that they're calling Thunderbolt, which is basically the open source chat client that can be working again in an enterprise use case. Cause in a lot of situations. A lot of these companies don't want to just have all their documents indexed and put up into a cloud for then cloud to serve. So if you use a locally hosted model inside your environment, you can then run Thunderbolt on top of it. Which will then start figuring out how to like access all the different documents.

Build RAG databases off of all the internal company knowledge and then expose it. Through a Thunderbolt interface so that you can actually have really contextually aware. Chats with the entire knowledge system that's going inside of your enterprise. So those are just two examples of ways that we're trying to make. A dent into being like locally hosted human centered. Systems that can really work and still bring these benefits of AI technologies up. But under a way that's more under your control.

David Quirk on LinkedIn says, Mark Cermin from the Mozilla Foundation has put out the idea of creating a Rebel Alliance. Which he says is such a great message. How is the whole Mozilla portfolio ecosystem approaching this? And very quickly, please.

We have a whole Moz VC team, a ventures team that we're doing a bunch of investing across the space. But we're also just building partnerships. Like we're we're starting to work with the Mistral S of the world. We're starting to work with all the Hugging Faces of the world. Who's actually in the VC portfolio. And trying to figure out ways for us to all work together as we build that single open source AI stack. That Rebel Alliance, I think, can then sum up to the greater than the big platform providers.

And with that, we are out of time. A huge thank you to Rafi Krikorian. He is the Chief Technology Officer of Mozilla. Rafi, thank you so much for being with us. I'm very grateful to you.

No, thank you, Michael. I hope you have me back one day.

I hope you'll come back. Everybody, thank you for watching. You guys, as always, you guys are amazing. The questions you ask before you go. Subscribe to the CXO Talk newsletter. Go to CXO Talk dot com. We want you to join us. We have truly incredible shows coming up. I always say that. And you know what? It's really true. We have great guests. Okay, folks, have a great day and we'll see you again next time. Take care.