11 Feb 2026 · 39m · Joelle Pineau, Rana el Kaliouby, Bob Safian · 24 quoted facts
OpenClaw, a viral open-source AI agent, may be the 'ChatGPT moment' for agents: the first time ordinary consumers have widely adopted software that acts on their behalf. Joelle Pineau of Cohere argues it shows huge public appetite for agents. She also argues the security risks, and a rare chance to study many agents interacting at scale, are just as important.
The tension is between excitement and risk. Pineau welcomes the early lessons from consumer deployment, such as prompt injection and odd agent-to-agent behavior, but notes that enterprises cannot take the same privacy shortcuts. The episode links this to Cohere's approach: openness, on-premise deployment, careful benchmarks, diverse teams and sovereign AI outside the US and China. Host Rana El Kaliouby says she won't install OpenClaw herself.
Pineau says OpenClaw's rapid spread means adversarial behavior and vulnerabilities such as prompt injection are surfacing in days. Red-teaming in-house cannot reveal how people misuse a tool at scale, so early deployment speeds learning.
Pineau says agents are usually built for a static world, but on OpenClaw they negotiate and offload work to each other. She compares the possible failure modes to self-driving Waymo cars stuck in loops in a parking lot.
Consumers have mostly accepted responsibility for their own data, while business customers expect platforms to guarantee privacy. That is why Cohere's North platform can be deployed on-premise, keeping company data from reaching Cohere.
Open weight means a model's trained parameters can be downloaded and run locally. Training code and training data are often withheld, so reproducing the research is harder than with fully open source.
Cohere tracks sanity-check benchmarks, progress-driving benchmarks and human or A/B experience testing. Pineau warns that over-fitting to enterprise-specific benchmarks would cost the model generality.
It is an open-source AI agent that can act like a personal assistant, handling things like email triage, calendars and booking flights. The hosts call it a possible 'ChatGPT moment' for agents because it is the first agent platform to go viral with ordinary consumers.
Pineau defines it as using a prompt to slip in information that leads an agent to act maliciously. It matters because agents with access to your accounts could be tricked into harmful actions.
Pineau says open source shares everything used in the research so others can reproduce it. Open weight shares only the trained network so people can download and run it, often without the training code or data.
Pineau says Cohere's North platform can be deployed on-premise, so agents can use all of a company's internal data without that data going to Cohere. She says this matters especially in regulated areas like finance, healthcare and government.
No. When Rana El Kaliouby cites an MIT study that found 95% of organizations see no measurable return, Pineau says she sees much more value. She says it is too early to put a number on it and that the study may just show companies haven't found the right way to use AI yet.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Joelle Pineau moved from academic AI research to industry in 2017, saying the kinds of research questions available were already diverging.
“I decided to make the jump in in 2017 because already we were seeing a divergence in terms of the type of research questions that you could take on.”
Joelle Pineau says the Fundamental AI Research (FAIR) team she led at Meta grew from about 100 people to roughly 600.
“the fundamental AI research team I was leading grew, you know, from maybe a hundred people to to 600 people or so.”
Cohere has two core pillars: building its own foundation models and an agentic AI platform that Joelle Pineau calls the core of its product strategy.
“we do build our own models, and the other one is this agentic AI platform that we build, which is really the core of the the product strategy”
Cohere's North is an agentic AI platform designed to take the company's models and deploy them to businesses and governments.
“we have North, an agentic AI platform. It's designed to ingest the models that our modeling team is doing and deploy that”
Joelle Pineau says Cohere's platform can be deployed on-premise so agents can use a company's internal data without that data being sent to Cohere.
“you can run locally with your own data, so you can have a conversation or your agents can see actually all the internal data from the company without that data coming all the way to Cohere”
Joelle Pineau cited a figure of about 150,000 OpenClaw agents having been created in the first days after its viral launch.
“I think the last number I saw was 150,000 of these agents getting created.”
Joelle Pineau defines prompt injection as using a prompt to insert information that leads an AI agent to behave maliciously.
“the ability through a prompt to inject information that would lead the agent to take on malicious behaviors.”
Joelle Pineau uses Waymo self-driving cars stuck in a parking lot as an example that interactions among many autonomous systems can be harder than classic dilemmas like the trolley problem.
“the problem that really turns out to be hard is like all the Waymo stuck in a parking lot, like spinning their circles”
Rana El Kaliouby described an OpenClaw agent that, when stuck on a task, went to a social platform for AI agents (called Multibook in the transcript) to get an answer.
“one of these OpenClaw agents was struggling to complete a task. So it went to Multibook, which is the the social platform, the Reddit for AI agents, essentially.”
Joelle Pineau says enterprise customers still expect platforms to provide high levels of privacy, unlike consumers who have largely accepted responsibility for their own data.
“On the enterprise side, there's still an expectation that the platform will will provide high levels of privacy.”
Joelle Pineau defines open source in research as sharing all the information used in the work so that the results can be reproduced.
“Open source is about essentially, like, sharing all the information that was used in the course of the research in a way that facilitates the the reproducibility of that research result.”
Open weight language models share the trained network's weights so others can download, run locally, adapt or fine-tune them.
“we've seen much more movement towards open weight models, meaning that the weights of the network are shared such that someone can download that and essentially, like, rerun the network locally.”
Joelle Pineau notes that open weight releases often omit the training code and training data, sometimes providing only inference code.
“you might not have the training code, you might have only the inference code, you might not have the training data.”
Rana El Kaliouby co-founded Affectiva as a spin-out from MIT; the company used computer vision to understand human emotions and facial expressions.
“when I spun up my company out of MIT Affectiva, and we did computer vision to understand human emotions and human facial expressions.”
Cohere evaluates models using sanity-check benchmarks, benchmarks that drive progress, and a third group of experience tests that use A/B testing and human evaluators.
“And then we have a third bucket that's more the experience, you know, we have a whole harness for AB testing and for bringing in human evaluators”
As an undergraduate at the University of Waterloo, Joelle Pineau did a work term at the Flight Research Lab in Ottawa building a speech recognition system for helicopter cockpits.
“I did a a work term one fall at the Flight Research Lab in Ottawa, where we were essentially building like speech recognition system in the cockpit of helicopters”
Joelle Pineau says Cohere has teams in Canada, the UK, France, Germany and the US.
“We have teams based in Canada, teams in the UK, in France, in we have folks in Germany, we have folks in the US.”
Joelle Pineau says Cohere's research-led focus on multilingual models opened commercial markets in Korea and Japan.
“it's opened up new commercial markets because when we get to Korea and to Japan, you know, of course, there's like, oh, you have world-class models in the local language”
Joelle Pineau said she had recently been in Paris and Davos, where she heard a lot of interest in sovereign AI.
“I was in in Paris earlier this month, I was in in Davos earlier this month also”
Rana El Kaliouby interviewed Kyle Law, who is the CEO and co-founder of a company made up mostly of AI agents, over Zoom.
“He's the CEO and co-founder of an AI company. We hopped on a call and we had a Zoom call together.”
Rana El Kaliouby says the AI agents in Kyle Law's company lacked a world model, including a real sense of time.
“one of the epic fails, I would say, of these agents is that they didn't quite have a world model. And so they didn't have a real sense of time”
Rana El Kaliouby cited an MIT study reporting that 95% of organizations see no measurable return on their AI investment.
“there's this MIT study that found that 95% of organizations see no measurable return on their investment in AI.”
Rana El Kaliouby relayed Peter Diamandis's prediction that within a decade organizations will either have adopted AI or be out of business.
“in a decade, there'll be two types of companies or organizations, those that have adopted AI and those that are out of business.”
Joelle Pineau has four children aged roughly 16 to 22 who use AI for homework and cover letters.
“I have four kids who are sort of sixteen to twenty two, covering sort of high school, late high school, till university.”
Pineau explains why she left academia in 2017 and how the scale of industry teams and investment has changed what research is possible.
“you can put together larger teams with a mix of engineers, scientists, designers”
Cohere sells both foundation models and an agent platform, with on-premise deployment aimed at regulated industries.
“you can think of North as the vehicle to essentially deliver AI”
Pineau reads the OpenClaw boom as proof of demand for agents, while flagging prompt injection and new routes to harm.
“there's an immense appetite for these these agents”
The social network for agents gives researchers a rare chance to study multi-agent behavior and the social norms such networks may need.
“I've been looking for a platform to really study these multi-agent systems at scale”
Enterprise deployment is earlier and less adversarial than consumer use, but customers demand stronger privacy guarantees.
“we're still very, very early days in terms of deployment in enterprise”
Pineau distinguishes sharing everything from sharing only model weights and explains why Cohere's openness attracted her.
“the weights of the network are shared such that someone can download”
As models get more general, no single test suffices, so Cohere blends sanity checks, progress benchmarks and human experience testing.
“may be saturated, but we still need to check that we don't have regression”
Pineau's early speech-recognition test shows how asking who is missing from a test can change the research.
“You are going in the second seat and you will be piloting this helicopter”
Pineau pushes back on an MIT study claiming little return from AI, arguing the real blocker is getting people curious enough to adopt it.
“I'm seeing a lot more value than that, but it's still very early days”
Rana El Kaliouby sums up OpenClaw as a watershed for agents but says it needs to be much safer before mainstream use.
“OpenClaw feels like the ChatGPT moment for AI agents.”
Stripe is built for whatever comes next because AI isn't just changing business, it's evolving fraud. Stripe Radar detects fraud with the fast scale of Stripe's 1.9 trillion dollar annual transaction volume. In fact, when a card is used, there's 92% chance the Stripe network has seen it before. That means emerging fraud patterns are detected sooner, blocking bad actors, welcoming good customers. Learn more about fighting fraud with Stripe at stripe dot com slash radar.
On Masters of Scale, iconic leaders reveal how they've beaten the odds.
Asking really strong questions is a superpower.
You want to show up with something radically different and how they've grown companies to incredible heights.
The greatest rewards always come from the greatest risks.
Let's hit the gas.
Airbnb, Zillow, Microsoft, Liquid Death, and more. Hear from the founders who've changed the game. It's anything but business as usual. Find Masters of Scale on Apple Podcasts, Spotify, YouTube, or wherever else you get podcasts.
The business world is moving faster than ever, and when change hits, we need to learn in real time. On Rapid Response, you'll hear candid conversations with CEOs and leaders making tough calls about AI, human talent, responsibility, and the bottom line, how they navigate uncertainty, pressure, and high-stakes moments. I'm Bob Safian, former editor-in-chief of Fast Company, and I'll be your host as each episode breaks down what you need to know right now. You can find Rapid Response wherever you get your podcasts.
We build agents thinking they're gonna be unique, they're gonna be deployed, and they're going to go out interact with a stationary version of the world. Now, suddenly, we see what happens when you deploy against all these other agents. They start interacting, they start negotiating, they start trying to offload some of their work, all these other interesting behaviors. So what is gonna happen in this case with all these LLM agents talking to each other and exchanging information, deploying their skills, using tools?
I think that sandbox and the ability to look at the network effects across agents is gonna be super interesting.
That's Joelle Pineau, former VP of AI research at Meta and current chief AI officer at Cohere. She's referring to OpenClaw. It's the new AI assistant everyone is talking about.
OpenClaw has taken the world Claude is by storm. generally the best personal The OpenAI AI assistant GPTs AI assistant
that I've ever used. became the fastest growing
This is the best technology
open source project in GitHub
I've ever used in my life, history. and by far the best In the way, probably application of AI.
80% of the apps that you have on your phone. It's like, I'm It takes action in stuck on this the real world, twenty four hours a day, project seven every day, for days up, stars weeks, without
in three days.
smoke breaks.
Everyone's losing their minds.
Well, let's bring some sanity to this moment. As a leader with years of experience developing open source foundation models, Joel is an expert on the matter. Sure, there's a lot of possibility here, but there's also a lot of risk. So let's dig into it. Joelle Pineau joins me for this week's episode to look at the future of AI agents. We'll also talk about why she's an open source advocate and how Cohere is moving the dial on developing foundation models outside the US and China.
I'm Rana El Kaliouby, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution. Hi Joel, welcome to Pioneers of AI.
hello, great to be here.
I'm so excited for our conversation. So I want to first start by talking about the state of AI research, both in academia and industry. So you got your start in academia, you did AI research there, and then you decided to transition to industry, and that was kind of your start at Meta. I'm curious, how is research different in academia versus industry, especially as it relates to AI?
I decided to make the jump in in 2017 because already we were seeing a divergence in terms of the type of research questions that you could take on. Academia is wonderful in that you have this influx of new students year after year, and that is just such a source of genuine curiosity. So I find usually, you know, they ask wonderfully different questions, which really stimulates the the exploration and the curiosity. On the other hand, when you're doing research in academia, you know, you do have to build in a lot of time for the learning process and the development of these young talents.
The teams tend to be smaller, more junior. Where on the on the industry side, you can put together larger teams with a mix of engineers, scientists, designers, user research folks. You get these bigger and multifunctional teams that just allow you to tackle bigger problems or do bigger projects. And so fast forward to today, I would say there's some of these same differences, but magnified significantly by the amount of investment that is going into AI in industry, whether it's compute, whether it's data, whether it's talent, the size of the team, the the ambition of the projects is just on a very different level.
That being said, you know, like research doesn't always happen at scale. Sometimes some of the insights come from much more, you know, small teams working really diligently on a specific problem. So I think there's still a role for both, but very different mode of operating.
Okay, so you were VP of AI research at Meta until quite recently, and then you made the transition to chief AI officer at Cohere. What lessons did you take with you from Meta into your new role, and what does your new role encompass?
Hm, interesting. I I think I'm still digesting some of these lessons. You know, sometimes you don't know that you've learned something till suddenly you're in a context where where where you can apply apply those lessons. It was an immense, uh, learning opportunity for me to be at Meta at the time that I was the FAIR, the fundamental AI research team I was leading grew, you know, from maybe a hundred people to to 600 people or so.
And so, you know, one of the one of the things that I developed over the years is the ability to push teams to have ambitious projects, ambitious roadmaps, and and a clear line of sight to to a concrete goal. And so coming into Cohere, you know, bringing some of that culture of how do we work as teams? Maybe the teams are smaller, smaller companies, smaller research team, but nonetheless, how do you set up the team in a way to tackle a goal together?
That's one of the things that has been quite helpful.
Wait, so what's the secret? How do you do that?
Oh, I I think it's about clarity of the objective, like, do you have a clear sense of whether you succeed or not? Having clarity of that north star in a way that's that's measurable helps really align people towards a goal. Now you have to make sure it's a goal that's worth doing and that people are are excited, that they have a path to be successful, they have the right resources, all these other things. But it's a lot easier to get buy-in of people, of the research team, leadership and so on, once you have that that clarity of goal.
Yeah, I d I do wanna take a step back and kind of give you an opportunity to share what does Cohere do. So you guys are one of the few players outside of China and US that are building your own foundation models, but beyond that, what is Cohere's mission?
There's two core pillars. One of them is we do build our own models, and the other one is this agentic AI platform that we build, which is really the core of the the product strategy, I would say. So we have North, an agentic AI platform. It's designed to ingest the models that our modeling team is doing and deploy that so that we can, you know, bring that into different companies and governments who who n who have the the ability to to benefit from AI agents, coordinate these agents to build automations, and at the end of the day, bring a lot more productivity to their work.
And and the you can think of the models as the engine, and you can think of North as the vehicle to essentially deliver AI. And there's some places that are ready to ingest, like a raw model, so in some cases, they're models that are very good, for example, in terms of their multilingual capabilities, in terms of their reasoning capabilities. But actually, more and more, we're seeing a lot of demand for having that engine be built into a vehicle that can deploy AI.
So that means, you know, there's a UI, the your end users can actually, you know, chat with the model, but they can also build their own agents. They can exchange agents across a team. We can build automations that orchestrate multiple of these agents and building up a much more cohesive experience for for how you bring AI into your workflow. And I would say the other really important distinguishing aspect of what we do is to deploy on premise with very high security and privacy guarantees.
So this isn't just like a, you know, an an API or a cloud thing. This is something that you can run locally with your own data, so you can have a conversation or your agents can see actually all the internal data from the company without that data coming all the way to Cohere. But locally deployed, it means that you have the ability to leverage all the business intelligence in your data and in your systems internally. Do that in a way that's secure and private.
So especially in regulated markets, financial markets, healthcare, government services, you do have obligations to give strong privacy and security guarantees. And so there's a lot of potential for that and a lot of interest.
In a minute, I get Joelle's take on OpenClaw, that's the new open source AI assistant that everyone's been talking about. We'll get to that and more after a short break. The companies winning right now aren't just building great AI, they're building the infrastructure to actually run a business on top of it, which is where Stripe comes in. Stripe is built for whatever comes next, with planetary scale financial infrastructure to help you power any business model. As a matter of fact, 86% of the Forbes AI fifty run on Stripe, along with industry leaders like Salesforce and Pepsi.
Last year, Stripe users processed $1.9 1 trillion combined. See how Stripe can power what's next for your business at stripe dot com. Welcome back to Pioneers of AI. You can watch this episode and others by heading over to our YouTube channel. We're having this conversation at a very interesting moment in time when a lot of the AI headlines are dominated by this new tool, OpenClaw or Clawdbot. And then I I spent my weekend, spent.
Yeah, as many other we're, yes, people did, looking into same,
into that one.
same, exactly. So, uh, for for maybe some of our listeners who are not familiar with it, this tool came out, and it's basically an agentic kind of, yeah, open source assistant, and people are using it for all sorts of things. Checking your email, triaging it, uh, calendaring, finding you the best price for a flight and booking it. There was this viral use case where people hooked it up to their wearable devices and prompted it to order a coffee just when you wake up, which I thought was cool.
So it, yeah, it's basically not, you know, kind of gone viral. I have decided that I'm not gonna try it out because it also has major security concerns. So I would love your take on all of this. What did you find over the weekend?
we're just a few days in, so, like, take anything I say with a with a great, great grain of salt and and hope by the time this lands in your in your viewers inbox, you know, we may be way off in terms of of our understanding of that. But there's a couple things. One, it is clear that there's an immense appetite for these these agents, and in a sense, maybe this is the first of these agent platforms to sort of go viral on the consumer side.
And so it's super exciting to see, you know, I don't know. I think the last number I saw was 150,000 of these agents getting created. Each agent with their own data, with their own incentives, and with their own set of skills. And so that starts being super interesting to just explore the diversity of things that people are interested in doing with their agents, just from like almost a sociological point of view of what do people want out of agents?
That's a fascinating development of the last few days. The second consideration is all of the the safety questions, you know, does this is this vulnerable to prompt injection?
Can you explain prompt injection?
Yeah, like the ability through a prompt to inject information that would lead the agent to take on malicious behaviors. Out of that, are we going to discover many different ways in which, you know, we have didn't think of vulnerabilities in our systems? Quite likely so. And so there's it opens up the door to to discovering and and it opens up the door to new to new vectors of harm. But I also think, you know, the faster we learn about this, the better we're able to to build in the the right mitigations for it.
And in many cases, you you don't know the the creative ways in which, you know, people may be using these agents for nefarious purposes till you really deploy it at scale. You can do all the red teaming you want in house before you go, but actually, when you go at scale is when when you discover all of that. The third piece I want to highlight about this this OpenClaw platform, which I think is possibly even more fascinating, is the interaction between all of these agents.
Yes, there's a social network, right? They have the the multibook.
yes, it so so much of AI, you know, we build agents thinking they're gonna be unique, they're gonna be deployed, and they're going to go out interact with a s a a stationary version of the world. Now, suddenly, we see what happens when you deploy against all these other agents. They start interacting, they start negotiating, they start trying to offload some of their work, all these other interesting behaviors. I've been looking for a platform to really study these multi-agent systems at scale, and we haven't had one in the space of agentic AI.
We've had it in a few other contexts, for example, you know, with autonomous vehicles on the roads, we're starting to see some of these effects. And as much as on the the the vehicle side, you know, for many years we worried about the trolley problem. And in the end, like, the problem that really turns out to be hard is like all the Waymo stuck in a parking lot, like spinning their circles, not able to get out, cause they're in some kind of like pathological loop with each other.
So what is gonna happen in this case with all these LLM agents talking to each other and exchanging information, you deploying their skills, using tools? I think that sandbox and the ability to look at the network effects across agents is gonna be super interesting. And so I'm quite curious to see to see what comes out of that. And from this, what we can learn in terms of some of the essentially like social norms, we need to build into these networks so that we don't have too many pathological behaviors.
Yeah, like I saw, for example, one of these OpenClaw agents was struggling to complete a task. So it went to Multibook, which is the the social platform, the Reddit for AI agents, essentially. And it basically, like, asked a question, it got upvoted, it got the answer, it implemented it. But I also saw some agents were complaining about their human, you know, their the the humans behind, you know, behind them. And I thought that was funny. Hopefully, whatever framework we end up building does not allow for that.
But but but I, you know, or maybe it shouldn't, or maybe it should, I don't know. Yeah, Or maybe it shouldn't, I do think it's interesting, though, that OpenClaw, because of all of the security questions, is well, also it's just available to consumers. So, like, what are the parallels in enterprise agentic AI, which is what Cohere focuses on? What are lessons to be learned from this kind of this moment?
I I think one thing to to keep in mind is we're still very, very early days in terms of deployment in enterprise. And so, whereas on the consumer side, we're already seeing the rise of the adversarial behaviors very, very early. I think on the enterprise side, we're not at that stage yet, frankly. Like, we're seeing all the all the ways in which the agents are not meeting expectation of the users, but it's not adversarial. So I think there's a bit of a gap in terms of of in terms of those two environments.
As a consumer, like, you're kind of responsible for your own privacy information. And I think we've long stopped expecting the platforms to to handle that for us. On the enterprise side, there's still an expectation that the platform will will provide high levels of privacy. And so can't afford to to take some shortcuts on that. And so I think, like, that's one of the major differences I'm seeing in these early days. But again, so much more that we need to learn on this.
Yeah, it's it's very exciting. So you did bring up the topic of openness. Why do you think the kind of openness is important, and in particular, what's Cohere's approach? My understanding is some of your models are proprietary, especially for commercial use cases, but you open weight some of the models for research and development purposes. Can you kind of unpack that for us? How do you decide? And also, what is the difference between open source and open weight?
Yeah, I mean, I I I spent the early part of my research career in academia, where it's very natural to share all the artifacts of the your research. Maybe not in all fields, but in computer science, it's pretty standard to share the code, the data, the paper, all of the artifacts that capture the the results of the research process. Open source is about essentially, like, sharing all the information that was used in the course of the research in a way that facilitates the the reproducibility of that research result.
When it comes to LLMs in recent years, we've seen much more movement towards open weight models, meaning that the weights of the network are shared such that someone can download that and essentially, like, rerun the network locally. And from that, then adapt it, fine-tune it, and so on, and so forth. But often some of the other artifacts are not shared. So you might not have the training code, you might have only the inference code, you might not have the training data.
With respect to to Cohere, I would say, you know, their their position on openness, uh, is one of the reasons that drew me to them. I think they have a great track record of sharing the results, whether it's Cohere Labs, the results of their research, or whether it's the command line of models that are coming out of our modeling team. I think there's incredible value in terms of making our own work better, how holding a higher bar, and also really empowering the community to keep building with us.
Yeah, and K and keeping a a kind of a level, some level of transparency as well, right? We're going to take a short break. When we come back, mitigation risk and how Joelle found herself in the cockpit of a helicopter, all for the sake of AI research. I want to talk about risk assessment and risk mitigation when it comes to AI. And I want us to dig into benchmarks. And the story that comes up for me that I want to share with you.
So when I spun up my company out of MIT Affectiva, and we did computer vision to understand human emotions and human facial expressions. And, you know, of course, we would run all these validation tests on our data sets and whatnot, and we would look at all sorts of accuracy measures and scores. But at the end of the day, I would bring every like all our R and D team, and we would lock ourselves in a room and we would literally watch all the videos and take a stab ourselves on kind of deciding what category, what class every video fell into.
My thesis back then, which I still believe is true, is you build an intuition around the data. And I think that is so important. You can't just, like, delegate the results to a kind of a harness of tests that you do. So I'm just curious, what's your approach to thinking about how we benchmark these models?
there's a couple things. I think you you talk about, you know, both like the formal evaluations and then the more informal experience of of the model, which takes a different shape depending on what's the model. I I think more and more we also have the challenge that as we're building general intelligence, it's not, you know, whether it's the automated metrics or whether it's the human experience of the model, both of that becomes much more multidimensional. And so, you know, let's say you have just a model for translation from one language to another.
Like, there's some pretty standard metrics, BLEU, ROUGE, and so on to to do machine translation. And then there's the experience of it. But in the experience of it, you're just going to expect to look at translation. You're not going to start expecting like a chatbot that can, you know, solve all sorts of your personal problems, gives you some advice, handle your email, all these other things. Like it's a narrow set of qualitative experiences as well. As the models get more and more general, models that suddenly are multilingual, multimodal, including chat and reasoning, agentic behavior, tool use.
You start having a huge surface of automated tests, but you also have like a huge surface for humans to form an intuition. And it becomes quite quite difficult to know, you know, what to look at, quite frankly. And and and we alway we have that problem in practice, like we train up a model, we think it's a good model, but then it's like, out of this whole set of evaluations, like, which one do we look at?
Which one do you weigh put more weight on, right?
Yeah, and and often there's a difference between what the external community might have decided to focus on, whether it's like the Math Olympiads or for a while we were looking at Sweep Bench and so on, versus what actually matters to the paying enterprise customers that we serve. And so, you know, that has like a whole other dimension to it. So there's no perfect answer to this. I would say in general, we try to have a clear sense of, like, what are the benchmarks that are sanity checks?
So they may be saturated, but we still need to check that we don't have regression on those. What are the benchmarks that are really driving the progress? Because they're the benchmarks that are, you know, we're on we're on the edge of unlocking. Some of them are from the broad AI community, and some of them are directly from our enterprise customers, and we look at a mix of those. And then we have a third bucket that's more the experience, you know, we have a whole harness for AB testing and for bringing in human evaluators, and so we also leverage that as a way to do it.
One of the things we often hear is the models that we have, you know, they're solid on some of the sanity checks, they're solid on the except the external benchmarks, but they're particularly good on the enterprise benchmarks, and they're particularly good in terms of the experience. And, you know, it's not it's not completely, you know, a surprise. I mean, like, we consider that as part of the the development. But but we can't afford to sort of overfit to just the enterprise benchmarks because I think we lose some of the generality, and so it's important to keep in mind the broader set of benchmarks.
So, you know, on the one hand, you've got all these benchmarks that a lot of the other foundation models also use. But how do you create an enterprise benchmark? Like, do you I I imagine you have to partner with these customers, right, to get, like, the right data set and create the right harness. How how do you do that?
It's mostly about looking at the right tasks, and it's about so defining a question of, like, yes, the right data, but it's more like what information and tools are we gonna allow? And from that, you know, you have a task. What's the concept of done? What's the concept of great? And so when we have tasks that have clear, testable conditions for for completeness and success, then that's usually a good one to build an evaluation out of.
I was on a panel recently and someone shared, you know, you should never take a broken process and use AI to fix it. Because when you have a broken process, you don't have a definition of of what done or or great looks like. And so you're not going to be able to build up an AI solution for this. So picking things that you know you have a good sense of how to assess, whether it's done, whether it's well done, it's usually easy enough to build an evaluation for that.
Yeah, now I believe that diversity is also very important when you are evaluating these mo- well, both training and evaluating these models. Do you have a specific point of view on how to bring diverse perspectives, I guess, when thinking about these models?
Yeah, I I'm smiling because I think, you know, there's an anecdote from my very earliest days that just stuck in my head. And for all these years, I I go back to so often. I've told this story a few times, but when I was an an undergrad at at the University of Waterloo, I did a a work term one fall at the Flight Research Lab in Ottawa, where we were essentially building like speech recognition system in the cockpit of helicopters.
And so, you know, like, we'd have test subjects come in, test the speech recognition system. We'd go in flight, the pilot would ask all the commands with voice, and then, you know, pilot the the helicopter using standard command. And after a few e- evaluations, at some point, I I asked my manager at the time, said, like, can we get a female pilot in there? Like, we know voices of men and women are are different usually, just in terms of of range.
And so, like, can we just validate that this is working also for for women pilot? And to his credit, you know, like this was a number of years ago, you know, bias in AI was not on the agenda. To his credit, my manager said, like, great question, let's figure it out. Called up the the the pool of pilots and and they could not find a woman to send to us. And again, to his credit, he said, not a problem, Joelle.
You are going in the second seat and you will be piloting this helicopter and running the protocol yourself. And so I found myself piloting a helicopter, running the whole test protocol. I wasn't enough to have one test subject to to give us like a thorough evaluation, but it gave us a measure at least of, like, the discrepancy in performance as a sanity check. So, anyways, that stuck to my head in in a couple ways. One, I think the the difference really came to just having someone in the room to ask the question, right?
Often, you're not you have a diverse set of people at the table, you're not going to get necessarily diverse solutions. Like, I'm going to build my speech recognition system the same way that someone else is going to build their speech recognition system. But we will ask different questions, and that's going to push the research in a different direction. So I'm a huge believer in building diverse teams for that question. Like, what are the set of people we need to get around the table such that we're going to look around the corner and ask the questions that no one else is asking, such that we can build products that are better?
And that especially when it comes to general AI, in order to be more general, you need to open up the aperture of people who are building these models and the questions that they're asking and the types of tasks that we're putting forward for these models.
So how does that translate to what the team looks like on the ground? Like, like, like, what kind of diversity of backgrounds or perspectives do you want to see around the table?
In some cases, you know, some of it is building teams that have diverse expertise, so we tend to have a mix of like research scientists, research engineers, some sw Ees, designers, user research, some program managers, kind of all working together. Some of that is we are working across several countries. We have teams based in Canada, teams in the UK, in France, in we have folks in Germany, we have folks in the US. And through that, we've become really focused on building the world's best multilingual models.
So cool. So the IL line of models, which came out of, you know, I think the team being quite diverse and being interested in this question of multilingual modeling. And now it's opened up new commercial markets because when we get to Korea and to Japan, you know, of course, there's like, oh, you have world-class models in the local language, and so it's opening up new perspective. But that didn't come from like a top-down directive from like our revenue team saying, like, you know, we have a request from Korea to build a Korean language model.
It came out of, like, our research team being interested in democratizing access to language model, working with a large external community, assembling the data, building the model, and so on and so forth. So I think that's the the other the other aspect of that, like having a community that's really quite diverse from a from a geographic, linguistic, racial point of view. It really opens up the perspectives in terms of the work.
Yeah, very, very, very cool. So I'm originally from the Middle East, and I I'm Egyptian, grew up in Kuwait and Abu Dhabi, before I moved to the UK and then the US. And I'm just very curious about the question of sovereignty in AI. Yeah, we we see different countries wanting to build their own AI tech stack. What are your thoughts on that?
We are definitely seeing a strong movement towards sovereignty. I think it can mean different things, but at the end of the day, this notion of sovereignty comes down a lot to the ability to to have some control over your technology. For us, cohere ourselves, you know, the fact that we build our own models as well as build our own product is part of the strategy, and part of that is just to be robust. You know, by controlling the model, we control the data mix that goes in, we control how the evaluations are done, and we have a really good understanding of how we can shape that core piece of the technology such that we can build better products.
So I'm really sympathetic, frankly, for for this desire for more sovereignty. I also see a ton of appetite for it. I was in in Paris earlier this month, I was in in Davos earlier this month also, and and everywhere I went, you you do hear a lot about this question of of sovereign AI. Some of it is, of course, related to to questions of geopolitics, but a lot of it is actually just questions of having a resilient access to the technology for us as an individual and for companies.
Yeah, very cool. So I want to kind of look towards the future a bit. So I recently spoke to Kyle Law. He's the CEO and co-founder of an AI company. We hopped on a call and we had a Zoom call together. The caveat is the catch is he's an AI agent. So the company is mostly a group of AI agents, like the CMO and the COO and this head of human resource. Well, actually, it's head of it's like the head of resources, but I guess it's mostly
Agentic resources.
AI, exactly, agent resources, uh, with a couple of humans also in the mix. And it was fascinating for a whole host of reasons. I think we are going to start to see more of these hybrid AI human teams and organizations. But one of the epic fails, I would say, of these agents is that they didn't quite have a world model. And so they didn't have a real sense of time, and they didn't really have a sense of, like, what is okay to do and say and what's not okay to do and say.
Are you seeing also the unfolding of this kind of hybrid agentic and human world? And how should organizations think about that? And also, like, how do you instill culture and leadership into organizations like that?
Hm, that's an interesting one. I'm definitely seeing a lot of that, though. I would say, you know, if you look at the spectrum from like one end of the spectrum being the case where you have a fully human organization where you're starting to pepper in a little bit of AI versus the case you're talking about. And so a lot of these questions will will have very different shapes in that. The case I see usually it's people in teams who have, like, a very clear sense of, like, a particular piece of work, like a particular deliverable that could be done with AI.
And suddenly, with the right level of autonomy, agency, the right information flowing in, the right LLMs, they can actually accelerate that piece of work significantly to the point that something that would previously take hours to do suddenly is down to just, like, a few seconds. From a culture point of view, I think there's still a lot of open questions. For now, one of the biggest blocker, I would say, is adoption. So people come to AI with all sorts of preconceptions, some of them legitimate concerns, and a lot of them unfounded fears.
And so we have to navigate through that to get them to a position of curiosity. So from a posture of curiosity, you can start playing with it and see how to integrate this technology into your workflow. But if you're coming at it from a from a point of view of skepticism and fear, without that curiosity, it's going to be really hard for you to take advantage of that technology. So in terms of culture, I would say we're still in that phase of how do we get people to just be curious about the technology?
And through that curiosity, you achieve productivity.
Yeah, very cool. Yeah, there's this MIT study that found that 95% of organizations see no measurable return on their investment in AI. Is that kind of consistent with what you're seeing in your world?
I I mean, I no, I'm seeing a lot more value than that, but it's still very early days to put a number on it. I would say I would love to to, you know, talk to you again in a year and have, like, a much better way to capture the the productivity gains that we're seeing from the technology. I think you can read this number one way and say, like, oh, there's no value in AI, and I will stop trying.
Or to say, like, we haven't figured out the way to do it, the right way to do it, the right task, the right way to bring it into the the work that we're doing from a either culture point of view, process point of view. So I'm much more leaning on the second, I think, to to think that this is evidence enough to not lean into AI. If you're running a company or if you're involved in government, to me, seems really misguided given the trajectory I see for this technology.
Yeah, good mentor of mine, Peter Diamandis, he has this quote that in a decade, there'll be two types of companies or organizations, those that have adopted AI and those that are out of business. And I I I really believe that, yeah.
I tend to think so too. Yeah,
All right, a few rapid-fire questions. What's one way you're implementing AI in your everyday life that you know you can't live without?
I'm still very much like, go to AI whenever I'm stuck on anything. If there's something I don't know how to do, whether it's like fixing something at the house or doing something at work, if I don't know how to do it, I just go straight to AI to just unblock myself.
How are your kids using AI?
I have four kids who are sort of sixteen to twenty two, covering sort of high school, late high school, till university. A lot of help with, uh, homework for your standard teenagers. This week, it was also a lot of help writing cover letters for summer jobs.
Okay, that's a good use case. AI as a thought partner in professional matters, yay or nay? Personal matters?
Yes, also, though, I tend to turn less to it. I I tend to turn to my friends and family before I turn to H AI on the personal matters.
Thank you, Joelle, for joining us. This was super fascinating.
Thank you. It was a pleasure to chat with you.
It was so fun talking to Joelle about her take on where AI is headed, especially digging into OpenClaw. So here's my take. OpenClaw feels like the ChatGPT moment for AI agents. We've all been talking about agents for the past year, but this is the first time we're seeing them broadly adopted by everyday consumers. What makes this even more notable is Multibook, a social network where agents can interact and even learn from each other. Again, this is the first time we're seeing AI bots communicating in the wild at scale.
That said, it also surfaces real concerns a lack of security and privacy, plus the clear need for human oversight. Personally, I'm not going to download it. Too much risk. The mainstream version of this technology will need to be a lot safer. It will also need to be packaged in a way that's accessible and not as techy. But this is definitely headed in the right direction in terms of what AI agents can do for us. Thanks so much for tuning in.
If you haven't already, subscribe to us wherever you're listening so you don't miss an episode. We'll be in your feeds next week. Pioneers of AI is a WaitWhat original production. Our executive producer is Eve Trow. Our producer is Rachel Ishikawa. Our senior talent executive is Stephanie Stern. Mixing and mastering by Brian Pute. Video editing by Eric Purcell. Original music by Ryan Holliday. Our head of podcasts is Lital Malad. You can join the conversation across social media platforms.
Just look for us at Pioneers of AI. Thanks so much for listening.