Building AI Into Systems That Can't Afford to Be Late - with Drew Barbier of MIPS

Welcome everyone to the Emerj AI in Business podcast. Today's guest is Drew Barbier, Vice President of the IP Business Unit at MIPS. Drew discusses why teams building AI into vehicles, robots and industrial equipment get better real-time performance when they start from the software workload instead of the hardware. And he shares an example where that approach delivered a three to five times improvement in a motor control application. He also outlines what changes on a team side, not just the technology, when real-time requirements are treated as a starting point rather than an afterthought.

Today's episode is sponsored by MIPS. In this episode, we cover the shift from hardware first to software first design for real-time AI systems in vehicles, robots and industrial equipment. To go deeper on this topic and to learn how to structure landing pages for higher conversion and how to use self-qualification systems to prioritize high intent leads, download our free PDF report B2B AI lead generation guide at Emerj dot com slash AIG one. That's M E R J dot com slash AIG one to download your copy.

Now, the conversation with Drew.

Drew, welcome to the Emerj AI in Business podcast studio.

Hi Yolandi, happy to be here.

Yeah, so very excited to pick your brain. I think our audience members will find that a lot of what you have to share today will give them a better idea of what's happening in the world of vehicles and robots and industrial equipment. So when I look and listen to the current conversations, a lot of the AI conversation right now is about how smart. The system can be, but for something like vehicles or industrial equipment, being smart is not gonna be enough, right?

Um, especially if you have response times that come in even half a second too late. So that's where I actually wanna start is how we should be thinking about this. So what's the broader approach teams take when deploying AI in environments that depend on hardware, software systems running real-time workloads?

Yeah, great, great question. At at MIPS, we we call this physical AI, so as as AI sort of leaves the data center and and moves to the the broader edge. I I think what you're gonna find is that the the applications and the things that people care about, especially at the edge, is gonna be more narrow, more more specific, right? Getting away from general purpose AI and and running AI on things like GPUs and and the like.

So I think first, you know what, what people are gonna find is that they're gonna start with workloads and look to optimize for specific workloads. Cause that's the only way you're gonna get the performance gains in these sort of constrained environments and and systems, while maintaining your your thermal envelope and and compute requirements. So starting with the workload, understanding specifically what you're trying to achieve, and then driving requirements down from there. I'm a hardware guy, my my background is is CPUs, been in CPUs for a long time.

Obviously, I always think about problems from a a CPU first perspective, that that's not the case with AI. AI, you really start with the software side, with the workload side, and then drive that down and understand the implications of that, not just on the CPU, but on the system as a whole, right? How is data gonna flow in, how are you gonna compute on that data, where does data need to flow out? What are the responsibilities in terms of response times and latencies and memory requirements and and things like that?

That really turn this into a a much larger system problem than than a raw compute problem.

So is that just a general starting point or is this different from a team that's just optimizing for accuracy or raw performance? Do we start at the same level, the same platform, the same idea?

Uh, s somewhat, but again, it goes back to workloads, right? Cause not all workloads require the same levels of degrees of accuracy, things like that. You can be flexible in some systems, whereas in others, you you absolutely care, right? And you want the largest data types possible, so you can have the most accuracy. It real there there is no one size fits all answer here, right? And it's gonna be very dependent on the workloads and the systems that that are being deployed.

Now, obviously, from a MIPS standpoint, we try to capture as as much of that a as we can. But even from an IP design perspective, we are starting from workloads first and using that to drive design decisions on the hardware side, on the CPU side, on the the accelerator side.

Yeah, and that's actually what I wanted to ask you about is that where in the design process do we start thinking about it? Is it something that we integrate right from the start, or do we first get something else up and running or something else checked and then think about the real-time requirement?

No, it go it goes right back to the workloads, right? So coming in from the workload first, and a lot of times this, uh, the way we realize these workloads is in virtual environments, virtual platforms, right? We have tools like Atlas Explorer that enable us to glean very detailed performance insights on larger workloads, larger data sets. And we start there, and a lot of times this is before we even begin designing or writing any RTL at all.

So we start there and we have models of microarchitectures, models of systems, things like that that we use to to start the process. And you refine that based on the whatever design objectives that that we're trying to achieve.

Okay, so quite important to have that built in right from the start, and like you said, even before the design process starts. So, so what happens when a team thinks of this as an afterthought instead of a starting requirement? What have you seen happen?

Well, well, like I said, as an IP provider, right? We we can't be so narrow in in where we deploy these things, right? So a lot of times there are driving factors and decisions that we make and products that we pursue. Obviously, we try to to find as many applications that fit on top of that hardware as possible. But again, it it always goes back to the software first, right? Getting the workload in house, running that either on the performance models or on the RTL, if we have it, understanding the performance of, you know, maybe workloads that we didn't conceive of and how that runs on the hardware that we have.

And then from there, you can start making design trade-off decisions, right? Do we need to make a a new product? Do we need to make a an enhancement to to capture another use case? Uh, all those trade-offs that that come in at that point. But again, always trying to go back to this software-first mentality, especially on these AI workloads that that's so important.

Yeah, so the theme there is, you know, your workload will determine, uh, what the first step is, and it's not just about bolting any software into what you have. It's about thinking back, going back to the original problem and actually solving a problem and not just creating more problems for yourself. So Drew, can you walk us through some real examples, a team that was struggling with a system that couldn't keep up and then something changed and it actually worked out the way that they needed to.

Any use cases you can share with us?

Yeah, I I think a a good one, and it's one that we've talked about before, but kind of drives the point home where we've seen substantial improvements by by being a little bit more market and application specific. So we have a product called the M8500. It's a high performance 32-bit microcontroller IP that is specialized for motor control applications. And this is another one where we had workloads in house, analyze those workloads on the performance models before we ever design the RTL.

And that led to specific design decisions inside that product, right where we added very specific real-time capabilities. So this this core has something that we call real-time multi-threading, which helps lower latency for context switching quite significantly. But even further than that, we added specific ports to to move data into the core and out of the core as fast as possible that we might not have otherwise have have put. And then when you're looking at motor control specifically, there's specific compute that that happens in those control loops, specifically, uh, trigonometric functions, and from there, we we were able to to add those to those type of compute units to the core.

So any one of those things helps, but really, when you put all those things together, you get substantial benefits. And in the case of M8500, we see uplifts from from 3 to 5X on some of these field-oriented control algorithms. And those are numbers that that that are meaningful and get customers excited and interesting. Um, and you know, just just going back to the workloads conversation that we had a few minutes ago, you know, it's only by specializing that you're able to get these levels of benefits.

So building general purpose processors, we do that. We've been doing that for decades. We're good at that. Um, but the only way to to really move the needle in these very constrained environments and applications is to get a little bit more specific. And in that sense, open standard serves us well. We we're big supporters of of RISC-V, big contributors to RISC-V. And RISC-V is great because it gives us that foundation, that ecosystem to to start from, but still enables us to go implement more specialized functions on top of that without losing compatibility to to that ecosystem.

So in that way, we're able to to specialize, but still have very broad support from from an ecosystem.

Okay, so so before that, what was failing for for a team like that to realize, okay, we need to integrate, uh, and something, we need to make changes. Was it was was speed a problem? Was consistency a problem? What was the what was failing before?

It It wasn't really a a a failing. This is more of how do you differentiate yourself in a market, right, where 32-bit microcontrollers are are everywhere? They're they're ubiquitous to to some degree, but at the same time, differentiate ourselves, but also provide real value to our customers. You know, a lot of times, again, being a hardware centric person, all of these conversations devolve into PPA, power, performance area, and you're looking at sort of standard sets of benchmarks and silicon area and these types of things.

And and what we were able to do on the M8500 is kind of elevate that that conversation, right, where, yes, we have good PPA, all those things are important, but you missed a customer on your workload that that are doing motor control and motor control like things. You know, not only are we gonna be, you know, 20% better in terms of raw IPC, but on your workload that you care about, it's actually 3 to 5X better, right?

And that that's a much different conversation than going all the way down to core mark scores and and silicon area right right from the get go. And there's also conversations that that led to where, um, not only is it better from a a CPU perspective, but now customers aren't having to put down a microcontroller and an accelerator. All of a sudden, those functions, both of those functions, get consumed by by the M8500. And so it changes the conversation, right?

Cause all of a sudden, the the area comparisons, no longer just a 32-bit MCU versus a 32-bit MCU. It becomes our M8500 area compared to a 32-bit MCU plus an accelerator, right? And then you start looking at system level comparisons and the the amount of data movement across the MCU is much less because everything's done on the M8500. And that leads to better power conversations, lower power consumption, all those types of things that come into play.

So it elevates the conversation, right from classically fitting a CPU into this sort of defined box that that can't change to now, all of a sudden, the the scope of that box is much broader than than it was before.

Yeah, so, like you said, it's not about failing, it's about being good enough and wanting to be better or even the best. So, uh, I'm thinking that if if you have that change in platform, software, hardware, all of that, you're gonna have to get a team that matches the energy of wanting to be better. So what changes on the team's side to fix that? Uh, not just the technology side, do we? Is there a transition that needs to happen?

Not not just a transition, but just a mindset shift, right? Because, you know, every company has hardware folks, they have software folks, they have system folks. And again, you know, just to go back to being a CPU guy, we always start from the CPU and think about all these cool things we can do in the CPU with no regard for the impact of that on the software world or on the system side. And so if you flip that around where you start with the software and the application, that then drives design decisions downstream.

And I think it makes for for a lot more interesting conversations and trade-offs as opposed to designing the hardware first and then trying to figure out the software glue to fit that in a customer's, uh, actual application.

So I guess the question that will be on most of our listeners tongues is, um, results. If if we think back to the use case you gave us in the beginning of this question, what told that team that the fix had actually worked? What what results did we see? Um, what actions led to interesting conversations where people left with big smiles on their faces?

Yeah, so results from our end is customer adoption, and it's still early days, right? So there's not a lot of customer silicon that's out there that we can talk to in in terms of the M8500. But really, this is something MIPS has been doing for a long time, having slightly more, more specialized compute even predates my time here. So MIPS is famous for for multi-threading. Multi-threading is one of those things where if your application can leverage it, you can see substantial results.

So MIPS has traditionally been multi-threaded and has a lot of success in networking and and data movement applications and things like that. M8500, specifically, the the messaging, quite frankly, is is led to a lot of deep engagements with customers and change of mindsets and willingness to to engage with us and on roadmap products and understanding their workloads and using that to drive roadmap decisions on on our end that can impact customers down the line, two, three, four years from now.

Yeah, I like that, and I guess that kind of weaves in perfectly with what I was about to ask you with the the roadmap and the lessons. So how can our enterprise leaders and our listeners take lessons from these real-time use cases and apply them in their own environments to make meaningful decisions and meaning and to make a meaningful difference right now in their current environment?

So good good question. So I I think not being not going into these conversations with a mindset that you understand all the components that make up these systems, cause we're we're just as guilty of this as everybody. We tend to put things in a box and not have a not think about that box expanding or the scope of things expanding for for these different compute applications. So I I I think coming in with a more open mind from that perspective and coming up with creative ways to solve problems and really understanding what the problems are.

And again, it goes back to being software driven. So if you can start with a software worldview and really understand the pain points of the software engineers that are that are coding the applications and where the bottlenecks are in today's systems. That can help drive much more meaningful conversations with suppliers like ourselves in the the compute IP space to to open up the scope of the problems that we can address and and help customers solve. So helping us and engaging more at at at a system level, even though we don't have responsibilities for some of these system level components, can certainly help us design better products going forward for for these enterprise type customers.

Understanding workloads, helping us understand workloads, you know, we have systems engineers, we have software engineers as well. We have a lot of tooling and capabilities both inside of MIPS as well as that we make available to to customers to to help understand this and be able to share some of these things with us so that we can, in turn, go design better products for for our customers. I think it's incredibly important.

And if if we have a customer on board or an enterprise on board, what is a sign six months in that they're on the right track? Uh, again, results driven, so they wanna know, okay, six months in, how do I know this is actually working and this is actually going to scale?

Again, it comes down to, uh, to results, right? So, uh, a lot of early engagement pays dividends later. And and going back to tools that we have, like Atlas Explorer and a lot of these virtual platform, um, tools, digital twins, those kinds of things where we can engage much earlier in the process. But also understand what the res what the solution is gonna look like much earlier in the process as well. And not waiting till the IP is done or waiting till the the system is put together to to understand that.

There there's a lot of that happening in the industry right now, and I expect that that to continue. And it's because it's expensive to to make mistakes here, right? Um, if you, uh, don't deliver on what you're promising on a system, I, it's very expensive for for everybody involved. Um, so understanding those things earlier, uh, is always better. So engaging with a lot of the the tooling and technology that we have to understand workloads, to design SoCs and systems around those workloads, and then understanding the implications of that, not not just at a compute level, but at a system level, is certainly something I would I would encourage everybody to be doing if they're not doing already.

Yeah, absolutely. And that's exactly what encourages our listeners to tune in every episode is to try and learn from other people's mistakes and avoid those expensive mistakes. Uh, Drew, we we we've we're headed towards the end of this conversation. So is there any specific thing during this conversation that you just wanna highlight again and just tell our audience members this is what you should remember from this conversation. This is what you should take to the boardroom with you on Monday.

Absolutely, I'll I'll keep hammer hammering the the point home. Uh, software first, start from the application and the workload, constrain that as much as possible. And if you engage with your partners earlier with those constrained workloads or applications, you will see much more substantial upside to to that engagement than you would otherwise.

Uh, great highlight. Drew, thank you so much for for giving us some insights and sharing your use cases with us today. It's been a lovely conversation.

Thanks, Yolandi. I appreciate it.

Wrapping up today's episode, let's look at the three key takeaways from our conversation with Drew. First, real-time performance has to be a starting design requirement, not something added on after the hardware is already built. Second, the clearest gains come from starting with the software workload and letting that drive hardware decisions, not the other way around. And finally, engaging with the specifics of a workload early rather than treating it as a generic compute problem is what separates teams that see measurable performance gains from those that don't.

In this episode, we covered the shift from hardware first to software first design for real-time AI systems in vehicles, robots, and industrial equipment. To go deeper on this topic and learn how to structure landing pages for higher conversion and how to use self-qualification systems to prioritize high intent leads, download our free PDF report B2B AI lead generation guide at Emerj dot com slash AIG one. That's M E R J dot com slash AIG one to download your copy.

On behalf of the team at Emerj, we'll see you on the next episode.