AI Proficiency: From Users to Builders

Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X or Bluesky to stay up to date with episode drops, behind the scenes content, and AI insights.

You can learn more at practicalai. fm. Now, onto the show.

Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host Chris Benson, who is a principal AI and autonomy research engineer. How you doing, Chris? I'm doing good today. How's it going? It's it's going great. I'm continually impressed by just the the amazing AI work that's happening around. Uh, n- not only on on the coast, but in kind of the, I guess, the heartland, the the middle of the country, where a lot of actually the large enterprises of of our country are are located.

Um, and we have a guest related to that today. But, um, but I do wanna remind folks, uh, we're. We're also involved and a sponsor of the Midwest AI Summit, which is coming up October 15th in Indianapolis. Um, really cool event, Chris. You were there, you saw what was going on. There's tables where you can sit down with actual AI practitioners and, you know, rather than just hear a bunch of talks, you can actually get feedback on what you're doing, suggestions, design, et cetera, and hear great talks.

So recommend people check it out, Midwest AI Summit. You can use the code Practical AI 20 for 20 off. But we have an amazing, uh, an an amazing AI practitioner from from close by my area, um, more towards, uh, Cincinnati. Um, Mike Lewis was on a previous episode with us. We got a ton of great feedback on that episode. He's chief AI architect at TiER1 Performance. Welcome, Mike.

Oh, thank you, Daniel. It's great to be here again.

Yeah, like I say, we got I. I had multiple people comment on just the utility and and insights that they got out of our previous discussion. And just just for context, cause I think it's so impressive. What, uh, what, what you're doing and what you're involved with, could you just set a little bit of context for? Kind of the the types of projects that you work on, like, give an example of kinda some of the types of companies that you work with in relation to to AI initiatives?

Yeah, sure, uh, so, uh. TiER1 Performance is an end to end organizational performance and transformation partner. What does that mean? Uh, we, we, we kinda help, uh, some of the most of the largest organizations in the world rethink transformation. Um, I w- I would add in, though, that we have a nontraditional approach and so and and a and a multi decade uh uh track record of success there. So, you know, the types of clients we'd work with day to day, uh, would be like, uh, Google, Eli, Lilly, Takeda, um, Air Force, you know, I mean, you know, just most most of the force Fortune 100s, and um, we see everything related to transformation.

So, uh, obviously, uh, AI would just represent a sliver of the types of challenges we we encounter as an organization day to day. But act m that sliver is my whole life, you know? So I, I don't really see all the other stuff. I g- I kind of get a glimpse into it every now and then, because more and more AI touches everything. Um, but uh, uh, we, we are sort of a transformation partner. And um, so you can just imagine how, uh, the phone starts ringing when, you know, a an a massive disruptive technology is birthed into our lives, whether we want it or not, right?

y- yeah, and a part of that, I think people are struggling with on the transformation side is there's just so many sources of information. There's so much noise around, like what you should be paying attention to, what people are actually doing, what, you know, people are saying they're doing. What is hitting the news? Like, there's all sorts of a range of things, and obviously a lot of that can be noise. And one of the things I've appreciated in interacting with you over time is that ability to kind of pick out some of that signal from the noise and develop some kind of structural ways of thinking a about AI and, uh, transformation I in an actual real world organizational environment.

Uh, in your kind of day to day, obviously, you're hearing a lot of things, both from customers that you're working with, the the news cycle, the things from, you know, Anthropic, OpenAI, Hugging Face, whoever, that that's coming out with things. What? W- what does your process look like? Or, or how do you feel like your your rhythms day to day help you kind of isolate some of that signal? Maybe kind of distill down some of what you should be paying attention to?

Any, any suggestions? I I think it's something on all of our minds, certainly on my mind.

Yeah, well, maybe I'm I'm very good at this because, um, I, I am just like the the weirdo, uh, oddball guy who, like, stumbled into this at the the perfect moment. I mean, Daniel, you you know this about me, and your audience might remember. I was a portrait painter for almost 20 years. I mean, from when I graduated college till 2016, I I painted, uh, portraits and and I owned a fine arts company and and apprenticed a bunch of artists.

And I only I only got into this because, um, I had heard about the DALL-E model from OpenAI and applied to become one of the commercial art's beta testers. And I lucked into that. I got a email one night at dinner and I was like, Oh, wow, I just got access to this thing no one had heard of at the time. And so, and I knew a little bit about, like, Python and coding or whatever, so enough to like, fumble around and get things installed.

And I, I fell in love with language models, I mean, almost instantly. I just couldn't believe w- all the new things you can make computers do. And, you know, this is before we really thought about them as chatbots or whatever. In my mind, it was like something you plug into code to just manage edge case weirdness. To, you know, pick a path and and make your app keep working. Um, and you know, context models were minuscule back there.

I'm running things locally on my gaming computer, you know, um, and then, um, you know, medical records companies start calling, and, you know, and I build this book of business and and tier one acquires my company. Here here's the only reason that's worth unpacking is to say, like, I don't have this like muddy, long history in the industry where I had opinions about everything or like a solid, you know, way of thinking about it. It was all just fun toys for me.

And then I think the way I've made it through business my whole life, I've owned maybe a dozen businesses, all s all small. But um, is I've? I've always just, I've just had a a firm rule is like, I do not concern myself with things outside of my sphere of influence. So, you know, there may be this big thing going on at the edge of the AI sphere. But the the reality is like, if I, if I don't have influence, or if it doesn't impact, like, the way I'm interacting with my clients, if it's not, then I just don't.

I just ignore it. These things are not going away. Like, I know for sure, whatever this is, it will be here probably forever. And and in in my experience, I'm a much better worker. I mean, I can fool almost anyone into thinking I'm a smart guy when I'm using AI and I'm aligning context to, like, whatever my work project is. I mean, it just works so much better, uh, using these tools, and I just have that conviction.

That's probably true for for most people. So so, Daniel, my trick is really only focusing on, like, what, what directly impacts, like, the things going on in my day to day. And and at tier one, what that is is a workforce enablement performance, you know? Um, basically, um, uh, not not worrying so much about tools or whatever, but just outcomes and performance.

yeah, that that makes a lot of sense, I and one of those things, you know, last time we talked, I looked up the date. It was like, 2024, right? Uh, yeah, some sometime in 2024. Obviously, a lot has has changed. It's impacting all of us in different ways. At that time, you t we had a discussion about what's, I think, doable, scalable, ex et cetera. Um, you, you've, I know, spent a bit more time recently thinking about this concept of AI proficiency, and I know myself, you know, obviously, if you're doing AI things, you're searching them, you get targeted online.

I'm all this all the time seeing these like ads for whatever it is. Uh, Harvard school, uh, like, what the engineering or what? uh, uh, becoming AI proficient in your business, or whatever the, you know, the course or or thing is. And so there there's a lot of people trying to understand this because of, y- like you say, the wide reaching impact across a across the workforce. So n- next week, our our company's doing an AI accelerator with one of our customers.

I know that there's gonna be a whole range just from discussions from y- y- uh, of quote AI proficiency, um, but general, like, other than thinking, well, I know that there's that range, right? Um, it's another step to go and say, well, in light of that range of proficiencies, what do I need to change for different proficiencies? How do I advance people through proficiencies? Do they even need to ad advance through different levels of AI proficiency?

What is enough for some people? What? What is too much, right? So I I don't know if you wanna help us launch into this topic. Uh, maybe just starting with a little bit of definition of what we mean by proficiency. W- would help us just so that we're all talking about the the same thing.

if you don't mind, Daniel, I'd like to even go a step further back and just give you a s a sense for how and why I even started thinking about this. So if if inside of our, if at tier one, we have, you know, most of the largest we and three or four other companies are helping the largest companies in the world navigate this disruption. And we also see the same story playing out inside of every company.

Like, I mean, I could, I could tell you 95 of the companies. I could walk through the timeline and you'd say, Yep, I saw that at fill in the blank, fill in the blank, fill in the blank. They've made all the same mistakes that I I. It's kind of watching that happen, sort of helpless at the periphery, you know, as a consultant, you, you don't have a lot of control, but, you know, and y- you don't have perfect ideas either, you know?

So everyone's just sort of watching this thing unfold and like, Oh, I really thought that was gonna work, but that didn't stick. And and the big thing that I focused on early on was developing tools that help people learn and get good with AI. So I I developed our, you know, solution archetypes, frame framework, where we kinda, like, tried to pin down. We talked about that on the last episode. And then I built our superuser habits index, which is basically it's like, Hey, these are the 24 habits we see superusers displaying.

And and and then I built a coaching tool that people could use. And and and in my mind, it was like, I'm gonna put all this together and turn people into superusers. And guess what? It just kinda didn't really happen. For some people, it did, but I couldn't help but feel like most of them it was gonna happen anyway. And at the same time, I'm watching, um, I'm at the same time, I'm sort of reading all the Substack articles on it, watching things unfold at companies, looking at my work history, you know, thinking through the peop- the looking at the statistics we create, or, you know, stuff in the in the literature.

And then, um, I think the breakthrough moment for me, I was invited to a, uh, a bachelor party with some young guys who used to work with me. I, you know, never again. But, um, I brought 200 pages of, um, research citations and synthesis. So this is not 200 pages of research, it's 200 pages of, like, the name of a paper, a URL, and maybe a sentence about why it might be relevant to my work. Just to read, um, during the that week during the bachelor in, I party.

thought I was probably gonna need to pull away a lot. These are all younger guys. Um, and I did, I read it all, and there were just there were. There were just patterns in there that answered a lot of questions for me. And and the breakthrough moment, with all of that just sort of happening, and within like 30 to 60 days, that stew forming. Um, I read a Substack article, it was Peter Yang. Um, I'm kinda skimming my screen here for the details on it, but, uh, basically, uh, it was called, Uh, your new job is to onboard AI agents, how AI native companies actually operate.

And and they just had this th- th- this way of thinking through the L0 through L3. So I didn't invent that, I I read it in that subseg- L0, um, through L3 classification, so L0, L1, L2, and L3. And the L0s are the people who aren't using it, and the L3s are the ones who are, um, you know, just making a difference across the entire enterprise. And we all know those, and then they labeled L1 and and um, L2, though, like, knocked me off my chair almost when I when they talked about the concept of a non-technical builder.

And um, they just gave a name to a thing that I I just thought, Wow, this is. This is something we need to really focus on, is like the idea of a non-technical builder. We're all focusing on converting L0s, people who are disengaged, and and we'll unpack what that is, I think we need to cause. There's so much important stuff there. A And they're trying to get people from L0 to L1, and L1 is what everyone is, the, uh, what everyone knows.

It's the user, it's the one who's like, I couldn't live without this. Like. And they kind of feel like it ends here. And then that leap, you know, the big jump from like, okay, someone who, like, uses AI to wait a minute. No, this is a non-technical builder. This is a person building durable solutions that emulate the company's work in an uncanny way. Um, uh, that that jump is actually the one that now I sort of run around evangelizing, like, this is actually where you need to focus your energy.

But so so, Daniel, I just, I guess I unpacked all that to say, like, for me, I had the like, th- the stew, I guess, is the way to say it. Of like, you know, the the guy who, like, didn't have a dog in the fight, built a company on accident. It was bought by Tier One. Now I'm here, I'm I'm inside of all these companies, trying to help them make sense of this thing. And then, you know, all these external inputs sort of just clashed.

And and uh, so now we, this is, this is the framework we we push in advertising help, ask people to think through.

if you've been listening to the show over the past few months, you realize how transformative agentic AI is. Whether it's Claude Code or Hermes Agent or your own s custom software that you're deploying for operational efficiencies or as a product into the market. These systems are transformative and it's where the industry is going. But agents have a lot of risk associated with them. They have agency, they take action within your environment, within your infrastructure, within your systems.

And that's why security teams and governance teams view it as risky to deploy these agents within high impact environments and industries like manufacturing, logistics, financial services in the public sector. And that's why I've personally been spending my time working on a product called Prediction Guard, with a great team of AI engineers. Prediction Guard is an AI control plane that you host within your own environment, whether that's on prem, hybrid or in your cloud VPC, even air gapped scenarios.

You can set custom AI policies or those that are aligned with NIST and OASIS standards, and those are then enforced automatically across every handshake, across every agent. Whether that's being powered by your own self-hosted models or models in your cloud environment, like those from Azure or AWS Bedrock. And all of that telemetry goes back to your monitoring and observability system. I would love for you all to check out what we're doing. Schedule a demo and a call with our team to find out more at predictionguard.com/practical ai.

Like I say, schedule a call today. We're happy to show you how this works and why we think it is so beneficial to the market. People already have this deployed in production, powering transformative agentic systems across a number of industries. So check us out at predictionguard.com/practical ai, that's predictionguard.com/practical ai.

So Mike, uh, c- wanted to wanted to ask you a question. As you are, uh, thinking about how you get the employees of one of your client companies, you know, moving from the L zeros up to the L threes. Um, want to throw another variable in, and that is is the attitudes that people have, which are, you know, very varied in terms of like, there's there's always people who are excited, as you you mentioned before. There are people that will naturally wanna go to that L, L1, L2, and up to L3, but there are also people that are gonna push back hard on that.

And so when you're, when you're kind of coming in and you're trying to start your process of helping them achieve that, how do you bring along the L zeros that don't really wanna move? Uh, or even the ones that have done a little bit, but they're also pushing back, uh, on that? Maybe they fear for their jobs or something like that, so there's that kind of L zeros and maybe some L ones. How do you get over that process as you're trying to bring everybody into this to to kind of level up through those?

Uh, any, any thoughts? Can you kinda throw that in as you're as you're talking about your process?

Yeah, I love that you're you're asking me about L0s because, uh, the reality is, we don't, we don't try to move an L0. Here's, here's, here's the thing, uh, the the article I mentioned, the Substack article, I disagreed with 98 of what I read in that article. I just latched onto the concept of an L2, a non-technical builder. That name resonated with me, the L0s in that article. Um, at least at ramp, the way I interpreted what I read, um, they were saying that, hey, if you're an L0 at ramp, of course, it's a software company, you know, if you're an L0 at ramp, that's grounds for dismissal.

You have until this date for our system to assess you as an L1. And man, that just broke my heart because it's kinda It's like, Okay, wait a minute, let's terrifying. let's back up, let's back up. Name another technology that we would say that's true, but can you imagine you just go company-wide? Hey, by the way, everyone, we want you to geek out about Smartsheets and be great at Smartsheets by the end of this, you know, year.

Or, or, you know, AWS, or, you know, like, just in. It's ridiculous, this

It's crazy.

is just it's another technology like that. Yet I think a lot of executives are kind of scrambling and panicking and they're saying, Hey, guess what? We're gonna be AI enabled, we're gonna be AI activated. Everyone in our company needs to speak AI by X date, or there's not a place for you here. And so 1st of all, you remember, I read 200 pages of science, and a lot of it was like it, it was just about learning, it wasn't even about AI.

And and and so, um, threat framing actually does slow down adoption. So, you know, if you put, if you back someone into a corner and say, learn this, or there's not room for you here. They will not learn it as well as if you came to them and said, let's figure this out together if it interests you. And I will acknowledge that there are some positions where you absolutely need an AI enabled individual. It's true, like, you know, and so there there are cases where an L0 is not gonna work in that role.

But I think we're gonna realize in ten years, looking back at all this, that it's just not true in every role. And I think the one that hit me the hardest is my wife, probably the smartest person I know in the world, my wife, smartest person I know. And she has no use for AI in her job. She's a nurse. And, um, she kinda just like, I, I know what she does. She pokes people in the arm with needles and, you know, some other things, and there's medical records.

Maybe someday it'll be useful to her, it's just really not right now. It would be in the way. And, um, and I started doing research on the concept of, uh, l, uh, l0, people who are disengaged, and I realized there are multiple buckets. So when we talk about an L0, it's not a thing, so an L0 is not just someone who doesn't use AI. I found five major categories, so one is the performative user. They've been doing this, they've been nodding their head in all the trainings, the meetings, they've logged into the thing, they've submitted a chat, and they've gone about their job.

Like, you know, like, they're not really getting, and and hey, they're still getting their job done. Uh, there's the disinterested, uh. I think the moment when this hit me the hardest was when I was looking at all the research. And I saw, uh, a chart that showed the adoption rates of personal computers in the office space back when PCs were first released and AI. And there's almost the exact same adoption curve over the same period of time, and it looks like it's happening at the same rate.

And when I saw that, I thought, how silly is it like to think back to that? And and and like, would no one is in a panic because the people in their company haven't figured out personal computers? Now? Like, it took care of itself because everyone wound up with one, and the same thing is happening with AI. Everyone went home and and just created a chatgpt account or whatever. It's free. This is not true with your CRM.

This is not true with smart, like everyone didn't run home and create a Smartsheet account. Because, you know, like, they just gotta have one, um, and so, you know, but the disinterested are still detached there. Okay, here's another bucket, the too busy. I could give you ten names at tier one. We almost don't want them running around trying to figure out AI, cause they are just so critical in their role, they're so good at what they do.

It's just a it's like a it's a don't break it or don't touch it. It ain't broke type role. And you know, who cares if they're using AI or not? And I think that is also really hard for the people who are are hearing communication. Use AI or you're out of here. And then they can look around and see the people who, like, very often, it's some of the highest performers in the company who don't really care about it because they've they're in a rhythm.

Um, so those I kind of push past, but the next two, these are still buckets in L0. These are people who aren't using it really matter, and there's something you can learn from these people. And I think when you say, like, Hey, you're gonna be doing this, you're gonna be using AI at this company. Th- They will just zip their lips and they will not, they will nod their head, and, you know, and they'll say, Yes, you know, um, but here's the thing.

1st of all, we have the job fearful, and and Chris, I think you mentioned this, you might have mentioned this earlier, the job fearful. What do we do? what do we do about the job fearful? 1st of all, um, very often, these people don't really understand how the tool works, and they they're under the impression that if they use AI to do their work, it's going to learn how to do what they do and replace them.

And that's almost never true. I mean, just if you kinda know how these things work, I mean, you know, maybe self driving cars. I could think of a few examples of like, the AI is actually learning from watching us. But my vote doesn't really impact whether or not it's gonna replace that profession someday, you know? Um, and so I think with the job fearful, there's an opportunity to help them see. Like, hey, you might be in the middle of a self fulfilling prophecy here.

Like, if you continue to push back against this, you might lose your job. But it won't be because the AI replaced you, it'll be because Joe, who's willing to use AI, will replace you, you know? So so that's one way to address an L0, but another one, and this is actually 6 out of 10 people that we talk to, and I've seen this number in research hover around 60 in more than one place. Are the quality?

This is where I think every executive's ears should turn on. They should lean forward in their seat, and they should realize, do not dismiss these people if they are resistant to AI. There could be some very valid reasons why. And I met a person, uh, at a at a company who, um, her complaint was it's not good enough for what I have to produce for my work. And she kind of showed me, and she's just like, humans do this better.

And it was a it was a marketing role, it was copy generation, it was some other stuff, and and she was just right. And it may not stay true forever, but no one had ever actually just no AI expert had ever just sat and listened to her. And I, I walked away from that conversation with just a brand new perspective on how much value there is to mine. From the group of people who say it's not ready to do this work yet, or e- either it's not, or they need better tools or better training.

But you, you, you can't really do that until you you listen first. And so, and and I also don't think about it like a ladder. I don't think an L1 is better than an L0, and I don't think an L2 is better than an L1 or an L. Actually, if any it, it, it, I don't see it that way, I just see it as as kind of a division, um, within an organization, for how different ways of thinking about the toolkit.

And I, and maybe one follow up question on that. I'm wondering, you mentioned maybe part of the, uh, part, part of the stumbling block here is the quality disillusion, or, or, however you put it. Um, you know, getting into these tools, I I sometimes I also wonder about the way in which, like, I would put myself into the category of the AI people. Like, yeah, I'm building an a product, right? How much of this is us providing AI in a way that is just another thing that they have to add, like a different tool that they have to add in their workflow, versus something that actually works in their workflow?

Um, and what I mean by that is, you know, if, uh, if I'm, uh, t to give an example, you know, we're working with, uh, an organization now. They have teams of translators, we have essentially a team of digital agents that helps translate with a with an agent harness. And they use existing tools. They t go through the, you know, drafting and quality checking and post editing, they do these cycles, but we we don't actually say, hey, go to this different interface that you've never used and do things the AI way.

The the sort of agents work, and then stuff pops up in the translation management system that they already use, right? And so what they see is, uh, uh, yes, there's change, right? But it's actually like allowing them to do the thing that they want to do with the tools that they're using. I in a in a way that actually they they w- want to do that more. Now. That may not always be possible, like there may need to be a shift of, like, how people, you know, the interfaces that people use to do that.

But I don't know, do you, do you have any experience with that? Like, how much of this is? We're asking people to use yet another tool, and they've already got tools that they like, whether they're AI or not, right? And how much of it is the actual AI? Um, output? I I don't know if you have any sense of that.

no, I, I hear, I think I hear the the sort of the spirit of the question. And so this is where I land, you know, when it comes to, you know, how do we, who do we train to do what, and what do we give them, and how do we, you know? And and honestly, at this point, I just feel like that is part of the noise to me. Um, in in all the conversations, the the main thing it all traces back to is executives are complaining we're spending too much on all of these licenses, we don't even understand what we're buying.

I can't measure, I mean, yeah, I see this or that anecdotal piece of data, but I can't measure, like the real impact, like, what is the value of the the, uh, this investment compared to the cost? And um, so we, we just say, like, whoa, let's zoom way out, let's zoom way way out, and let's think about where we should actually focus. I mean, if we, we, if we've already said, okay, L0, we understand.

Some people just really aren't gonna be using it, maybe ever, but definitely for a while. Okay, let's not even think about them right now. We got the L1s. This is kind of who you're talking about, Daniel. It's, you know, it's the people who, like, Um, believe it or not, it's the people who would say you could pry it from my cold dead hands. I could not live without this. You think that must be like an L3 or whatever.

No, no, actually, it's just people using it, but we can look at that and we don't really understand the value there. I mean, they, they don't wanna lose it, but they're just doing their email faster, or they're interpreting technical documentation. But the the company can't look at that and say, there's $4 million we, you know, saved or whatever. It's just so. So we say that's noise, too. That's gonna happen, it's gonna take care of itself. I wouldn't focus too much on that when we talk about L2s.

So at tier one, we have a we have a process for assessing and identifying quality L2 candidates. Now, Daniel, I'm gonna work my way backwards to your actual question from here. So the idea is with an L2, this is a person who you might think, oh, great, how do you assess them? And so they have some aptitude for AI. No, we really don't even kind of start with any sort of technical aptitude assessment. What we care most about is company DNA.

Do they understand the work in an uncanny way to the to the company? Do they know which cell of a spreadsheet to like fight over? Do they like? Do they know how to deliver the work in a way that the company wants it delivered? Is their work style uncanny to what we want? Like? That is is absolutely necessary before we screen anyone for L2 training candidacy. And here's the reason why. Right now, the wrong people are in the driver's seat.

It's the we call them the AI excited. The AI excited are the ones who are getting all the attention in the company. They're running down the hall, screaming words like nano banana and methos class models. And everyone's just like, Well, they must be the g- they must be the person, you know, who should be building things. When, in reality, what we see, Daniel, is these people who, maybe by definition, are often distractible, um, and not very focused on the work.

Um, uh, don't know exactly how to align an agent to work in a way that is absolutely uncanny. To what the the smartest and best and most qualified SMEs inside of that function would call good or accurate, or what we want done. And so we say we wanna start with. It doesn't need to be the SME necessarily, but it needs to be someone who who, in the Venn diagram, like, can replicate the work the company needs done in the way they do it.

That honors the brand and has the aptitude to learn these models, the interest, and though, Daniel, we only want one of those on each team. We find that it is not you. It is not a one plus one equals two thing. Uh, too many chefs in the kitchen. If you have two L2s on a team, they are not necessarily as good as just an L2, a few L1s, some L0s. And we wanna make sure every L2, every non-technical builder, has an L3 within arm's reach so that they can double check the work.

The L2 can make it uncanny, but the L3 can make it scalable, durable, make a thing that won't break, um, or doesn't break laws, or, you know, violate, you know, governance policies. And Daniel, this is like your whole world, right? Um, but so this is kind of what we tell. And the other nice thing about thinking about it this way is if you can see, like, well, we really only want one L2 inside of each, you know, like, high performing team.

That also means we don't have to think about as many expensive licenses, and we can focus training on pre-fol- pre-qualified candidates. Um, so if we're just letting the L1s happen naturally, we're saying, okay, L0s will deal with them someday, but what can we learn from them now? And and L2s are really the ones building the machines that we want to work in a way that honors, honors the work and think about it. How many tools have you used?

It's like, who built this? where'd this come from? Like, you know, these things tend to last for years or decades, and it better work, right, you know? And it was par particularly when you're building something that works at scale.

If you're listening to the Practical AI podcast, I'm guessing that you value practicality, not just the hype around AI, which is why I think you should check out the Midwest AI Summit. This is an amazing event. I'm gonna be there this year. It's happening October 15th in Indianapolis. This is an event like no other I've been to. There's actually an AI engineering lounge where, for free, you can go up and get expert advice from practitioners and get feedback on your architecture, your design, your agentic harness.

Whatever you're looking at, you can get feedback on in real time, in between amazing speakers that are on the main stage. So don't miss this event again, October 15th in Indianapolis. And you can use the code Practical AI 20 for 20 off. So go to Midwest AI Summit dot com and grab your ticket today. Use code Practical AI 20 for 20 off. Midwest AI Summit dot com.

So Mike, you got me pretty interested in. I'm really thinking about, you know, this, this kind of L2 process that you're describing. And and I have sort of a I want to rephrase a little bit in my own words and kind of finish with a a question from there. And that is. It feels from what you're discuss, what you're describing there, that that L2 kind of has the knowledge of the, you know, the value that the company is producing and locked in their heads.

And so they may not be, you know, the L1 that's running down the hallway screaming, Hey, AI is cool, we should do it, uh. But they're the ones that are fundamentally and historically bringing that core value into the products and services the company's trying to produce. And that, a and that's kinda how it sounded to me, and they, they, wh- whether or not they're into AI, they they have that. So it seems like you're trying to get kind of the L2s to be able to best use these capabilities to enhance their ability to drive value creation in the company.

Is that a good way to interpret that? Um, and and and you know, depending on what your answer is, can you kind of give me a course correction, or can you kind of, eh, you know, kinda go down that path and and explain it more? Because I'm I am pretty keen on that idea.

Yeah, great, so you, you know, an L2 is a is a good one. When the smees in the area and don't complain about whatever the AI tool they created, you know, like, whatever its output is, you know, um, you know that, you know that they are a good L2 candidate. When, um, uh, when they can align an agent. So that, you know, when it creates outputs, it feels familiar, um, to to the type of work we do and doesn't require, you know, a whole bunch of, uh, babysitting and and hand holding.

Um, but also to to to back up a little bit, if you, if you think about what that actually means to an organization. Because, and and in a minute, I'll give you an example of a recent, a project I'm on right now. Um, where that I think would help you see the value of this instantly. Um, a good L2 is also not only building tools that, you know, can, y- uh, sort of address, like, just repeatable work at scale.

They're also converting tacit knowledge to documented process. Because once, once an AI model is aligned, uh, now you have documentation. Maybe it's in code, but it is, it is documented. And so that's a big problem in, like, every industry right now. It's like, oh, you know, the aging workforce, and you know, what are we gonna do? We got don't get hit by a bus guy over here. And if, you know, if that's me, if we lose them, you know, the plant shuts down, you know, kinda kinda thing.

And um, so, um, I think in in my mind, like, that's how, you know, you have an L2 there, that you don't have to worry about the tool they're building and whether or not it's it's gonna work well. So, but here's the other piece of it. And and really, like we executives, lean forward in their chair at this part of the conversation. Because when, when we say, um, you know, if if you're wondering, you know about the the value of this investment, um, let's let's stop thinking about this as AI work.

And and really, you know, like, the work of yesterday is going to look like the work of tomorrow, believe it or not, the more things change, the more they stay the same. And I know it feels like everything is shifting beneath our feet, but the reality is we will be doing the same things before tomorrow. Whenever I talk to AI teams, it feels like what they want to build is agents, what they want to build is tools.

But if you actually go look inside of the work at companies, they don't. They're not thinking about AI, they're thinking about this particular problem in front of them. So with one of my clients, uh, let's just say one of the top four, um, pharma companies in the world. They were sitting on a stack of, uh, I can't remember how many thousand documents they needed converted to look from to look like this, to look like this. And um, I, I can't probably can't say too much more about that project.

But, you know, it was a $4 million job, it was a $4 million job. Uh, they they knew exactly how much it would cost to convert each one, and it's requires smees. And an L2 took a look at it and said, This feels like a Claude skill. Had the Claude skill built within 3 h. Um, we drug one of their documents onto that skill, you know, out the other end squirted. The almost exactly what we were hoping these things would look like at the end of the process, you know?

Of course, everyone's jaw hit their desk, like, Wait a minute, you're telling me that was gonna be $4 million? We? It needs to be done, and now our Claude, which we've already paid for, you know, you just drag them on and and wait. You can also run a thousand concurrently, and like, this whole thing can be done in hours, like, wait, what? Um, that is measurable, you know? And so we, we kinda push people to thinking toward, like, you know, you, you're not gonna find out the value of this investment based on how many tokens people are spending.

Or, you know, like, you're gonna find it. When you start intentionally forming teams with a couple of strategic people in there who know how to spot an AI opportunity and just make the headache vanish. We call it doing their laundry, you know? And so, like, this is where the real money is, and this is where the real savings is. And Daniel, I know you know about this, cause I've heard you kinda almost complain about it. Like, you know, it's like, this stuff's not fun, um, but it's like, it's real work.

Uh, Chris, is that helpful?

It is no that that helps frame it very well for me. I appreciate that.

y- yeah, and I, I would be curious of your take on this, Mike. I think you're the right person to ask for a critique on some examples that I've I've been using, even personally, but I've been, I've been trying to use this example of, you know how? In like, leadership retreats, you? They always used to show like the the F1 pit stop and how it advanced from like 20 seconds to t 2 seconds, right? And how that happened was like, everybody knew knows their job in the pit lane, right?

And one guy's job is just to move the tire from here to there, and that's all they do, like, that's their full responsibility. And it almost seems like in some of these use cases that I'm seeing, at least th- those kind of very targeted jobs o- or outcomes, right? But like, you're talking about this document from here to there, right? That that needs to be done, and that is is maybe a good candidate for, um, however you frame it, a, uh, use of AI a, an, an agent to take care of it.

How, however, you word that, like, that's a that's a thing. And and it kind of, in my mind, it then it, it doesn't remove the human, but y- it. It actually kind of elevates, in some cases, the dignity of the human. From being the person. Is like, all you're gonna do every day is move the tire from here to there, or to, you know, do this task. And certainly, I recognize there is legitimate, you know, jobs will shift right, but um, a, and you know, people will will struggle a around that.

But it seems like now kind of the human, y- you're not framing your team of humans as, hey, don't go, don't get out of your lane, just move the tire from here to there. To actually being able to leverage AI in these individual tasks. And you kind of coming into the orchestrator mode or the team principal or the the strategist mode. And actually thinking about outcomes. Well, what? What is the outcome I want here, and what are the individual things associated with it?

Many of those which can be individual tasks that that AI accomplishes. I don't know if that that rings true at all. I've I've been trying to think about, um, yeah, how, uh, pe- people are very used to thinking now. I I think, or many people are very used to thinking of the one to one interaction between a human and a AI tool, let's say, or a chat interface. And maybe less familiar with this, uh, or or familiar with this way of thinking of.

Maybe one human is actually aided by multiple different instantiations or manifestations of AI that help them work towards towards outcomes. But I don't know if any critique on that rambling example of kind of the the the F1 pit stop and some of these tasks. Um, you know, what are the tasks in your in your, uh, pit box that need to be accomplished, and what? What of those are good candidates for AI? How do I think about managing the set of AI workers or tasks, or whatever you, whatever, however you might frame it?

Yeah, one of the one of the things that popped in my head while I was listening to you, Daniel, which I always enjoy doing, is, um, yeah, I'm very often thrown into rooms with people who, you know, live on a spectrum of, you know, attitudes. And, um, you know, one of the big ones is just like all the fear wrapped up in this. And um, I, I kinda always open with, Hey, guess what? If you're worried about losing your job to, I think there's thunder in the background.

I don't know if you can hear Yeah, it's that. definitely it's Yeah, It's, yeah, thunderstormy in yeah, it's, the Midwest it's giving,

right now, for sure.

it's giving the answer extra drama right there.

yeah, that's the fear. No, really, you know, I? When I go in a room and I know there are people who who are dwelling on that, hung up on that. I I mean, you know, I, I just open with, Guess what, you are losing your job, you like, your job is going to change. And that's true for all of us, always, whether or not AI was invented or not. Like, I don't know how many careers I've had, and I think the average person has seven.

And even if you stuck with one, your job would change year over year in time. And if if you're, if someone is uncomfortable with the concept of change, guess what, you're not cut out with the for the workforce. I mean, like, the reality is, um, the reality is like, if if my daughter was scared, there's a monster under her bed, I wouldn't go and say there's no monster under your bed. I'd say, you're stronger than a monster, like, and monsters aren't real, you know, like, um, you know, and and you know, it's kinda like that in this job.

It's like, hey, um, uh, adapt or die, and I don't think you're gonna die. And and and maybe some of this, it might sound a little heartless, but it comes out of the research is that, um, disruption is always it. It turns out, in hindsight, it was not as hard, um, to adapt to as you thought it was going to be, and you adapt more quickly than you thought you could. And so this is just, you know, documented, um, truth, and so, but it, it never feels that way.

Um, I think I'm just the weird bird who, uh, really likes the idea of, like, uprooting my entire career and just trying something new. You know, like, I just ca- I just have done that so many times and I almost enjoy it. Um, so, but then I think the rest of what I heard you talk about, Daniel, Yeah, I addressed that. But like, remind me, like, what else? What Y- was the the other Y- yeah, I think part?

it's this idea of, uh, one element of the change, I think, is is understanding that maybe your your position, which might be consumed by this task right now, might not be just come into work and do this task. But it, uh, it, it sort of elevates more to an outcome kind of creative orchestration mode, where you're actually orchestrating work that needs to be accomplished. Like you say, the same functions, the same roles, like the same outcomes need to be accomplished in your, in your business, right?

But um, but yeah, it, it's. It does take a different level of thinking to think about, well, how do I orchestrate these things to get the outcomes versus how do I do this task? Cause I know how to do the t I, I know how to get the task done right.

Yeah, yeah, I hear you. Um, maybe maybe. Part of why I struggle with just just rapid firing an answer to this is my mind darts back and forth from the hundreds of like work environments and situations I've had to kind of inspect and solution for. Um,

Let me let me frame, literally, oh wow, let me frame

yeah, go.

it for a moment, uh, cause I, I, I think I see where Dan's going, and and I think you are the right person to answer this. And that would be, you know, if you go back to that analogy of the F1 pit stop that he was describing and the nature of the jobs that everyone had there. And you're kinda looking forward, and so, you know, we're, we're kind of. What is our expectation of the future? And, you know, at least I think I think one of the things that probably the three of us would agree on at some level is the fact that the human that's in that process.

It may m, uh, may have these AI agents that are able to take over some of those very specific jobs along the way. And you're almost becoming the pit stop manager or orchestrator overall, so you're still doing the pit stop. Because that's the value of what your organization's doing. But the nature of the humans may change to better. Do it, to do it faster and to grow, rather than be kicked out of the, you know, it, rather than loss of job, it's a change of job that's actually more valuable in a lot of ways.

Could you talk a little bit about what that future might look like for those individual employees that are adjusting their pit stop job? It may be, you know, maybe moving a little higher in the abstraction layer to owning that whole process, where they are valuable as humans, and yet they're taking full advantage of of the agents. Any thoughts, like, as you take companies through the process, how, how you would do that and what your expectation of the future will be?

Um, on how that progresses,

I don't. I don't think you're gonna like my answer. it's all good. Yeah, I've decided to stop worrying about it. I don't even think about it anymore. Like, and maybe maybe it's just cause I don't fully understand the question. I think I do I. I spent probably two years inside of all of these organizations, trying to, like, lead and organize, and, you know, um, and implement, uh, enablement and activation campaigns. And in the end, it just kind of felt like I was just watching water run down a river.

It was gonna go down the direction it was gonna go. The river is is just winding the way it winds. And and and I think I'm always just searching for, like, where can I actually make an impact? Where can I actually activate change in an organization that is measurable and and something that we're all glad we did after we're done doing it. And you know, when I come in, and, you know, and we try to just, you know, really analyze and think through, you know, like, well, what's the impact of this gonna be after we do that?

And you know, what's it? In the end, like, what, what it's gonna be is what it's gonna be. And people tend to adapt to these tools very quickly once they've used them once or twice and seen the result. And so in my mind, it's kinda like, I'm gonna do everything I can to get them to do that. But beyond that, if I can't get them to, I don't care. What I care about is, can I find that L2?

Can we get them equipped and activated and it and and synergizing? That's a better word than that. Um, effectively, uh, cooperating and contributing inside of their team, uh, in a way that, like, I wanna know the difference before and after. And and so it's not just that, you know, L maybe the pit C- Crew thing was a distraction. Comi- for me, cause I was, I was focused on, like, well, it's faster now, and it drives me nuts when people say, I want things faster.

At tier one, we say, like, faster's not always better, especially when y- it's a, it's. It's a system with a whole bunch of parts, and you've just created whiplash and angst for everything around that fast piece. You know, like, nothing else is ready to go that fast, so that's not necessarily better, cheaper, sometimes that doesn't even matter, believe it or not, a lot of the companies, that's that's not like the main thing we're talking about, like, but new emergent types of work like that is interesting.

So what can we do now? That human minds just really couldn't do well before? And and equipping these language models inside of, like, really clever harnesses. So, again, it's kind of like Daniel's world, you know, like, whoa, uh, new kinds of work, um, and so what's that mean for, you know, the product, or this, you know, department or function, or whatever it is? I mean, um, so, um, so, I guess I put that that whole concept just right outside the periphery of what I can control, and so therefore not something I worry about.

And I'm sort of done thinking about,

yeah, that that makes sense. I actually do like that answer, I did, I did Mike, cause, uh, too, but I I think that was there there there are a lot of people and may maybe this is, um, I don't know a a little bit too personal. But certainly, like a mode that I think all of us as humans get in is not being present in the moment and thinking about, of course, the past, you know, dwelling on that, but also the future.

And the all the stories that we tell in our head about how things might go right, which won't there. There may be elements of that that are true, and it's not. O- Of course, I don't mean that we shouldn't innovate and look at new ways of doing things, I just mean we often dwell on eventualities in the future that won't, that will never take, take place. And there's dynamics that are happening that we need to dig into and think about how they matter in in the moment, which I, I, I really, I really like that, that perspective.

As as we close out here, Mike, um, after kind of working across these different levels of proficiency. And I know also you're thinking about structured ways of thinking about the the actual things that can be done by AI, the outcomes, that sort of thing. What, what's on your mind, you know, today, as you're going into your next meetings? As like, the the the kinda challenges or the things, maybe like you had done this work to parse through all of these, this research and, and, um, work on AI proficiencies, right?

Um, what's that next kind of, or is there a next kind of area in your mind where you're like, Eh, I don't? I don't totally have a grip around this bit of it yet, but I'm really curious to dig in more and see if I can see if I can parse through some of it. A anything that comes to your mind as as we close out here.

Yeah, I think I think the thing that's been most exciting to me is like an idea, or like a rallying cry. When it comes to the rank and file inside of organizations, or with leaders who look at me and say, like, what's, you know? Like, where is their energy? where is their momentum? Like, what, you know? Um, one. One idea that I've kinda had lately and I've seen pay off over and over and over is the reality inside of of most of these massive organizations.

It's mostly I'm talking about big business now. Um, is that everyone's kind of got a toolkit, uh, in place, like most? Uh, if you're, you know, uh, Eddie punch clock Procter Gamble. You probably have access to 1, 2 or 3 AI tools. And they don't work well together, and they don't necessarily work well. And it is very easy to try to use them and just see it doesn't work, you know, super well for the thing that you need to get done and give up on it.

And I think, like, the thing I'm realizing is that there is massive, massive value in I in the people who are willing to lean against the brick wall and push and push and push. And you. What, believe it or not, you'll start to find ways to use these tools that are are hugely transformative, formative, in spite of the fact that they're so burdened by necessary governance. And, you know, rules and restrictions. And why can't we turn that connector on, and why can't you know?

But um, you know, when Claude tells you something's not possible, you know, ask it to get creative and, you know, and e- or, um, you know, think through, you know, alternate approaches. I think creative solutioning is gonna be really, really valuable over the next, uh, 2 years, 3, 4, 5 years. Before, you know, all the wrinkles are ironed out. And, who knows, you know, terminators or whatever, whatever Comes after that. Um, but um, I think the yeah, the thing, Daniel, that, like, I've been preaching lately, you know, with with like my trusted colleagues and, um, with different leaders in organizations is like, Um, do your best to to encourage, um, uh, that type of thinking and behavior of like, okay,

I know they aren't perfect. Figure out what you can get done though. Um, so.

Yeah, that's that's I think a great rallying, rallying cry to to end with. Appreciate you joining us again, Mike. Um, we look forward to having you, having you on again, uh, to to learn from you in, ho- hopefully not too long again. Um, but yeah, appreciate you taking time. It's been a great conversation.

Thank you.