21 Oct 2025 · 52m · Michael Krigsman, Tim Crawford · 19 quoted facts
Tim Crawford argues that AI is turning the CIO from a technology manager into a business leader. Efficiency gains from tools like Copilot are a useful place to start, but the real value comes from innovation projects that do things a company could not do before. AI projects fail mainly because they are run as tech experiments with no business outcome attached.
The central tension is speed versus substance. Companies are in an AI bubble of hype, yet are also adopting too slowly because they will not rethink their processes. Crawford says leaders must understand how their business makes money, fix processes before automating them, train people, show results within weeks or months, and kill failing pilots quickly. They should also keep expertise in-house instead of handing it to outside vendors.
Crawford says many failed AI projects were run as technology projects with no business goal. Without a defined outcome, a team cannot pick the right metrics or decide quickly whether to stop.
Unlike past technology that was simply dropped in, AI changes how people work. Crawford says that without training, even simple tools like Copilot struggle to gain real use.
Layering AI on top of existing processes creates a mess quickly. It also removes the human who would notice outliers and say something looks wrong.
To prove ROI for efficiency tools, Crawford advises measuring real time savings with a small group, such as ten people, and then extrapolating. Innovation projects are easier to judge because they tie to specific dollar outcomes like a new product or market.
Outside partners can fill routine work and help train staff. Crawford calls outsourcing your organization's learning and knowledge a critical strategic mistake that can leave a company dependent on a vendor.
Crawford says most are run as technology experiments rather than business projects, with no clear outcome or metrics, and without the training people need to change how they work. He also says teams are too slow to stop experiments that are not working.
It is the state of running many small AI experiments that never scale into something that matters to the business. Crawford cites an IBM CEO study in which only 25% of AI projects were proving their return on investment and only 16% were going to scale.
Efficiency projects help people do existing work faster, such as using Copilot. Innovation projects let a company do things it could not do before, like launching a new service or reaching a new market, and Crawford says that is where the real money is.
Crawford says no. He argues that a company asking for a chief AI or chief digital officer probably doesn't have the right CIO, and that a VP of AI or VP of data reporting to a strategic CIO is more appropriate.
Crawford says no single person is. He describes organizations forming a cross-functional AI council, led by the CIO or sometimes the CEO, that collectively handles prioritization, guardrails and costs.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
According to Tim Crawford's description of the IBM CEO study, only 25% of AI projects were believed by CEOs to be proving return on investment and only 16% were expected to scale.
“only 25% of the of AI projects, according to those CEOs in the study, um, they believe are proving out the ROI, and only 16% are actually going to scale.”
A recent MIT study on AI projects was widely quoted because of its dramatic failure figures, though Tim Crawford advises taking it with a grain of salt.
“you saw the MIT AI study, which a lot of folks want to quote because the numbers are so dramatic.”
Michael Krigsman said the MIT study concerned proofs of concept failing.
“yes, the that MIT study, uh, is about POCs failing.”
Apptio, which belongs to IBM, along with the TBM Council and the FinOps Foundation, is working on understanding what AI and cloud efforts actually cost.
“companies like Apptio, um, from IBM and the TBM Council, they're trying to get their arms around this as much as even just the FinOps Foundation.”
Tim Crawford says IT began by focusing on business outcomes in the 1960s and later lost touch with them, creating a gap between IT and the business.
“I have a whole talk track on the anthropology of IT and how we started by focusing on business outcomes in the sixties, um, and we kinda lost track of that.”
Tim Crawford pointed to a separate MIT study about 'meek models' that looks at how smaller models compare with big ones.
“there's an interesting MIT study, different than the MIT AI study that talks about meek models, and so just probably just look for MIT meek models”
Tim Crawford said Amazon was asked not to bring more workloads into Ireland because of energy concerns.
“Amazon got asked not to not to, uh, bring more workloads into Ireland or something along those lines.”
Tim Crawford says communities hosting data centers are seeing their power costs rise.
“Communities that have data centers, they're seeing power costs go up.”
Tim Crawford observed that OpenAI, Oracle and Meta are pushing toward ever larger AI models, and he expects enterprises to shift toward smaller, more specific models.
“it's kind of crazy when you look at the whole OpenAI, Oracle, Meta, and they're just driving toward bigger, bigger, bigger.”
Tim Crawford cited a study saying six out of ten executives globally are expected to change roles within three years.
“six out of ten executives across the spectrum, so globally across the spectrum, are expected to change roles within the next three years.”
Tim Crawford told a group of executives that today is the slowest their company will ever operate, because pace of change will keep increasing.
“today is the slowest your company will ever operate. Think about that for a minute.”
Tim Crawford says that as an IT leader, his first relationships in a new organization were with the chief legal counsel and the head of audit.
“some of the first relationships that I built when I walked into a new organization were with my, um, chief legal counsel and head of audit.”
Tim Crawford described an organization where half the staff was outsourced and much of its innovation was being done by outsourced workers, which he called expensive and detrimental.
“I ran into one organization where half the organization was outsourced.”
Tim Crawford argues that a company seeking a chief AI officer or chief digital officer probably does not have the right CIO, and favors a VP of AI or VP of data instead.
“if you are looking at a chief AI officer or chief digital officer, CDO or CAI, oh, you probably don't have the right CIO.”
Tim Crawford says some organizations form cross-functional AI councils, led by the CIO or sometimes the CEO, to jointly handle prioritization, guardrails and costs.
“some organizations are putting together an AI council that can be driven maybe by the CIO, sometimes by the CEO of the company.”
Tim Crawford argues that people and CIOs who use AI will outperform those who do not, rather than AI itself directly replacing workers.
“AI is not directly going to take your job, but AI could take your job.”
Tim Crawford said he had just published a blog post on his blog about the CIO role moving from traditional to transformational.
“I just published, um, a blog post, um, talking a little bit about this last night on my blog on Avova dot com.”
Michael Krigsman announced that a future CXOTalk episode would feature two members of the UK House of Lords discussing big tech and AI policy.
“Two weeks from today, we have two members of the House of Lords in the UK talking about big tech and policy around AI and big tech.”
Tim Crawford argues AI efforts should begin with business analysis rather than conventional systems analysis, to avoid simply refactoring existing systems for AI.
“too many people would go, Oh, okay. So I just apply my normal system process, take this the way the system was built, understand how the system was built, and refactor it for AI.”
Crawford explains that AI changes how CIOs work, make decisions and engage the board and business. Those who use it well will pull ahead of those who do not.
“AI is playing a pretty significant role from a number of different dimensions for the CIO”
Efficiency gains are a starting point, but CIOs must keep their eye on transformational outcomes. The technology matters less than what it does for the business.
“it's not about the tech, it's what you do with the tech to address those business outcomes”
Citing IBM and MIT studies, Crawford says pilots fail without business alignment, quick go or no-go decisions, and training.
“only 25% of the of AI projects, according to those CEOs in the study”
Crawford says AI is both a hype bubble and under-adopted, and explains how to measure actual time saved and track costs.
“you can't start with assumptions”
Crawford argues that AI changes how companies operate, so IT leaders must know how the business makes money and serves customers.
“With AI, you have to start understanding this.”
Governance of data and models is unsolved even for large vendors, and AI's energy and water demands are returning as a concern.
“there is no way to remove that from the hive”
Layering AI on broken processes is dangerous, and politics should not keep failing pilots alive. Results should show in weeks or months.
“you're taking the human out of the loop, and we underestimate the value of the human”
Crawford says to use partners to train staff, not to hold company knowledge, and argues that AI ownership belongs to a council rather than a Chief AI Officer.
“you probably don't have the right CIO”
Many companies are stuck in AI pilot purgatory, endless experiments, but nothing scales, nothing gets done. Today on CXOTalk number 898, Tim Crawford, a top CIO strategic advisor, reveals how CIOs and IT can break through to real ROI. I'm your host, Michael Cragsman. Let's get into it.
A lot of the work I do is advisory with CIOs, but then I do a fair amount of advisory as an industry analyst with vendors as well.
Tim, you spend so much time with CIOs. Wh- when you look at AI, what is the impact of AI on the CIO role today?
It's massive from a number of different, um, lenses, number of different directions. When you look at the role itself and what the CIO needs to think about, there are impacts to how they work, how they operate, how they conduct themselves, uh, how they inform themselves, uh, and then also outwardly, how they engage with others within their organization. Whether that be the enterprise leadership team, whether that be the board, whether that be customers, as well as their own organization.
And so AI is playing a pretty significant role from a number of different dimensions for the CIO, both today. But then as we go forward, in terms of giving them the insights that they're gonna need to be successful.
What does this mean on a daily basis to the life of a CIO and to IT?
One of the things that that is coming up again and again is how do we get to the insights to make really good business decisions? Um, and so there was a lot of manual labor, a lot of manual work that went behind those, um, those decisions, whether it be through spreadsheets and documents, conversations, um, kind of just, uh, experience, even coming into the mix. And so today, you can automate and bring a number of those insights based on core data to the surface that you never had access to.
And so what that means is you can now make decisions faster and more accurately, uh, in terms of where you go and what you do. And that impacts both the organization, the IT organization, whether it's from a cyber perspective and understanding what threats are coming your way or or how to prioritize those. If it's from a service perspective, being able to do things like ticket deflection, and we see that even in the contact center, too, uh, being able to use AI to address some of these core issues that we had no other choice but to put people in front of.
And so, from a day-to-day perspective, uh, the CIO can leverage AI from a number of different dimensions, and they really should, because this is really one of the attributes that's going to differentiate those CIOs that succeed into the future and those that really get held back.
Are we talking about driving efficiency, saving money, or are we talking also about innovation opportunities for the CIO to forge a a new or really improved relationship with the business partners? Now, let's take a moment to learn about Emeritus, which is making CXOTalk possible. If you're a business leader navigating change or driving growth, explore executive education programs with Emeritus, a global leader. They offer programs developed in collaboration with top universities designed for decision makers like you.
There's a program tailored to your goals, whether that's AI and digital transformation or strategy and leadership. Find your program at www dot emeritus dot org.
At the end of the day, it's allowing us to do things as IT leaders that we couldn't do before. We just didn't have the means in which to do before. Um, but before we get ahead of ourselves there, we can definitely use it from an efficiency standpoint to help understand what are some ways that we can leverage AI to, um, to do things quicker, to use the multiples that come from using AI tools. Um, and this is where the co-pilots and even the chat GPTs come into play.
Some of the, um, not chatbots, but when you get into agents and agentic, there are some ways that that you can use that from an efficiency standpoint to kinda catapult the process and automate some of your day-to-day work. Um, but once you get past the efficiency pieces, you really cannot stop there. You have to keep your eye on the ball, which is really one of those big transformational ways that you can bring innovation to the table to transform the business.
Cause at the end of the day, tech is tech, and whether it's red, blue, green, yellow, it doesn't matter, it does not matter. Um, and when I talk to CIOs, they just exemplify that point. It's not about the tech, it's what you do with the tech to address those business outcomes. And that's really where AI can can help kinda catapult and and move you forward.
But what about this notion of AI pilots and your term AI pilot purgatory? What's going? what? Yeah, what's going on with that?
there are a number of studies that are out there. Um, there's the IBM CEO study, which they run every year. Uh, they've been running that for a number of years now, and the latest one kind of touches on success of AI efforts and the perception that CEOs have of how successful those AI efforts have been. And and in that particular study, only 25% of the of AI projects, according to those CEOs in the study, um, they believe are proving out the ROI, and only 16% are actually going to scale.
That's a pretty low number. And then just what in the last month or two, you saw the MIT AI study, which a lot of folks want to quote because the numbers are so dramatic. Uh, says that only five of AI projects are proving out, um, their worth. Y- take the the MIT AI study with a grain of salt. Um, there are some very specific aspects that it focuses on, um, that you need to make sure that you understand when you're looking at those numbers.
Don't just read the headlines on it. But I think, you know, kinda back to your question, Michael, it's it's really a question of do you understand what it is you're trying to accomplish with AI, or are you just doing some sort of of random experiment? And to date, a lot of folks are doing just random experiments as a means to learn, which is good. You know, we have to learn about new technology and new innovation. This is new for all of us, um, but they ha- they're not making that transition from understanding how to turn it from an experiment into something that is meaningful for the business.
And I say the business very specifically because this cannot be just another technology project. It has to have some business outcome. And when you are focused on those business outcomes and understanding the outcome that this p- this project or this experiment could potentially have, then that helps you understand very clearly what are the metrics you should be using to measure the success or failure of that effort. And then kinda working backwards from there, you need to quickly understand is this gonna prove out your hypothesis or not?
And if not, dump it and move on. And this is where I think people get caught up is couple of things. Number one, they're not making that that determination quickly enough. Number two, they're not thinking about what the outcome is that they're focused on, and they're not aligning it from a business standpoint, they're looking at it from a technology standpoint. And then number three is they're not considering that AI and the effectiveness of using AI requires another component that we haven't necessarily had to do in the past.
In the past, when we brought innovation to bear, we just kinda dropped it in and people started to adopt it and and be able to to use it and leverage it. AI is very different. We're we're very quickly learning that without the training to help us understand how this changes how we do work, there's a very high likelihood that that AI effort is going to fail. And we've even seen this with relatively simple things like Copilot, where when you start to drop it in, it changes how people work.
And if we don't help them along that journey through training and other means, um, it's really gonna struggle.
I wanna remind everybody that you can ask your questions. If you're watching on Twitter X, use the hashtag CXOTalk. If you're watching on LinkedIn, just pop your question into the LinkedIn chat. And we love your questions. You guys are so smart, so ask your questions, take advantage of this opportunity. So, David Morales Weaver makes the comment that he scaled a pilot by proving ROI, not chasing features, and it saved months. So that's that's a that's a really interesting point right there.
Absolutely, but again, he's probably looking at the ROI from the perspective of what is this really gonna do for my business or for that that effort. This isn't a this isn't about technology, this you cannot look at this as tech for tech or tech for tech's sake. You have to look at this from the context of what is the return on investment, and what that means is what's the business value that this change or this technology is gonna bring.
And you have to be able to demonstrate that very clearly, and I don't mean demonstrating it with a whole bunch of of dots that you have to connect. You have to be able to connect those in pretty short order.
Now, let's quickly hear from Emeritus, which is making CXOTalk possible. If you're a business leader navigating change or driving growth, explore executive education programs with Emeritus, a global leader. They offer programs developed in collaboration with top universities designed for decision makers like you. There's a program tailored to your goals, whether that's AI and digital transformation or strategy and leadership. Find your program at www dot emeritus dot org. Our mutual friend Isaac Sokolik, he says this The elephant in the room is it an AI bubble or are CIOs moving too slowly?
We are definitely in a bubble, but I also think that we are also moving too slowly in the way that we are adopting the innovation. So what does that mean? So, we're in a bubble in the sense that the AI conversation is incredibly frothy, right? Everything's now with AI, my toaster has AI in it now, um, but is that really gonna move the needle for our business? No, do we really care? Not really. Um, but it's a mar- it's the marketing buzz, and so you kinda have to parse through that.
When you get down to kind of the the, you know, core building blocks of what this could do, and you start looking at the opportunities for efficiency, are we really kind of embracing those? No, and here's the reason why. And I know Isaac really well, and I know Isaac's probably rolling his eyes, going, Yep, yep, yep. And that is that we're not rethinking how we have to change our business and how we operate. And this is something that has to happen within the organization as much as with the CIO.
You know, Isaac talks a lot about what it means to be agile within the organization. That has to play out with AI. We have to think very differently about how we work, how we operate, our processes, our workflows. You can't just take AI and apply it to the way you've always done things. You have to simplify, understand how you can optimize, and then figure out how AI can help catapult that. And that's very different. That is very different, but love the question.
I totally agree, and you run the CIO Think Tank, which is, uh, so you're you're actually walking the walk. But but let's go to Craig Brown's question. He says this Once the gen AI pilots have started, how do you monitor processes and people to justify the cost of the j- gen AI tools and show the value? How do you show the value, and what metrics are key to monitoring, and how do you compare it against current operations?
I mean, it's basically what he's saying is how do you justify the cost, show the value, and how do you compare it and measure it?
Depending on the specifics of the type of work you're doing, the answer's going to change a little bit. But you have to you have to go back to let me use some examples. Um, if let's say you're using, uh, I mentioned Copilot earlier, how much efficiency is that really kind of driving into the organization? And what that means is you have to spend the time to understand how much time is actually being saved. You can't assume that you a and you you are gonna make some assumptions along this process, but you can't start with assumptions.
You have to start to understand actuals, like how much time of the ten people in in a given department, how much time are they really s are spending? And you have to spend the time with each of them, and then you can extrapolate from there. But it requires more footwork to truly understand that, and then you can extrapolate to your organization of twenty thousand one hundred thousand two hundred and fifty thousand people. Um, when you're starting to build out discrete, non-efficiency types of projects, but rather innovative projects, those have very discrete outcomes.
Because now you're talking about doing things you couldn't do before, and in those cases, you typically have some dollar value that's tied to the outcome. Maybe it's a a new product you can come out with, maybe it's a a new service line or a new market you're going after, and so there are very clear metrics that you can track as part of that. In addition to that, on the back end, companies like Apptio, um, from IBM and the TBM Council, they're trying to get their arms around this as much as even just the FinOps Foundation.
Um, from a cloud perspective and AI perspective of how do you start to understand what you're actually spending on these efforts to be able to factor into that value equation, cause that's really what you're asking here is what's the value that drives the ROI? And a lot of folks focus on the cost, but cost for AI is not, it's not clear. It can be convoluted, and so there are efforts that are starting out, but we're in the very early days of this too.
And so this is where I'd encourage you to figure out with your partners, whether that's the SIs, the GSIs, but also the vendors, the enterprise vendors that you're working with, work with them and help them understand what it is that you need help with in terms of clarity around this because they're trying to figure it out too.
Let's go over to Twitter to X. And an interesting question from Arslan Khan. He says, Do these AI fail? It's interesting, he says, Do these AI failures? Well, we haven't really been talking about AI failures, except, I guess, pilots not succeeding, I guess we could say, is an AI failure. And yes, the that MIT study, uh, is about POCs failing. Arslan says this Do these AI failures occur because of lack of vision of non-technical executives, or because tech executives just don't know how to convince their peers that AI is a holistic solution?
Both are true. There is an understanding around what is the value of the project, what is it the value of the experiment, and you have to tie that to a business outcome. And one of the problems that that I often see is that, and this kind of goes back to something, um, Michael, that we were talking about kind of at the top of the show. But when you start talking about the role of the CIO and how the role is evolving, and I'm sure many folks haven't had a chance to read it, but I just published, um, a blog post, um, talking a little bit about this last night on my blog on Avova dot com.
But as the CIO role transforms from traditional to transformational, and what that means is from a technology focus to a business focus, what that does is it gets the IT leadership to be thinking more about those business outcomes, not technology outcomes. Many of those failed projects, they're technology projects, they're not business projects. And this is one of the problems is when you're looking at efficiency, it's efficiency of what? And so what is this project really, um, intended to to accomplish, and how do you get to that business outcome?
But first, you have to understand what those business outcomes are, and many folks in IT don't have the level of depth that they need to have around what those business outcomes should be, what their business objectives are. Like, for example, do you know, if you're in IT, are you crystal clear on understanding what your company's objectives are, what the business objectives are that the executive leadership team is oriented around? And if the answer is no, well, okay.
There's step number one is understand what the nature of your business is. Step number two is understand how your business operates. How do you make money? How do you spend money? Step number three is understand your customer. What does your customer do? How do they engage with your company? Why do they engage with your company versus your competition? Now, some in technology might go well. Wait a second. I work in IT. I don't work in marketing or sales, but guess what?
With AI, you have to start understanding this. We've needed to understand this, um, earlier, but with AI, it's it's absolutely necessary.
Why does AI force someone in IT or force a CIO to think more broadly a across the company? Because I was under the impression that CIOs should have been doing this since, oh, basically, forever modern times.
I don't wanna go down a rat hole on this. I have a whole talk track on the anthropology of IT and how we started by focusing on business outcomes in the sixties, um, and we kinda lost track of that. And we started to say, look, you know, we got this problem, we can figure it out solving these business problems. And we started to create this chasm between what IT did and what the business needed, and the business kind of lost touch of what technology was doing behind the scenes.
We have now kind of come full circle where, guess what, that chasm needs to be closed. And the reason for that is because what is AI really doing? It's changing how we operate our business, whether it's from an efficiency standpoint, whether it's from how we engage with our customers, it is changing the way that we operate our business. And the only way you're going to know or understand the impact of that to your business is by understanding your business.
And unfortunately, as you said, many folks in IT don't have more than just a basic understanding of how their company operates. You know, how it spends money, how it makes money, how it engages with customers.
And Simone Jo Moore on LinkedIn really wants to emphasize this point that we need to not focus on quote unquote just efficiency and really think about the effectiveness for work for organizations and people.
Completely agree. The real money, the real opportunity, is gonna be when you start to move into what I call innovation projects. These are things that you could not do before AI. That's different than an efficiency project. Efficiency projects will have value, and they're a great way to learn. But very quickly, you have to figure out how to shift gears and start adopting some of those innovative efforts, some of those things that you couldn't do before. Um, AI, and I completely agree, that's w- that's where the real goal is for companies.
It's the efficiency is important, but the real goal is when you can do things that that will allow you to open doors you couldn't open before. Um, provide insights that you never had access to in the past, allow you to engage with customers in ways that you couldn't before. And and here's the kicker, it starts to differentiate you from your competition. And so this is where the big race is is how do you start to differentiate your company from your competition?
And being able to address customers where they are and where they're going, so looking ahead, not behind, but also looking around the corner.
Rocky Vienna asks about the security risks in what are using, in essence, he says, public LLMs. Where do you see movement in the development of platforms to provide a security envelope around corporate use of AI? It's a very important question.
We absolutely need to be thinking and talking more about not just security, but governance. And this is something that all of you should be hounding on your vendors and providers to be talking about is how they think about governance of data. Um, so there are different aspects. Rocky's asking about, uh, security around the actual models themselves, and there's a, um, there's an interesting MIT study, different than the MIT AI study that talks about meek models, and so just probably just look for MIT meek models, and it's a it's an interesting read, talking about how the changes in big models versus smaller models are gonna play out.
Um, when you start to think about security, you have to think about the data, you have to think about the outcome of how you're using that data and protecting that data. I think this is where, um, there are going to be problems with big models that we haven't solved for. A good example. Here's a great example. I make a public statement that says, You know what, Michael Crichton is awful, he's a terrible person. Mind you, none of this is true.
Oh, it is true. I didn't know it's true, but go on.
no, no, I've known Michael for years, he's a great human being and love working with him. Um, but the problem is when a model picks that up, there is no way to remove that from the hive. There's no way to back that out. So even so, once we go down this path, the problem is you're gonna start ending up with more misinformation and disinformation, and so that's gonna be a problem. But you have to also think about governance around the data you do have access to, and especially proprietary customer data and s even some of those challenges.
When you start to think about, um, user governance models, as well as AI agent models, as well as knowledge models, and then the different governance models that come from the different systems of record, how do you start to to bring all that together to make sense of it? And in my conversations with some of the largest enterprise vendors on the planet, they're still working that out too. So hang on, it's gonna be a rough ride, but we've got some some pretty interesting challenges ahead of us.
And, of course, we're dealing with LLMs that have an insatiable desire and greed to suck up everything in their path. Yeah, well, And you have to be there, careful that the configuration of your interfaces to those LLMs exclude the sucking up of your corporate data.
there's that, too. And so I think this is where you have to think about your data strategy, you have to think about how you're protecting what is important to your organization, and that's probably a totally different conversation. But AI starts to factor into that when you start to think about the governance of what AI and especially as you move into agentic with learning models, reasoning models, discovery, a2amcp. I could go on and on just on that alone around the challenges to govern data through those processes.
Greg Walters has been patiently waiting on LinkedIn, and then we're gonna jump back to Twitter, cause we got questions stacking up over there too. He says, Uh, the AI failures seem to occur when AI is dropped into existing processes instead of the processes adapting, being a dig- digested by AI. But he says, he, he says, a ray of hope that it seems that finally, we're looking to focus on deliverables versus adherence to the process. But I, I will just mention, I've heard this same conversation going back, you know, twenty-five years.
This is not new.
There's a whole nother talk track around just policy development, you know, how did we create some of these business policies and, uh, business processes in the first place? And usually it's because something didn't go right, or we wanted it to work a very specific way based on the parameters at that point in time. Well, guess what? Those parameters have been changing, in some cases, over decades, but the company never changed how they did things. And so one of the things that that you're kind of alluding to and really needs to be thought out, um, for success is how do you start to understand that process?
First, so set AI aside for a minute. How do you understand that process? How do you optimize that process? And then how do you bring AI into the mix? Cause if you just simply layer AI on top of existing processes, you're gonna end up with a mess pretty darn quickly, and especially as you move into an agentic, uh, framework.
Arslan Khan comes back. He says, As a recovering enterprise architect, we should aim for AI, we should aim for AI taking our jobs. Some enterprise architects are okay with that, but some are not. How much of the AI race is just self-preservation?
The problem is this is where the marketing is really kind of settled in. AI is not going to AI is not directly going to take your job, but AI could take your job. And what I mean by that is there's gonna be a difference between those folks that use AI and those folks that that don't leverage AI. And the folks that leverage AI are the ones that are going to succeed and excel, and the ones that don't, and this goes all the way to the CIO, um, are the ones that are going to perish.
It's the same reason why traditional CIOs are in decline and transformational CIOs are are in demand. It's because of what they focus on and how they leverage the tools at their disposal, and also, um, how they can provide greater value over time to their particular role and their particular organization.
Chris Peterson on Twitter makes a couple of interesting points. He says, number one, the recent s. There was a recent study from Anthropic that said only 250 bad documents are sufficient to poison almost any size of model at training time. So getting to your point about the bad data, basically poison data, and Chris asks this, he says that he agrees about the AI bubble and the value conversations that need to happen. How do we factor in the potential cost explosions when venture capital and circular deals go away?
As customers, can we even figure out our sustainability metrics?
I think we'll see more smaller models and leverage, uh, the large LLMs less. Um, it's kind of crazy when you look at the whole OpenAI, Oracle, Meta, and they're just driving toward bigger, bigger, bigger. And this is one of the reasons why, kind of to the earlier question about a bubble, I think there there's definitely a lot of bubble taste to, uh, to what we're seeing happen. At the end of the day, for the enterprise, they need something that is more perfo- performative and also something that's more specific to their industry or or their particular company.
And that's where you get to specific data and data, frankly, you can control. Um, so that's one aspect. The second one, which you bring up around sustainability, this is a real problem. Um, this is a real problem, and I've tweeted about this. I'm sure Chris, you've seen me, uh, talk about this. But we have stopped talking about sustainability, and I think some of this is somewhat politically driven for the time being. But we're gonna come back and look at this in spades, which is how do we start to, um, understand the requirements from of natural resources and the impact that AI is having, whether that be from an energy perspective, right?
Amazon got asked not to not to, uh, bring more workloads into Ireland or something along those lines. We're gonna see more of that, um, whether it be for water reasons or power, um, and driving costs. Communities that have data centers, they're seeing power costs go up. But that conversation is largely kind of subsided for the time being. We're gonna have to come back to that, and I think that's also going to drive our need to get smarter about how we use this technology, too.
A question from LinkedIn from Edward Monroe, who says, How are CIOs approaching the shift from se- SEO, search engine optimization to GEO, generative engine optimization? And are there early priorities that can guide the development of a roadmap from this shift from SEO to GEO?
Number one, the CIO doesn't care. Um, and I don't say that flip, but rather you have to look at what the CIO focuses on, and you're talking about something that is farther into the organization and probably in the CMO's org, specifically around how these technologies are used. Keep in mind, the CIO moving forward is very much focused on business outcomes, not specific technologies. You know, I often say it doesn't matter whether it's blue, green, yellow, red, blue, it doesn't matter who, whose technology, whose badge, whose which technology you're using.
What matters is how is this going to move my business forward? Um, but that particular question, I'd have to punt a little bit on because that's that's outside of the realm of the CIO.
From the CIO perspective, this is an internal set of operations of tasks, and IT, IT can help support that, the infrastructure. But once you get inside marketing, really, is that the the job of the CIO, or is that the job of somebody in marketing who has real deep expertise in that specific process?
The other piece that you're starting to see more of is partnerships between organizations. So CIO and CMO coming to the table together, CIO and CRO, CIO and CFO, CIO and COO, um, CIO and CPO. This is something that came up, um, this morning in our conversation within the think tank was around the the role of the CIO and the chief product officer. Um, and so, uh, you have to keep in mind that the level in which these folks are focused and the things that are most important to them are very much into these very strategic business outcomes that move the needle for the organization and for the customer.
When you get into the the how of it, kind of into the the details, that's when you start working into the organization a little further.
And also, I would assume that's that's when you start working with detailed experts who may live inside IT as well, but who have very narrow domain, uh, knowledge that's deeper on specific topics.
That's right, that's absolutely right.
So we have another question from Twitter, from X, a really good one, going back to the POCs, to the proofs of concept. Elizabeth Shaw says, How do you, and when do you terminate an AI proof of concept, especially because doing so is rife with politics? So unpack that one for us.
The issue there is politics sometimes can govern decisions, and it's it's kind of the necessary evil that you have to work through. If you've ever worked in large enterprise, as I have most of my career, um, you have to have the right balance. But this is also where relationships play a role. Um, you have to think about what's the impact of spending the money and spending the resources, cause it's not just about money, it's about resources that are going toward that effort when they could be applied somewhere else and ensuring that that effort is aligning with those business outcomes.
And I'm gonna sound like a broken record, but business outcomes and business objectives need to rule the day. Now, the politics will play a role in that, and you have to make sure that you have the right balance, healthy balance, and play the job of the politician in it. But you have to understand that they have motivations and incentives as much as you do. And so if you can understand what their incentives and motivations are of why they're driving that and pushing that to continue, um, that might give you some insights as to how you could drive the focus in a different direction.
So you can pull the plug and move on. But on the sum, you have to figure out how to quickly ascertain is this experiment going to succeed or fail? And if it's gonna fail, you need to pull the plug as quickly as possible. If it's gonna succeed, you need to be able to demonstrate success in short order. I had a conversation with someone that said, Oh, well, we're gonna be looking at a twelve-month, uh, run rate before or twelve months out before we can actually show progress.
And I said, Those days are behind us. Like, you have to show progress in very short order, in some cases, weeks and months, not years.
On this topic of succeeding with these AI projects or failing, Isaac Sokolik comes back with a real definition. He says, Failure occurs when AI exposes data or lack of testing to validate results or lack of change leadership.
He's absolutely right. You have to think about that, you have to think about how failure is going to play out. And, you know, to the blog post that I alluded to earlier that I wrote last night, um, there's a sig- there's a significant change coming. Um, there's a study that just came out executive leadership, uh, six out of ten executives across the spectrum, so globally across the spectrum, are expected to change roles within the next three years.
That's going to drive a phenomenal amount of change when you start to kind of boil it down into technology and what folks are going to be looking to do. So, change is in our future. If if you don't already see it today, expect more of it. Um, if you are seeing it today, expect even more of it, and it's gonna come at you faster. And so you have to figure out how can you accelerate your organization and your thinking and your decision making to be able to accommodate that.
I d just recently, uh, was giving a presentation to a group of executives, and one of the statements I made kind of caught a few off guard. And I said, Look, today is the slowest your company will ever operate. Think about that for a minute. Today is the slowest your company will ever operate. We're only gonna get faster. Strap in and hold on.
On the subject of the human relationships that can help accelerate and make these provide the the fabric, if you will, that makes these projects relating to the business in in the strongest possible way. Arslan Khan comes back and says on Twitter, Should tech executives invite non-tech executives to our echo chambers? What incentives are there beyond just cost and efficiency for non-executives? So how do how do we broaden this pie, so to speak?
I often talk about, in my past as an IT leader, some of the first relationships that I built when I walked into a new organization were with my, um, chief legal counsel and head of audit. Now you might go, well, those are kind of odd for a CIO to build relationships with. But here's the thing, you're gonna run across those folks from a legal perspective and from an audit perspective before you know it. And if you are at least building the relationship and setting the groundwork so you can understand, listen and respect where they're coming from, then guess what?
When it's time for you and the other person to sit down and have a conversation because you've been breached or you have a legal issue you have to navigate through, you already understand where they're coming from. You're not trying to figure it out in the heat of battle. And so it's a way to build respect, it's a way to to build those relationships, and I can't emphasize how important those relationships are. When things get tough, it's the relationships you will have to lean on.
It's not gonna be, um, some other factor or technology whiz bang. You have to rely on the relationships. So don't just look at at the value of what you get out of that particular transaction of a relationship. A relationship is something you build over time, you invest in over time. And when you think about it that way, it changes your paradigm in terms of why you invite these folks to your staff meetings or to present to your organization.
On LinkedIn, Simone Jo Moore makes the point that applying AI on top of current processes is no different from automation questions. If your base isn't good, we'll just do the wrong thing faster. And speed does kill until we slow down enough to make the speed sustainable. And Greg Walters comments that this is all classic needs assessment and system analysis stuff.
when you do apply automation, whether it be RPA, RPO, um, whether it be, uh, agentic frameworks, uh, on the other end of the spectrum, when you do, all you're doing is just automating an existing process. There's a danger with that, and this is something that people don't necessarily think about or understand, which is number one, have you taken the time to understand how that process works? No, number two, you're taking the human out of the loop, and we underestimate the value of the human.
So when when you tell a human do a certain task, if there are situations that come up that are outliers to that task, the human has the cognitive capability to be able to say, Hm, that doesn't pass the sniff test. I better ask someone because it's not my process, it's not in my capability. When you automate those processes, all of those checks go away, and so that becomes incredibly dangerous where you start automating bad processes. And so that's something you have to think about and why it's always important understand what you're trying to work with first before you start layering this in.
And frankly, is this the right place to start? I mean, that's a whole nother aspect, which is prioritization. And then, Michael, the second question,
He's saying this is classic, classic stuff. You need system
Oh, that's right.
analysis. Go ahead, yeah.
So the so I s will sort of agree with that. I think you need more business analysis than you need system analysis. And the reason I say that is it all starts with how you operate, how your business functions, how the how the work gets done. Understand that first, and then you can go into some systemis- systems analysis to be able to ascertain, um, how you should adapt that or change it accordingly. When you start thinking about, uh, infusing AI into the mix, but you should s you should definitely start with business.
And the reason why I make that, what may seem overly subtle of a distinction, is because if I were just to agree and say, yep, system process, which you're kind of right on, um, too many people would go, Oh, okay. So I just apply my normal system process, take this the way the system was built, understand how the system was built, and refactor it for AI. And that would be a mistake.
Elizabeth Shaw says, Who should own the AI ROI as an issue? And Arslan Khan says, Why is the O onus always on technology folks to know about the business processes? And I think these are related issues. Who's responsible for AI ROI, and why is it always on IT?
The easy answer for Elizabeth's question is there's no one person that's responsible for it. And this is why some organizations are putting together an AI council that can be driven maybe by the CIO, sometimes by the CEO of the company. But there's someone that leads the council, that's a cross-functional representation of the organization. And they collectively, not individually, collectively are responsible for prioritization and ensuring that, um, guardrails are put in place and prioritization and and costs are are being managed.
Um, and then I think the the second piece to that is in terms of just kinda understanding, how do you evolve that over time, right? And so it's an evolutionary process. We're still learning, um, but you have to be able to accelerate the decision-making process in terms of what you work on, first, second, third, and then evolve that, right? Fail fast. We've heard these terms before, we're just applying them into AI.
Let's just briefly talk about vendors, uh, technology vendors and services vendors. Historically, there was a real problem with services because companies would outsource that work and the expertise would lie outside the company with the third-party vendor, the the professional services vendor. And it created a cycle where IT never developed the expertise, and so they were it was this cycle of dependence. What's your view when it comes to AI now today?
number one, you cannot do it yourself. You can't do this yourself. You will need help. Um,
Wait, wait, wait, sorry. So so just so I'm clearly understanding you. So you're now advocating outsourcing knowledge elsewhere so that you don't have to worry. Is that what you're saying?
no no, that is absolutely not what I'm saying. So
sorry, I misunderstood.
what I'm saying is when when you go down the path of these these larger projects, so moving beyond efficiency and get into the more innovative projects, you won't be able to do them all internally. You will need help with that. But you have to understand where those partners play the most powerful role for your organization. To your point, if you are outsourcing your organization to a third party and you're basically paying for them to build the institutional knowledge and maintain the institutional knowledge and the learning, that's a mistake.
That's a critical strategic mistake. But the way that you can leverage a third party is by using them to backfill some of the more mundane things and help your organization, help your team learn and augment some of that with that outside thinking so that you can help them accelerate their own thinking. But at the end of the day, you want that institutional knowledge inside your organization, not outside. So I'm not advocating that it be outside. What I'm advocating is you need to leverage outside resources to bring that expertise and help train and educate your team so that they can move more quickly through this learning process, right?
They're not reinventing the wheel, but maybe they get a framework and then they can reinvent it for their own needs. But then you're also not building that that center of excellence outside of your organization, you're building that squarely within your company. Um, and that's not always the case. Unfortunately, I ran into one organization where half the organization was outsourced. Um, and the problem with that is a lot of that innovation was being done by those outsourced staff folks.
And yeah, it's it's an expensive venture and really detrimental to your organization, to the company and your customers.
So you're not at all suggesting handing off responsibility for your institutional knowledge to a third party and then letting them run with it. But you're saying prioritize carefully, use them to help jumpstart. One of the commenters, uh, on LinkedIn and I, I forget which one, spoke about this prioritizing. So you're saying, pick your battles very carefully.
Yeah, and if you are abdicating responsibility to a third party, you need to go. You seriously need to go, because that that puts the organization at risk. Um, and I've seen it happen, and unfortunately, I've seen situations where the the senior most person in IT is essentially, um, my words, they're holding the company hostage, uh, because of some of those strategic mishaps. And it never ends well, but it's gonna force boards and CEOs to make some very tough decisions.
Um, and yeah, it's gonna get r- rectified one way or another. But yeah, d just don't go down that path.
In one sentence, Chris Peterson on Twitter asks, Do you see a specific C-suite role for AI alongside the CEO, like some organizations have done with the CDO, or would that stay within CIO? I'm thinking of, like, chief AI officer, for example. So, very quickly,
I have long since said, and I will continue to say, if you are looking at a chief AI officer or chief digital officer, CDO or CAI, oh, you probably don't have the right CIO. It's not to say that you don't need a VP of AI or VP of of data, but at a chief officer level, no, absolutely not.
why? Why do you say that?
At the end of the day, is it really a role that is separate from the CIO? No, and then is it a role that is going to operate amongst the executive leadership team on par with the CEO, the CFO, CMO, COO, CRO, CHRO, CIO? And the answer is no. It's a very specific task. That's why I say a VP of AI or VP of data is a much more appropriate role. And then that would roll into, um, the CIO, the strategic CIO.
Cause, again, the CIO, their closest relationship is going to be with the CEO, and then secondarily with the rest of the ELT. You're not going to see those other roles kind of taking taking a similar path. And like I said, it's a longer conversation as to more of the reasons why and how that plays out, but that's kind of the top line view on it.
Tim Crawford, thank you so much for spending time and sharing your expertise with us today. I'm very grateful to you.
Michael, thank you so much for the invitation to join and love the questions coming from the audience. This has been great.
Everybody, now is the time for you to subscribe to the CXOTalk newsletter so that we can notify you about upcoming shows. You can be part of our community and you can participate. This show is all about the folks who are listening. So go to CXOTalk dot com right now, subscribe to the newsletter, and we'll see you again next time. We have really, as we always do, like, incredible shows coming up. Two weeks from t next week, there's no show.
Two weeks from today, we have two members of the House of Lords in the UK talking about big tech and policy around AI and big tech. So that's gonna be really interesting, right from the horse's mouth. So join us. Thank you so much. Thanks to Tim Crawford. We'll see you again next time.