Building trust in AI: A CIO's perspective | CXOTalk #869
CXOTalk · 18 Feb 2025
Source: https://podcasters.spotify.com/pod/show/cxotalk/episodes/Building-trust-in-AI-A-CIOs-perspective--CXOTalk-869-e2v1i6k
Transcript: https://podcasts.opennash.com/shows/cxotalk/building-trust-in-ai-a-cios-perspective-cxotalk-869-861f0c/ (via OpenNash Podcasts, machine-generated)

How can chief information officers survive and thrive amidst the turmoil, madness, and opportunity of AI? Today, on episode 869, we're getting advice from two of the world's foremost CIO advisors, Tim Crawford and Isaac Sacolick. Isaac, let me jump to you first. How can early adopters create effective AI strategy for? And we're talking to CIOs.

Very first thing I think of when I think about AI strategy actually is around governance. You know, I need to make sure that my staff understands what data they can work with, that the data is got enough volume, it's cleansed enough to be using inside, uh, some kind of AI model that I'm gonna use it with. That people understand what some of our high level objectives are in terms of, uh, what we're trying to go after.

So I'm actually starting with governance as opposed to just going after and trying out new things. I think a second thing, and this is where a lot of organizations do fall off, is I'm gonna talk about change management very early on. A lot of the early successes in AI has been happening in the IT groups. If I wanna start bringing AI into my sales and marketing customer support groups where there's a lot more value, I need to be thinking about how I'm gonna educate the groups, uh, where I'm gonna give them some leeway to use some tools and work with them.

So I'm gonna really be thinking about change management much early in a AI strategy than maybe I was able to get away with in the past. And then lastly, I'm looking for value other than just productivity. There's a lot of talk about AI lets us be more productive. I can't fund transformation efforts or AI, for that matter, on just productivity improvements. So I wanna hear where is it ch- transforming our business? I wanna hear about where workflow is gonna be improved, in what ways, what value, what, where are we driving quality?

So I'm looking for value beyond just, uh, productivity to be part of my strategy.

Tim, thoughts on this set of issues? I. J Isaac has just raised a bunch of topics, from innovation to culture and how AI is different from traditional IT. Thoughts on this?

You have to think about the end, uh, in mind when you start down this path. But I but as part of the prerequisites for AI, you have to start to break things down. You know, Isaac talked about transformation. I think about it similarly, but I use a couple of different terms in defining different types of AI initiatives. Uh, one type is around, um, innovation, and the other type is around efficiency. And as Isaac mentioned, you know, efficiency becomes kinda hard because how are you really driving toward efficiency and what metrics are you using and how are you actually measuring that?

There's a lot of gamification that goes into it and a lot of question around whether you can actually achieve reasonable efficiency gains, considering the amount of effort you're putting in, as opposed to innovation efforts, which have a much higher output and much higher value to the organization than efficiency in the long run. And so therefore, they tend to have lower hurdles to get over because efficiency is something that, technically, you could do with more people, but you're using technology as a means to to advance your position.

Innovation, as a definition, I'm looking at those types of efforts, is these are things that you couldn't do any other way. These are insights you could not get by throwing more people at the problem. And then the last component that you have to think about before you even start to lift a finger on AI is you have to start thinking about your data strategy. You have to have a comprehensive data strategy in place, because if you don't have good data going in, you're gonna have bad results coming out.

And so this is gonna require a different way of thinking for a lot of organizations in terms of just simply how they think about data, not just where the data is and how they start to bring it together. But also, as Isaac mentioned, where's that governance? How are we start to think about governance? There's a lot of regulation that's coming about over 400, uh, components across just the US. states alone, something like 40 in California. So there's a lot to, uh, get your arms around there.

And I think you have to start with the end in mind and be real methodical about how you traverse down this path.

On the efficiency side, I think we're at a point where CIOs can start looking top down and start reinventing workflow. Um, how are we hiring people? What does this, uh, mechanical operation look like if we built it from the ground up using data analytics, AI, automation from the ground up? A lot of what we've been doing over the last two years in fitting AI in has been taking a piece of that process or taking a process that we haven't done well.

And saying we're putting in a technology or a step in here that's gonna either make it more efficient or more scalable or improve quality. We're really at a point where we can start reinventing our workflow. Um, that's gonna rattle a lot of heads. Uh, we're always used to how things are running today. Uh, we're always used to what our job is in that function. And so that flows back into what, uh, I was saying earlier around change management.

We need programs for our organization, training programs, learning programs, so that they start an understanding and embracing what their work is gonna look like as AI is brought in more fundamentally, uh, into processes. And when I think about innovation, Tim, um, I go back to the days when we launched mobile technologies in the early days of mobile. It was, you know, we were taking web interfaces and slapping them on a small screen and saying, Hey, we have a mobile site.

And so the, you know, the technology got a lot easier. The, uh, uh, to do mobile first capabilities, we started seeing app stores come out, we started seeing mobile first capabilities, and now we start reinventing the user experience. We haven't seen that so much with AI yet. Uh, even our agents, the agentic AIs that we're talking about and bringing workflow in. There's most of the talk around that, and most of the value we're seeing is inside our organization.

But look over the next year or two in saying, how are we gonna do something completely different now that we can put an agent in place and how does that change the customer experience?

If you're watching on Twitter, pop your question onto Twitter using the hashtag CXOTalk. If you're watching on LinkedIn, just ask your question, share your comment in the LinkedIn chat and ask these guys pretty much whatever you want. It's a great opportunity. You can get free consulting right now. So take advantage of it. So you're you're both describing attributes of IT culture, deployment, and so forth that are not really unique or new. Uh, culture is a human nature phenomenon and change management and all of that.

Is there anything that's really new about IT about AI that changes the CIO role that has a big impact here, that's different from what we've had in the past?

We have to get more comfortable with how we trust technology, and also, um, how we bring automation into the fold. I mean, even with technology, a lot of applications still rely on a human component. And with artificial intelligence and some of the the newer technology that's coming or coming out, um, it requires us to get more comfortable with automation in ways that we haven't had to deal with in the past. And that is new, that's very new, because we haven't necessarily had to get focused on our processes, uh, such that we need to make sure that they are sound, they are clear.

And this touches into some of the things that Isaac mentioned around change management. You know, do we have good, uh, processes in place and good change management in place? And if you just turn around and say, oh, we're just gonna automate it, that's a very dangerous decision to make. And so one of the things that's new is it's causing us to kinda go back and rethink how we do things, to speed it up, to ensure that we are getting more accurate output and ensuring that we're not automating bad processes.

And what that means is that there's a cultural shift to embrace technology more fully than we ever have, and that is change. That's a and it's a it's a hard cultural change for a lot of organizations that have maybe been more accustomed to, especially as you get to larger organizations, more accustomed to, um, a more incremental change. You know, we're making these small improvements, that doesn't suffice anymore. You have to make tectonic improvements, you have to make transformational improvements, because that's exactly where your competition is going.

And if you don't make those, you'll be left in the dust.

One of the differences, Michael, is that it's happening to employees in our back offices a lot faster than what automation did. You know, their jobs are changing, and it's looking like the things I used to do, I maybe no longer have to do it at all or do it fundamentally different than I've ever done before. Software development, I mean, if you asked any of us four or five years ago, could machines code for us, we probably would have said, No.

I have a blog post around this. Now, somewhere around 20 to 30 of code being written by AI is being accepted inside, uh, I uh, being pushed and being pushed into production, being accepted by the development group. So it's changing fundamentally how our employees are working. Uh, and for some, you know, they embrace it, they learn it, but for a lot of other people, uh, they're still scratching their heads and saying, What do I want to do now that AI is being able to do some of the things that I'm doing?

I think of that as a writer, right? Am I writing for people or am I writing for, uh, for LLMs that are gonna bring my, uh, data and its content and use it to answer questions? So it's a is a fundamental shift. And then with the CIO role, I mean, we've seen this a little bit in the past, but it's getting harder. You know, every time there's a whole new discipline that we have to learn, you know, whether it was digital or data, and now AI.

If we didn't learn it as a CIO enough and build up enough competency around us to lead these areas, the board, this, uh, the executive group would look at and say, we're gonna go hire a chief digital officer, we're gonna go hire a chief data officer. Now we're saying we're gonna go hire a chief AI officer. And learning AI is fundamentally a little bit more difficult. We have to scale our ability to learn from the organization.

So I think the the the goal for CIOs need to move really fast and make sure that our lieutenants and our high potentials are getting the enough enough leadership and getting out there and learning enough so that they can inform me as a CIO about how to invest and not meet, uh, going back to them and saying, this is the area that I want to focus on.

On Twitter, Mike Boison says that to take advantage of these AI opportunities, he says you would need completely new business models to weave with AI, and those business models will drive what your data needs are. And then on LinkedIn, Greg Walters says that, uh, this is a long question, so I'm gonna shorten it. I'm gonna ask you guys to ask short questions so it's easier for the moderator, that's me, to to ask. So I'm gonna shorten this, he says.

We're transforming from manual thinking to artificial thinking. Does this mean we will proceed by ignoring silos and hierarchies to change everything and ensure fully integrated organization-wide adoption of AI? And and I think the the common thread here is the power of AI to drive disruption. But what kind of changes do we as CIOs need to make in order to take advantage of th- these opportunities?

It kinda comes back to, is AI gonna drive changes to organizational structure and and how we think about organizations and operate as organizations? I mean, let's not forget, an organization is a living being, right? It's an evolving living being, and it changes over time. And culture is just one component of that. I do think that AI is, and it all we already have examples of how it is changing how we think about structures within an organization.

Has it gone so far as to break down the silos broadly? Not yet. I think at some point that might start to happen. I mean, you could envision a position that we get to sometime down the road where an AI, um, type of solution, magical mystical solution, looks at how we work as an organization, starts to learn how we engage with customers, how we engage with employees, what the market dynamics are, what the environmentals are from geopolitics to weather, and is able to actually tell us how to run our business more efficiently.

That's actually something that we can envision and get to. And then kinda ta- kinda building off of Isaac's earlier point. Now, just imagine you need software to be able to do it, and that same system can then go and start to build some of this software. There's already talk, um, amongst vendors, big, huge vendors with really big, complicated applications and platforms around. Can we use AI to essentially rewrite the entire platform? And what? How can we learn from the way people work to improve upon that and then kind of feed that into the software?

So the short answer is not yet, broadly, but we can completely envision that. I do think that AI is already having an impact, though, on org change.

Join our community, go to cxotalk.com, sign up for our newsletter so we can notify you of other live shows. We have a question now from Arslan Khan on Twitter relating to this. And since you're the one who first brought up culture, let me address this to you, he says. Who decides what's a good process versus versus a bad process? Most standard operating procedures become archaic since people find other ways of doing things that are not even document.

And I'll ask you to Isaac to keep this pretty short because the questions are stacking up.

Our notion of process is changing dramatically. We used to think of linear process and linear handoffs. We used to think about how to automate pieces of this. And you know, what's changing is our ability to put intelligence to handle all the nuances in complex decision making. We wanna be able to ask questions and knowing that the analytics, the data behind it is far more complex than what can go into a linear process. And I'm gonna ask a question, you know, given what's happened in the United States over the last few weeks, how should I evolve my supply chain?

What are my options? This is not a linear process anymore. This is about having the right data in place to make tactical decisions as things change and evolve quickly. And going back to what Tim was saying, I mean, we we may not be able to do strategic oriented questions at that scale just yet, but we certainly can get our data ready for that. And I think that's a big challenge for organizations to think about today.

Martin Davis asks this question. He says, How does a CIO avoid pilot purgatory with AI? So who wants to, uh, explain briefly pilot purgatory and then give the antidote to pilot purgatory?

I just read a statistic that said, uh, 69 of companies are running 10 or more POCs, and 10 are doing 50 or more of them. And to some extent, I I don't have an issue with that. I mean, I think pilots and POCs, uh, you know, not all of them are gonna make it into production. A lot of them are learning exercises. A lot of them are really fishing into, uh, into the underlying data. Can the data provide, uh, enough of a backing to support the AI, the hypothesis around the AI?

And even when it does, um, does the AI that you've created drive enough business value to put it into production? So when you put that all into play, it's not surprising to me, at least, that a good number of these are not making it into production. I think the real issue is that organizations don't have a process to make that happen. When they really have a winning POC, a pilot that's delivering, do they have the change management in place to impact the people who are gonna be affected by it?

Can they bring that AI out at scale? Can they manage, um, the operations around the model ops and the ML Ops that go into monitoring a model and making sure that it's still relevant? So I think there's, you know, you do so much work just to figure out what POCs to go after, what data to go after. That even when you have a winning formula, you haven't done enough of productionalizing to figure out how to make it, uh, do that at scale.

Tim, maybe you can help me make sure that I understand what Isaac said. So let me read that back to you and you tell me if this is right, okay? So I think what Isaac just said is there's a lot of people playing with AI. They don't really know what they're doing, they haven't thought it through, their organizations aren't ready, they don't have the data ready. But, you know, they're spending money because, well, it's not clear why.

It's absolutely true. And every time I turn around, I see another survey, another, um, some more anecdotal information that comes in that's basically supports that. Now, I don't wanna, I don't wanna come down on the idea of experimentation. I mean, having a culture of experimentation, especially within a technology org, is really, really important. And that's important because you want people to use their imagination. You want people to bring that creativity in because ultimately, that's what leads to change within an organization and potential, um, ways for your organization to differentiate yourself from your competition.

And so CIOs know that see leading CIOs know that they need to find ways to really kind of explore these new areas. But, as Isaac said, you do need to be somewhat methodical with it. And as we've gone through the last twelve, eighteen, twenty four months, there's been a a market shift where the bar has gone up. That before you even start an AI experiment, you need to have some very clear business outcome, and you need to understand how it ties to one of your company's objectives.

And typically, that falls into one of three camps. It's either around customer experience, employee experience, or business operations and supply chain. It's one of those three. So I think the the thing, Michael, to to consider there, and and to Martin's, uh, original question is you just have to think about what you're doing and what makes the most sense. But still, don't don't make sure that you h- don't, uh, put everything together and expect it to be buttoned up from the start.

You still have to experiment, but you need to more quickly come to an outcome to determine whether this is a project that has legs or whether it's something you need to to pull the ripcord on.

Tim, I like that part of it. I think that's what companies miss a lot is how long do you give a team to experiment? And even if the experiment is a little bit open-ended, when do they come back with their vision? How is it aligned to the strategy? I mean, you can give people some time to go out and learn what the data is telling you, but you can't give them a a mile-long runway anymore. The expectation is that an ROI that a an AI is gonna be delivered in about 8 months, that ROI is gonna be measured in about 13 months.

That's pretty quick. I mean, we couldn't do apps at that rate 10 years ago. And we're saying we're doing all this data work, we're doing all this change management work, we're putting all this AI in place. We still have a skill issue here and we're trying to get to s to value between 8 and 13 months. As sentiment from boards and CEOs are looking at and saying, you know, the honeymoon's over. I better start seeing some value from all this investment I'm making.

Liz Martinez on LinkedIn is screaming out during this conversation, business case, business case, business case.

I agree there needs to be some connection to a business outcome. I would just caution on using the phrase business case and the reason why, and we might be agreeing or disagreeing on this, Liz, is because a lot of times when people think about the business case, they put together this multi-page document that could be 20, 30 pages defining why they wanna make this investment into this experiment and what the potential outcome is. And that's not what we're talking about.

We're talking about these very quick wins. But I agree it needs to have some connection to the business, as I mentioned earlier. So as long as that business case is very short and direct and to the point, and as you start down the path, you can show, uh, connection, kinda closing those connecting the dots, if you will, to the outcome, uh, in short order, in the timeframe that that Isaac mentioned. Um, then I'm great. I'm good with that.

Just don't look for the fully, like, put together business case, business plan.

The three-year plan, the full P and L. No, I Tim, I use a one-page vision statement, and, you know, and it's got, you know, information around customers around it. It's got information around strategy. You have to project the timeline, you have to project what OKRs you're impacting. Uh, very important, you gotta have a value proposition for the end user or customer that's benefiting. And, um, anyone wants it can come reach out to me. I'll be able to share it with anybody who's interested in it.

But I agree with you, it's gotta be simple. And, most importantly, what we used to call business case is really about alignment, making sure that people who are working on this, uh, making sure that stakeholders and making sure executives know what the objective is.

I'm gonna bring two questions together, and this is from Joseph Puglisi and Ashish Parulkar. I hope I have your name correct. And Joseph says, speak to the need for governance. What lessons have you learned around early efforts to use AI? And Ashish says, How can the company communicate its AI strategy and progress to employees and stakeholders that are inclusive and get adoption?

AI governance is an extension over what we've been doing with data governance. There's some nuances to it that are important around areas around bias, around ethics and AI. But the the main lesson learned I I find is that we treated data governance and data security very separate tracks from our innovation and very often lagging. So we'd figure out what the innovation was around data, we'd cleanse our data, and then we'd start looking at what are the governance implications around it.

With AI, um, you need to flip that. And even with data, the advice I've given to groups I work with is, you know, I do everything in agile. I have my data scientists working on the model, I have my application developers working on the user experience, I need my data governance specialists there, I need my AI governance on that agile team. To make sure that as we're working through this problem, we're doing things that align with compliance and regulation.

As Tim mentioned earlier, these are changing very rapidly. There's a lot of depth to them, so not everybody has the answers around them. But I'm putting people who are experts around this right on my team, so that I can do governance in parallel with my innovation.

A couple things to add in into that second question, uh, briefly, is number one, make sure you have good relationships with your head of legal, um, head of audit, uh, if you're the CIO, those are some of the first relationships you should be making within your C-suite, um, within your peer network. And then this the second part of the question, uh, around communicating this out. I mean, many organizations have had success with creating a governance body or a council.

And so what that does is two things. One, it starts to bring different ideas and perspectives into the mix, so you get those different personas engaged. But it also has a dual purpose, which is starting to communicate those shared objectives out to the rest of the organization. And so by bringing that together, it's not just sitting with the CIO or one individual, whether it's the head of audit, head of compliance, whoever, depending on your industry, but rather it becomes a shared responsibility.

And I think that's probably the best way to go at this stage of the game.

I'm gonna add one person to that, and that's the CHRO. I think is really important to have there when we talk about change in people's jobs and that changing that quickly. And a tip for CIOs, if you're looking for budget around training and learning and development programs, a lot of enterprises have that with the CHRO, right? And so bring them into the conversation around how AI is gonna impact the business.

And we have a question now on data from Michelle Clark on LinkedIn. And let me just remind everybody that this is your opportunity to ask your questions to get free consulting. If you're watching on LinkedIn, pop your question into the chat. I urge you to put these guys on the spot. These are sm- two smart guys. And, uh, ask the ask the hard questions, not to me, but to them. Michelle Clark says, How do you get clean data?

How do you know that your medical diagnosis data, for example, isn't full of racial and gender bias? And this is really, really important for every business.

Usually been a lot of financial services industry companies that have been using these data platforms, uh, watch longer with a lot more skill than in other industries. And because so many of us are investing in AI and looking at our data, we need to have those platforms in place. So if you don't know if your data is cleansed enough or is healthy enough or is trustworthy enough, the plat- go look at the platforms and see how to tune them for your type of data.

Completely agree. Use the technology to your advantage. They're getting far more sophisticated now, where where they're actually showing ongoing, real-time dashboards about the health of your data. Um, so leverage that, and I would just add, I mean, you have to also bring the human component into the mix too, which is help people understand why things like bias and how they're bringing that data, how they're putting data into these systems, how it starts to impact things downstream.

Let's go back to Twitter. A really important question from Naya on Twitter, who says, What are the top three challenges organizations face when pioneering AI apps?

There's two sides to this, okay? Uh, one area you're gonna see a lot of pioneering is built directly into the platforms that people are already using. So CRM, CHRO platforms, they are all putting agents in place. Uh, they're all trying to say, put more data in our environment or port more data accessible into my environment, because I'm gonna bring the AI to you. And and so the the real question there is, uh, understanding whether your data is ready for this is one of the things I don't see in some of the platforms is they can go, you know, do, um, sales forecasting, or they can go do assistance around hiring.

But they're not telling you enough about whether the data in your platform is ready for that. Uh, ask your people, all right? This is a great place to engage people. Let them go use the platforms that you've already sanctioned to try out these agents and come back and saying, what's where's it delivering value? Where's the data need to be improved on? And go from there. When it comes to building your own agents, right? And I know a number of companies that are starting to explore this.

It really comes down to a build-buy discussion and thinking about where are you having proprietary data, proprietary value to customer that you're gonna start investing in building out an AI agent with such good data that's gonna really be a game changer for you. And so that's what I'm looking for first is where is there something that's going to be a game changer in our industry or in our company, or because I have data that's worth investing in building out that capability.

I'm gonna counter Isaac a little bit on this one, um, because I do agree with some of the points, but one one thing is it's people, people, people, um, is one of the biggest challenges that organizations have. And the lighthouse companies that started to roll their own with AI learned very quickly they just do not have the capability to truly do that and maintain it. Cause, let's let's keep this in mind, it's not a one and done project.

And so what you're seeing now is companies starting to adopt AI technology built into those existing products, as as Isaac, uh, mentioned. The one thing that I will counter with is I would highly recommend at this point that organizations, CIOs and organizations, think about buy over build, especially with this technology and where it's it is today. And the reason for that is there are so many variables and parameters, and very, very, very few organizations, IT organizations or line of business organizations, are truly capable of getting their arms safely around the technology.

You potentially run into more risk than reward when you start to go down the build path, unless you can truly get your arms around it, which few can. So I would encourage buy over build at this stage. Um, of course, that's gonna, that's gonna vary depending on the particular use case and scenario and organization we're talking about.

I agree, today is a buy over build, but what CIOs have to keep in mind is the build is going to get easier and cheaper. You know, we saw that with mobile technologies. If you built, uh, too early, you were building a lot of proprietary code and getting into technical debt around that and getting into user experiences that needed to get rewritten. It's going to get easier to build your own agents.

It is, but the but the other piece to this that that hasn't come up in our conversation is the adoption piece. And we're already seeing a lot of, um, dragging of the feet and just not really good adoption numbers of just simple things like co-pilots, uh, within your existing productivity applications. So this kinda gets back to the people and the the cultural pieces, but adoption of these tools is gonna be really key.

Ashish Parulkar comes back and he says, Are organizations adjusting their OKRs, OK OKRs, sorry, and metrics to assess the impact of AI on business outcomes, or how are they planning to measure the ROI? This comes directly back to the question of alignment between AI initiatives and business strategy that you both alluded to. Tim, you wanna jump on this one first?

It depends on which OKR, which particular strategy you're talking about, because in some cases, it shouldn't impact them whatsoever. Whether you use a blue technology, a yellow technology, red, green, it doesn't matter. What matters is the business outcome, and so there are different ways that you're going to accelerate or hinder your business. But you need to stay focused on what that outcome is, whether it's AI or not. Um, when you get into some of the operational pieces and you start to wanna measure things like before and after of bringing new applications or technology into the mix, I could see those changing.

But you need to distinguish between those business strategy pieces and the ones that are kinda more in the weeds, and that's a really big differentiator to answer that question.

Isaac, here's another question. Let me, uh, direct this one to you. This is from Chris Peterson on Twitter, and he says, regarding POCs, to what extent are legal and audit functions actually ready to be involved in the development and innovation and not just be reactive after the POC? So, in other words, to what extent are legal and audit functions, uh, involved and should be involved?

They're understaffed to be able to keep up with the space of change. Uh, the technologies aren't transparent enough for them. Uh, that might change when you look at what's happening with agents being able to take natural language input and be able to share that natural language thinking and natural language output. Where, where, uh, uh, when it has a conversation with another agent, we're now starting to be able to create a, uh, an audit trail in English that an auditor or somebody in legal can start following.

So it's still early in this path, but because we're taking our services, moving off APIs and moving into natural language interfaces, and they're sharing their thought streams and where they're passing other questions to other agents, we're gonna start being able to have a lot more audit controls and legal controls around it.

They have to be right up there at the front of the line. So at the beginning of the conversation, audit and legal has to be part of the conversation as part of your data strategy. But the other thing is there's been a lot of concern around the black box nature of these LLMs, and that's starting to change. We just saw, um, OpenAI come out and start to share some of the reasoning, and I think DeepSeek is driving some of this.

But now you're starting to see companies like OpenAI that are saying, okay, we're gonna actually expose some of the reasoning that went into the answer that we gave you from your prompt. And I would expect to see more of that.

This is from LinkedIn from Jason Guterres, who says, Quick wins take smaller bytes, bytes, bytes as well, out of the problem you're trying to solve. And here's the question, what KPI are you trying to influence? Is your dev team skilled enough to deliver an AI app quickly, or are they still upskilling? Which raises the very important question of talent, do we you guys spoke about build versus buy technology earlier? What about talent and developing talent in-house versus going out on the market and recruiting people who have those skills?

How do you think about that balance? That

Skill set has changed from knowing as much as you can and how to do something into knowing what to do and whether what's being built is being built securely and robustly and and and has high performance. It's a shift in mindset into knowing how to ask the right question, than knowing how to roll up your sleeves and getting something done. And that's that's perplexing, particularly for us engineers and those working in IT, to be able to think about this.

Uh, but that's the nature of what AI is allowing us to do. It's not just around, can am I more productive? It's, am I able to do things that I wasn't able to do because AI is providing the assistance around it. Last summer, I did coding for the first time, but I didn't write a line of code. I had AI write the line of the code, and I just took the person asking the questions and saying, I need help to be able to do this function.

How can I get some code to be able to do this? And after five or six prompts, being able to do this. So, uh, I think the same thing is happening inside our IT organization. I think there's a real question for CIOs in particular. I saw a data point from McKinsey saying that IT is more than twice a advanced using gen AI than other departments. But the question is, you know, when we use the word productivity, when we use the word capability, and we start asking the question around ROI, if we're not showing where that's delivering value, we're gonna be asked to give cost up.

Jason Guterres comes back to both of you guys, maybe Tim, you want you can grab this. And he says, Sure, but doesn't that necessarily mean more opex?

There isn't a one size fit all for every organization. I think one of the things that that I back away from is trying to get too granular on that answer. Um, I think what's important for the CIO, one of the things I'm looking at is how do I start to measure my organization's value and impact to my business. And when I say my business, I don't mean a particular department of the the company. I mean to the business and our company's customers.

And so I'm looking at how I tie what we do as an organization with IT, with our business partners in HR, with our business partners in finance and operations and engineering. And I'm putting together OKRs and metrics around our impact and performance against those objectives that we all share. That's where you have to start. Now you can delve further into developer productivity and call center productivity, and and that's great. But when you start to get further into the weeds, that's where a lot more variables come into play and you ha- there there is no one size fits all answer at that point.

Isaac, we have a question from Derek Butts, and it relates to risk and security. And he says, AI, uh, risk guidelines and standards are continuously being developed. Can you recommend any AI frameworks to securely roll out and mature the value of AI tools for your business operations without increasing the risk across your business culture? So I think, really fundamentally, we're talking about risk culture and rolling out at scale.

When you're doing innovation, when you're driving change, you, by definition, are increasing risk, right? Um, there's no ways around that. If you wanna stay doing what you did today and continue to perfect it, that's, you know, you're gonna be in that box that can be disruptive. So as soon as I'm innovating, as soon as I'm looking at any new capability or I'm bringing change to my organization, I'm taking on risk. And the question is, you know, are you taking smart, parallel measures around that risk?

Are you identifying it? Are you bringing the right people in who are gonna ask the questions from a risk management perspective? Tim's brought up legal, audit, security. Am I bringing those people into the right conversations so that we're asking those questions earlier? What's gonna be the impact if this data leaks? Should we be using this data for this perspective? How are we securing this data? Is our data masked so that if anybody comes into our organization and starts using that data in a new way, that data is masked for PI information?

These are the kinds of things that are building blocks f- when we're doing innovation that we have in place so that we can do those things securely.

Tim, this is something that's orthogonally related, and this comes from Arslan Khan on Twitter, who says, How do you address shadow IT from frontline employees versus executives? Should we encourage shadow IT? And the reason I think it's orthogonally related is because one of the traditional arguments against shadow IT, of course, has been, Oh, we're going to increase our risk footprint. So what about shadow IT? And especially in this age of AI, where everybody's using ChatGPT?

I actually support shadow IT, and there are a lot of folks that that think it is a dirty word or it can be problematic for the organization. But you have to go back and understand why is shadow IT coming into your organization to begin with? And typically, it's because someone isn't getting what they need in the way they need it. Um, that's the simple answer. The the longer answer may have a lot more components to it.

Um, when it comes to AI, yeah, there's sure there there's potential for invoking more risk into the equation. However, if you put the right guardrails in place, you actually can enable shadow IT and that creativity, especially in business units that understand the business better than your organization, and you can embrace that. And organizations have been highly successful in creating a culture around shadow IT, so it becomes more, um, co-development, it becomes more, uh, collaborative in nature as opposed to us and them.

But it starts with the culture and then just kind of fans out from there. Yes, of course, you have to think about the data governance pieces. Yes, of course, you have to think about cybersecurity and and those kinds of components. And you also want to think about simple things like how many different versions of generative AI do you want running in your organization and all the data that goes behind it. So all of those do come into play, but I would start with what is your current stance with regards to shadow IT, and then let it fan out from there.

Let's go to another question. This is from Mark P. McDonald, who, uh, who's been a guest on this show. He's a distinguished vice president and a research fellow at Gartner. And he says, Who are the AI leaders, companies that we can follow that CIOCIO see as the leaders right now that we can study and who are doing AI well? Any thoughts on that, either of you?

There are some companies that are doing some amazing things. Can I talk publicly about them? No, and the the unfortunate piece to that is because we are in the very early days, they're using this as differentiating for their business strategy. And so they're finding ways to really kinda change the game, not just change the the chess pieces, but changing the whole game, um, and using a very different approach with it. So we're not quite there yet.

You do see some public examples of smaller examples of how AI is being used. You know, everything from code generation to summarization. Um, you know, think go back to the earlier conversation about legal, you know, being able to summarize the body of work around, um, legal. I mean, that's that's a massive opportunity. Um, so there are some of those opportunities that come into that efficiencies space. But the ones that I'm familiar with, the really big, demonstrable ones that that would be kind of those lighthouses, those beacons of opportunity to follow in their footsteps, those are pretty, uh, close to the vest.

Still,

Is it close to the vest because they have figured out the magic silver bullet, or is it close to the vest because they're crying in the corners and don't want anybody to see?

uh, you know, there's probably some of the crying in the corners too, to be honest. Um,

No, hey, listen, I I understand nobody wants to cry out in public, right? I mean, we wanna get in the back and lick our wounds.

I, you know, it's yeah, I mean, l- let's face it, you gotta, you gotta have a few cuts to to have success, right? Um, and the examples that I was thinking of, two examples that I'm thinking of, they're pretty significant examples. Um, but again, I kinda wish I could even give you some context, but it, I don't know how to do that without exposing who it is.

Look, I'm looking for the small examples, okay? And the reason they're slow to announce them is their fear of people coming in using the technology to find the gaps in what the AI isn't doing well. And then they're gonna be in the headlines for the thing it shouldn't have been doing in the first place. So they're rolling these things out slowly, but I'm starting to see customer-facing agents come out. I got one in an email this morning from a bank about being able to go through an agent around a car loan.

We hate going for car loans, it's horrible experience. So when you start seeing that being publicized, little things that people are doing that agents are starting to help, well, you know, that some of that is changing in the marketplace.

Tim Jason Genovese comes back, and he says, Regarding shadow IT, which you're in favor of, he says, as shadow IT steps in, at what point does enterprise architecture become a concern? And really quickly, please, we're cause we're running out of time.

Enterprise architecture has to still be there. Um, I will say a lot of enterprise architecture organizations get a little too big for their britches, and so there has to be a bit of a check and balance there. Um, but I think if you're, if you're taking the right approach with EA in such a way that it becomes more modular and more of a framework as opposed to a heavily over-architected, uh, structure, then it's accommodating for shadow IT.

Tim, here's one I'll direct to you, and this is from Ashish Patok, and he on, uh, LinkedIn, he says, Buying AI may come with a cheaper cost, but what about the fear of data privacy and security as the data is governed by the AI service provider? What are the key parameters to be kept in mind when choosing an AI service provider?

This kinda touches into what I was gonna respond, uh, to an earlier question, which is why many companies are choosing to buy and use AI built into their existing enterprise applications as opposed to building. Because those companies understand the challenges around the data management, data governance, and they're able to put the right guardrails in place. Most enterprise organizations don't necessarily know how to navigate that. This is all new for them, but those enterprise application vendors actually have the the girth or or scale to be able to do it for many different companies.

And so a good way to think about this is I can kinda build off the the backs of those organizations as opposed to having to figure out how to do the basics myself. And unfortunately, there's a lot of assumptions that IT organizations make. We saw this in data centers thinking, Oh, my data center is super secure. Guess what? Of all the data centers I have done assessments on, they are not as secure as you think they are.

And so the problem is you have to take those assumptions out, and unless you're willing to do that culturally, it's really hard to ensure that things are buttoned up to the same degree.

Isaac, Jason Genovese just wants to be clear that he disagrees. Well, I'll say this to to to both of you, that he disagrees with shadow IT. He thinks there's too much risk with data exfiltration. If you're going to allow it, you must have appropriate security controls in place. And he does agree, however, that responsible sandboxing of apps for testing or a P-A-POC is necessary. You just can't forego security controls. And, uh, it's a shame we're not in a bar, cause we could all step out and fight.

No, I don't think we're fighting. I mean, it's

I don't think we are.

Oh, well, okay, I'm trying to instigate a fight, and it's not well, and it's not working.

well, I, I don't. I don't agree that shadow IT is a good thing, but what I think CIOs have to understand is that shadow IT and now shadow AI is happening. Okay, that they can't put up the walls and pr- prevent it from happening. And then you can go back to Tim's comments and saying, Okay, what can we learn from what people are trying to do that we're not servicing well, or they're that learning enough in terms of what are the risks or what are the technologies that we've already put out in place that that they can go out and use?

So I think the real question is, well, how is CIOs monitoring for this and responding to it?

All right, so you're on Jason's side, I'm on Tim's side. Because personally, I think that if an organization has shadow IT, it means that the the the people out in the trenches are not getting what they need, and so they're bypassing and they're just doing it themselves. But that's why I say, I wish we were together and we could go out and fight about it.

Just to be clear on that, I'm not saying shadow IT free-for-all is okay. So just just so we're clear on that, I'm talking about a managed approach, an integrated managed approach to shadow IT. I'm not talking about the free-for-all that many people think it is.

So Tim makes clear that he's not talking about technology run amok, and he won't allow it.

As a CIO, you have a responsibility to the organization, right? And so, on one hand, you need to make sure and and make hard decisions around what you do and and how you support the organization and how you engage the organization. And so that's why I think shadow IT can be really powerful in a collaborative sense. But you're right, I mean, if it's run amok, d just going off and doing whatever they wanna do and however they wanna do it, sure, that's risky, that's not a good thing.

But again, that requires some, um, some mature thinking from a leadership standpoint to be able to get to that point.

Martin Davis, who says, AI, AI as a service, has the potential to turn all business apps into simply databases with an AI layer above to handle all business logic in one place, no longer siloed to functions, purposes, et cetera. How should we prepare and plan for this future? And I think it's a great place to, and and so Isaac, very quickly, how should we prepare for the AI future that lies before us all? Very quickly, please.

It's a great vision. Uh, I don't think we can get there that easily. Uh, we've been having a vision of a utopian connected system for a very long time. Um, I I think to execute on that vision really comes down to developing your people. Um, the the technology is changing so rapidly. When I talk about transformation, transformation evolves every 18 to 24 months. Uh, this is just the next wave of transformation with its own languages, with its own risks.

And I think it really comes down to having the leaders in place that are looking for the opportunities, uh, where to spend time on, where to do the experiments, and how they're gonna evolve. Not just the cost equations and the efficiencies that we're working on, but how are we gonna d really change our business model and evolve? Because AI is a a brand new capability for us to consider.

Tim, you're gonna get the last word.

I actually agree. I think one of the things that, um, that you need to think about is how can you accelerate the rate in which you're transforming? Transformation is an ongoing process. It's not a it doesn't have a start and stop to it. But you need to look at how you can accelerate not just the technology you're using and the innovation you're using, but also how your organization, both within IT and and outside, is evolving too.

So that's gonna require change in terms of the people, the personas, the skills, reskilling, upskilling is gonna be heavily, uh, engaged in this process, as well as the relationships you have both inside and outside the organization. That's a that's a huge remit for CIOs, and it's a different remit than they necessarily have had at this scale, at this pace, in the past.

All right. We have covered a lot of territory today in this hour. I want to thank Tim Crawford and Isaac Sacolick. Thank you both so much for coming back and spending your time with CXOTalk. Really, really appreciate you both.

Thanks for having me.

Thanks for having me, Mike. Great show.

And thank you to everybody who watched. You guys are an incredible audience. The questions and the insights that you have. But before you go, join our community. Go to cxotalk.com, sign up for our newsletter so we can notify you of other live shows. You see the discussions we have. So join us now. In two weeks, we are speaking with AT&T's president of consumer. She owns the lion's share of AT&T's revenue and business. So join us then.

You can ask her questions and share your comments. And with that, I hope everybody has a great day and we will see see you again next time. Take care, everybody.