The CIO Agenda for AI (with IBM Consulting)

I can go onto my observability layer and I can see the sixty models that we're currently using across IBM, sixty models. When it's proof of concept, it's it's relatively small amount of investment that you're making. Once you go to scale, it's completely different.

C E O s demand A I, but are C I O s ready to deliver? Andy Baldwin runs offerings and growth for IBM Consulting, which has $21 billion in revenue.

The demands on them and their functions and the business has never been, never been greater, never been higher. Particularly with the speed with which they now need to deliver, uh, a I applications for the business. The challenge is a lot of the time they're not in full control of those A I applications. A lot of what's happened, you know, with A I. If you broadly group it into enterprise based and then obviously personal productivity, either provided by the company or, in some cases, companies allowing employees to use their own personal productivity tools.

And at the same time, obviously, the C T O s, C I O s are being asked to deliver the returns that are needed for those A I investments. But then sometimes they're not necessarily in full control or have full oversight of those personal productivity, uh, assets and tools that are being deployed. They obviously do have more influence, more control over the enterprise level agents or deployments that are being pushed into the organization.

It sounds like the old shadow I T, but now on s very significant steroids.

It's a good analogy, and I think part of the challenge as well is obviously with, A I, and you've seen this coming through, um, substantially in the last probably twelve, six, twelve months. We've moved aw moved increasingly away from what I would call multiple small pilots, which, if you like, at that point, the actual running cost, the consumption cost were there, but not nas necessarily that substantial. But obviously, as soon as we started to scale some of those um, a I use cases and applications, then you are starting to see significant increases in the in the run cost, in the computing compute cost, the token cost.

And that's really what's put a lot of, I think, focus and visibility around A I governance. So that the c the C I O has a level of visibility and transparency of all the A I activity that's taking place across the organization, including what would be the personal productivity bit, which I think is what's analogous to the shadow I T organization. As you as you referenced it alongside, if you like, the more uh industrialized uh agent level deployment, that's obviously now starting to drive end to end workflow redesign.

This episode is brought to you by Gartner I T Symposium Expo, Ready to Scale Agentic A I from pilot to production. Join top C I O s and I T execs this October nineteenth to twenty second in Orlando, Florida. Over 300 Gartner analyst led sessions will cover top priorities shaping I T from A I value, governance and cybersecurity to cost optimization, I T operating models and beyond. Get practical insights and connect with peers tackling the same challenges you are.

Secure your spot today at gartner dot com slash us slash symposium. There are so many different tentacles that extend from this lack of control.

I think what's different this time is one, the the the level of impact across the organization from this technology. It's far more um pervasive, it's it's been democratized, but it's also got far greater, if you like, business value potential. If the organization can sort of manage the, you know, the process, there, the human, the the the data there, and obviously get the change management right as part of that as well.

Talk about that change aspect, which is also such an im important facet of this.

My own experience is, you know, in my last organization, you know, we we we deployed at scale, you know, o 150,000 co pilot licenses um, across the organization to really boost personal productivity. This was in a traditional professional services organization where one or two hours of, uh, you know, improvement actually does immediately add to the bottom line. Because you're effectively charging by the hour in that in that business model. And I think what we what I saw and what we learned from that is, you know, really thinking through how you start to incentivize and encourage people to change the way they work and to use the technology.

And recognize that that personal productivity does require changes in individuals personal behavior. And I think how you encourage that, how you incentivize that, how you make the training and development available, but also how you encourage people to, you know, work very differently. And I I use the phrase a lot that, you know, a I is a contact sport. It is not something that you can then just, you know, throw over the fence and hope it's gonna be used, hope it's gonna be adopted.

You need people to be playing around with it, you need people to be using it, and you also need people to be getting to the point where, you know, they start to build their own personal productivity agents. In parallel, you do the corporate, you know, more, the end to end piece, the agents that the organization will have. But that personal thing is is really is, uh, you know, I character it's a contact sport. You've got to get people engaged, you've got to get people comfortable, and you've got to help to s them to see the personal value and benefit of using the tools that you're making, uh, you know, making available to them.

And that's beyond just, you know, the way people have used traditional some of the chat tools. And, you know, effectively inquiring, actually starting to think and change the way they work. Uh, Michael, that's been a that's been a key, a key thing. The other thing I would I would add is just I think the balance with A I deployments versus traditional technology deployments. I think there's an increasing focus on the workflow and the change management that comes with that.

Um, I think more companies are thinking harder about that than perhaps they did with some of the previous, you know, technology deployments and iterations.

Are there unique aspects of change management, uh, in relation to A I versus other technology changes that we may have seen in the past? Or, or, in other words, how does A I drive these dynamics in in unique ways?

One of the certainly in a professional services organization like IBM, I think one of the biggest changes is the show and tell aspect of this technology. Because you're able to develop an agent or a prototype at a far, far faster speed than probably we could have done with any of the other previous technology that we were working with and using. And I think that also then applies to when you're engaging with clients or engaging with people around the technology, whether that's internal or external.

You can very quickly create a um, an M V P or or or a visual of this is what the agent can do, this is the way it could work. And I think that, Michael, that's incredibly powerful to get people in in in engaged, but also interested in how they can start to think about how that capability can start to reimagine. You know how something, uh, whether it's a process or a workflow, or how they've they've they've worked, um, can be can be done very, very differently.

I say that for me, if I think over the years, thirty odd years working in technology, you know, the the extent to which business owners, business line of business, functional owners are now engaging in and thinking through how generative A I and the technologies and the tools can change the way their business or their function operates. At a quite a hands on level, you know that that for me is quite different, cause historically it would have been, oh, I've thought about something, then I'll bring in people to do the requirements, then I'll bring in people.

And it it would have been. There's a time lag, there's a time distance, there's a need to bring in specialists to do that. Whereas I think now, with what we're seeing, timescales are being telescoped. But also the ability to have business users working with people who can use the tools, whether that's forward deployed engineers or whether that's the wrong people. It creates a very different dynamic in how you develop things. But also, I think the speed with which you can then see it in action dramatically reduces.

Doesn't mean, though, that you can shortcut the integration of those agents and security into the legacy workflows. I think sometimes clients and people have missed that. You can develop the agents very quickly, but some of the integration activity and the security provision still needs to happen. And that will take a little bit longer as you harden up those enterprise agents for full for full uh production.

So essentially, what you've got is this very large, decentralized mass of of technology, availability and power. And therefore, the control issues that you spoke about at the outset become heightened, emphasized, and, uh, therefore, all the various potential problems that may arise.

It's been democratized, so this technology, more so than in some of the other technologies we've seen over the last, you know, ten, fifteen years, has been democratized. Once you democratize something, you need to reimagine how you think about governance, oversight, new control, and obviously that democratization, it's it comes with a benefit, also comes with some costs, and also comes with some need from a control point of view. And particularly around, uh, data and the you know, the data that's being used and where that data's going and how that data is going to then be used by the tools themselves.

The tentacles now that are reaching out extend in so many different, uh, areas inside the organization.

I think this point about transparency and observability is why these two phrases is becoming even more important. Because, you know, it is becoming very pervasive. You know, this technology, you know, like, say, some of the technologies where it might have been developed in one place, typically this, you know, the c t o or the c i o function, and then deployed into the business. This technology is being deployed and then being developed, it's being applied, it's being utilized by line of business owners, by people in the finance function, the talent function, as well as in operations, et cetera.

And so how you get to that level of visibility and observability and transparency is is really important because of that democratization. And that's why I think you've seen, you know, and increasingly now, obviously everyone's talking about token maxing and the cost and obviously the large language models. And what also is happening because of this democratization. The phrase I use is, you know, people are using a Ferrari to go down to the corner shop to buy a paper.

Well, you know, you could use a Ferrari to go down to the corner shop and buy a paper. You might feel great in the Ferrari, but it's probably not a particularly good use from a cost point of view. It might be easier to walk, it may be easier to get a bike, or actually, you might have your ten year old Ford Fiesta sitting there. You can use that to go down down the corner. And I think this is going to be the next evolution of this democratization.

Michael, and this this point that you've you've said around the tentacles getting into all parts of the organization. I think from a C I O C T O, you need to know the observability and the transparency so you can see what agents being used, where, which models are being used, where. And what your you need to do as an organization is make sure that, you know, when you need a Ferrari, when it comes with the Ferrari cost, you use the Ferrari.

But where you can actually use a smaller language model, or where you can maybe even use some of, you know, machine learning or robotic, you use that to solve the task. You don't end up using an expensive frontier model. I think that's the next evolution you're seeing, um, you know, clients go through, and you're seeing that. And that's why you're hearing so much in the market about these, you know, agent management platforms, this notion of observability.

Because I think in the absence of having really sophisticated users, reality is somebody needs to be directing the user. To say, hey, listen, that's a great agent. You're going to get as good as an output on this agent using this model, which will cost a lot less than the FR, the the the latest version for one of the frontier model providers.

A few weeks ago, we had Aaron Levy, the C E O of Box, as a guest on CXOTalk. And he pointed out, given the points you're that you're making right now, he pointed out that we've been in this time when VCs have been effectively subsidizing our token use, and those days are coming to a close. This episode is brought to you by Gartner I T Symposium Expo, Ready to Scale Agentic A I from pilot to production.

Join top C I O s and I T execs this October nineteenth to twenty second in Orlando, Florida. Over 300 Gartner analyst led sessions will cover top priorities shaping I T from A I value, governance and cybersecurity to cost optimization, I T operating models and beyond. Get practical insights and connect with peers tackling the same challenges you are. Secure your spot today at gartner dot com slash us slash symposium.

We have definitely moved out of the proof of concept phase now with generative A I, and we're into the industrial scaling. You know, my own organization, IBM, we've been going through this for the last two or three years and we've deployed A I at scale across the entire organization, whether that's in delivery or whether that's in functions. I think there are very few companies who've probably deployed it to the extent that we have. We call it client zero.

But the reason I shared that example is I can go onto my laptop now, I can go onto my observability layer and I can see the sixty models that we're currently using across IBM, sixty models. Each of those models has a very different price point, very different cost point, and I can actually see which models are being used by which tasks and activities. What we can also now do, certainly in my consulting business, I can dial down access to one of the models if I decide.

Well, I don't think we need to be using that. We can direct people to using different models. When it's proof of concept, it's it's relatively small amount of investment that you're making. Once you go to scale, it's completely different. I'll use an example of when I was at my old firm, I launched a a we we we couldn't, we couldn't let people use Chat G P T, so we used the open A I version of it through Microsoft and we deployed it.

And I deployed it to three, 400,000 people. As like, we're saying, this is how we want you to do chat. Actually, within six months, we were running at 25 million, 25 million dollar compute cost within. But so the great news is everyone was using it, nobody was using any tool that they shouldn't be using. But all of a sudden the cost ramped up because that's scale. We were getting the benefit of it, but all of a sudden now you see the cost of doing of running these models at scale.

Which is why I said, you then get into a different situation of how do you manage that observability? How do you make sure you're using the right model for the right task? And also, I think, Michael, what a lot of clients have done is, you know, there's been so much pressure to show how they're using A I, they're deploying A I. That in some cases, they architected some of those things, you know, which probably, without being thoughtful about scale.

So in my own example of that exam that I just shared around effectively OpenAI, ChatGPT, internally, we actually re-architected that and brought the cost down to around 2 million. Cause we do wanted to deploy it quickly, we wanted to get it into the hands of our employees, cause we did not want them using the open source ver. the the sort the versions outside the organization from a risk and a control point of view. So we we did it quickly, but we did it quickly and we probably didn't architect it as efficiently as we could do for token consumption, cause we never expected the consumption to be that high that quick.

Let's take some questions from the audience. We have a bunch of questions that are stacking up and we'll go first to LinkedIn, and this is from a LinkedIn user named Logi Root Runtime. A I Governance. And his question is when, when it comes to CIOs, in practice, are they asking questions such as how do we write better controls, or how do we make sure that the controls that they're actually actually executing?

In the current climate, you've seen, uh, certainly a significant increase, you know, in obviously the focus around security. I think they're effectively having to ask both questions now, because I think the days, you know, the the focus around the. The controls from a production point of view, how the users are now using that, and how they're using the tools, and how they're using this is becoming far more, uh, to the fore. Just because of the sophistication that is now available to bad actors, um, with some of these, um, the models that are now available.

So there is sensitivity to this governance and controls issue that you're seeing among the

No, you know, for sure, for sure. I think the um, you know, I I think this is what's come with, you know, the democratization piece. We're putting these tools into the hands of more and more and more people. But obviously, from a, you know, a a a sort of a a risk point of view, um, you have to have the right environment and the right controls in in place. And I think, you know, if you just look at news coverage in the last two, three, four months, you know, this heightened focus around security, heightened focus on, um, you know, the the the security and cyber breaches.

You know, you, you'll have, so so if you, many of your listeners will have seen the announcement we made around Lightwell, which is really around us providing a clearing house for patching of, um, you know, open source. Because I think the models have started to highlight, you know, gaps and issues in some of the open source code that's that's, uh, that's out there. So I think this focus on, um, you know, control and how it's being used and deployed internally, I would say, Michael, is becoming as important as the the focus that was there before on building those tools and building those applications.

this is from Swami Vijayathan, who says, uh, and he's referencing a report that IBM put out, and he says, given that A I will be the business model by twenty thirty, I. Is there already early evidence that the shift from a technology heavy focus to something, uh, to something fundamentally different? And what are the two or three diagnostic signals a C I O should watch over the next few quarters to know they are moving in the right direction?

Twelve, eighteen months ago, a lot of the focus from an a I, a gen a I point of view in the business was around, it was around, um, coding. It was around making productivity, coding productivity, you know, these vibe coding tools, these productivity tools, you know. Then we started to see the application of gen a I into, you know, some of the call center environments, with, you know, much more sophisticated, you know, voice recogni recognition related capability through the A I.

I think what's what we're now seeing is this shift and pivot into reimagining A I at scale, but in some of those key functional areas. So if I look at the work that we're doing, interestingly, finance, H R and procurement are the three functions that we're doing most. W outside of I T, we're doing most work on from an IBM point of view. So what that what that tells says to me is that the technology is being increasingly democratized.

I think the role of the line of business and functional leader in the application and the deployment of this technology is becoming far, far greater. At the same time, you're seeing pressure from the C E O saying, tell me what I'm going to achieve, what benefits can I expect to get? The C F O is saying, where's my R O I? And because of that, I think you're starting to see increasing demands placed on the I T organization to say we need to be deploying, you know, these tools at scale faster.

And so some of the, if you like, um, signs that I think the I T organization is seeing to indicate this shift is happening is they are seeing rises in the compute cost, they are seeing rise in token uh, cost consumption. And that actually now is getting to the point where it starts to play into the R O I that a business looks at when it starts to think about. I wanna drive this transformation of either the functional or or the overall enterprise uh transformation.

I think you've then got, you know, Michael, I think certain industries where I think this technology shift is existential, being very open, I think, in, you know, consulting, um, at the moment, the impact of this technology in consulting is existential. You only have to look at the share prices of a lot of the consulting firms at the moment to sort of see that without me saying it. And so, and then if you look at the advertising, the agency, the creative industries, so you've got certain industries that I think are going through a next, you know, a much more fundamental transformational shift.

But I think a lot of organizations, you know, you you're seeing the signs of the adoption and the use of this technology. And it's coming up in those run costs, it's coming up in the compute cost, token cost, it's coming up with the need to, you know, the purchasing more and more of the licenses. Whether that's from the frontier model organizations or whether it's from, you know, some of the other providers of A I tools, whether that's the I S versus or whether that's the hyperscalers.

It sounds to me like everything you're describing is a pattern of maturity around the adoption of A I, but at the same time, a pattern of uncertainty. For example, the unexpected explosion in token use and token costs. But but at the but the bottom line is A I is being adopted and lots of people are using it. And we're trying to figure out what's the best way to adopt it, control it, share it, and so forth.

I mean, the technology is very versatile and it has multiple applications, which is different to some of the other technologies that we've seen before. Um, you know, I was heavily involved in blockchain when blockchain came out. Blockchain was a great technology, but it had a a very spe narrow set of use cases. It was particularly strong for, and this this is, this is different. And I think because because of that, that's why I think we're hitting the, you know, the the need from a control point of view and a governance point of view.

And obviously it's using data, you know, you've got to understand, do you own that data or is it third party data? Is your client data? Because sometimes these tools, if you don't have the right governance and the right data strategy, will be somewhat indiscriminate where they capture that data, you know, that data from. But I think there's also, you know, an important point here. And we had a we had a client session this morning in London, um, with about fifty, sixty clients and I.

A couple of my colleagues from the talent organization and the operations team who've driven a lot of the A I transformation. I B. M made a great, great comment, which was in certain areas, it'll be hard for us to see beyond twelve months. You know, that and that that is incredibly uncomfortable to say, you know, and that's, and I say this as an organization with, you know, 300,000 people, 80 billion business. But actually, when we look at, let's say, the impact on skilling and the impact on um, development, you know, we can see the impact of the technology in the short term.

But if you said two or three years out, it's really hard to to gauge and judge two or three years out, what really the impact will be. And I think that's what makes this technology both exciting and, to be quite candid, you know, it's quite challenging from an e at the enterprise level because you are learning and deploying and you're learning more. That's why I think, you know, the successful organizations are building in that ability to, you know, learn from how they're deploying the technology and refine then their future deployment approaches, future deployment models, and also how to get the best out of that investment in the technology.

This would be an excellent time for folks, for you guys, to subscribe to the CXOTalk newsletter. Go to CXOTalk dot com and subscribe to our newsletter so we can keep you informed of these amazing live discussions. So Chris Peterson says he would like to know if the consulting opportunities around A I look bigger bigger on the pure technology side, the execution adoption side, or the governance, risk and controls side.

If I think of consulting broadly, Michael in three categories of you've got advice based consulting, you've got implementation, and then you've got the run or operate part of consulting. I think on the advice based, um, I think what's incredible is, you know, if I look at some of the tools that we've got, we're working with at the moment in in consulting. I think the ability to do those advice pieces of work and getting, if you like, a roadmap, an answer to a client's question or problem, you know, far quicker than we could have done before, I think is quite profound.

And I mean, the taking a twelve week piece of work, you can probably do that now, including the client engagement piece and the client stakeholder management, you know, in two to three weeks rather than twelve weeks. Um, I think when you start to look at the implementation, if I take sort of like s a p implementation, if I take Oracle and some of those large platform based transformations. What's interesting is we can certainly see, you know, faster, uh, to do certain tasks much more quickly.

So we're getting productivity gains of twenty, thirty, 40% in those types of, uh, programs and and projects. And so it's a it's a quicker time to value, but you've still fundamentally, you're still doing those different stages of the work. I think where it gets quite interesting is in some of the operate part of the business. You know, if I take something like, um, you know, cybersecurity, I mean, you know, I've been involved in cyber for many years.

You know, the days when cyber security, when you had, you know, I think you're increasingly seeing, you know, cyber threats now that are agentic, and therefore the response needs to be agentic. And so when you look at some of the, uh, you know, cyber operation centers, they are increasingly becoming automated. You have an agent or an or an automated response to a a, you know, a threat coming through. And so in that part of the operate businesses, I think we're seeing quite significant, you know, agentification.

And and a lot of our clients, when we go in and we talk to them, and we, you know, propose for work or our existing clients, they're increasingly asking that question around. Well, show me how you're changing the way in which you're delivering this. And I expect that, obviously, to be, you know, at a different price point than what it was before, because you're going to be using less, uh, less people. But equally, I expect a, you know, more uptime, I expect a higher satisfaction.

So I think the dynamic across that that portfolio, Michael, of the of what I would say that the advice, the the transform, implement and the run and operate through, you know, gen. a i is quite is quite different. And you know, obviously, we're very focused on how do we basically manage that, um, across the three. Because, as I said, I think in consulting more broadly, I I think this this is an existential shift that we're seeing in how you deliver, um, work to the clients.

And at the same time, clients are changing how they work, given what we've been talking about. And at the same time, many of the technology partners we work with are also changing the way they work in terms of putting far more a agentic into their um, I, their business applications.

Andy, from a C I O perspective, how much of this is being driven by a desire or expectation of greater efficiency versus desire expectation of we're going to change? A I is going to, or agentic A I is going to change our business and yield new, potentially unexpected benefits as a result that go beyond faster or cheaper, greater efficiency.

I think there's a balance. I think over the last couple of years, for sure, you saw a lot of focus on what I would call productivity. And and invariably, to be honest, it was probably because the C I, the C E O, read something, he saw these numbers from someone saying you get thirty, 40% productivity. And like any good C E O, he immediately goes to the C I O and say, hey, listen, why can't I reduce your budget by thirty forty percent?

So I I think inevitably some of that initial focus, that initial emphasis, for sure, has been on productivity, efficiency related measures. I think a lot of the early use cases to prove out the technology, Michael, they were focused on that, that coding, coding efficiency, you know, and and organizations like us, you know, we've deployed something called Bob, which is our in hou, our our in house, you know, co coding platform model to all of our, you know, sort of 300,000 plus people.

And we're seeing, you know, significant improvements, significant benefits from that. So I think that was the focus initially, and I think what you're now starting to see, though, and I think this is an evolution of where companies are with this technology. Because the more you play with it, the more you see it, the more you can think about how you can reimagine your organization and how a process works or an activity works. You know, I always use my own example, you know, I have those three businesses, the transf, you know, the advice business, transform and operate under my remit.

I'm a consultant by profession. The way in which I am now reimagining that advice piece of the business and how it can be done very differently and add a lot more value more quickly to clients, is probably one of the most profound shifts I've seen in the market and the business. So I'm looking at that now as a growth opportunity. And so I think a lot of, I think a lot of clients are going through that, that evolution, they learn more, the technology's being more democratized, you know, they're coming back.

And I think it's a lot of the time now. I think it's the business users who are coming more to the fore, the functional owners, the line of business. You know, if I think some of the projects that I'm personally involved in, it's the C F O or it's the H R O who's saying, actually, I. I can really reimagine how I can do this very, very differently than before. It's the sort of underwriter looking at saying, the old way I looked at underwriting and insurance.

I can do this very differently and I can have a far better, far more effective, you know, underwriting process than I had before. Because they're now a little bit more comfortable with, a little bit more familiar with the what the technology can do and how the technology can bring in, you know, those data sets. So I think we'll see more focus on the growth agenda and on, you know, new business models as we move forward. But like to be honest and to be sure, most of the focus initially has been on efficiency and cost.

I have to just respond to one point you made, which I thought was, uh, very interesting, very insightful. That as time goes on, there's a learning curve that takes place. And it sounds like A I and agentic A I, uh, are not just democratizing the technology, but also empowering workers across many, many different functions. Because now, as they become familiar with the technology, they can help drive process change. And because if those processes are being driven by A I and and agents, it's easier to make those changes.

That productivity, that democratization, you've had people looking at and and starting to redesign certain parts of a process flow. I think we're now starting to sort of capture that, that learning and actually start to see how you can, you know, reinvent, if you like, the overall end to end process, not just the single, the the individual activity. But I think this thing around the the agent learning and the um, the employee learning and how organizations, you know, capture that.

I've seen, I think, alongside the governance framing that we've seen coming through, I think that point about the knowledge and the learnings. And you're seeing this the way companies now, you know, our research showed that most major enterprises will be deploying close to one, 500, I would call enterprise agents by the end of 2026. Those enterprise agents are obviously then deployed, people use them, but the ability to feed back on those agents and how those agents can be enhanced and improved is far better, far easier than it was with, I guess, some of the other, you know, traditional technology.

And at the same time, a lot of the value from the agents comes into the quality of the data that the agents are then using to sort of, you know, make the decisions. I think that's the other dynamic we're seeing at the same time. I think you're seeing organizations looking to improve the quality of the data that's feeding into those agents. And what's interesting is that doesn't necessarily mean they go and do a huge, you know, data management program across the whole of the enterprise.

It means they can get quite focused on the data that's needed to feed into a particular agent to complete a set of tasks or the or the information that flows, um, you know, through a process. So you get a quite a calibrated and a more focused uh, data approach, data strategy approach with this, with this model.

This is from on LinkedIn, this is from Rafal Shimana, and he says, as A I becomes the core business model, how should leaders balance A I driven growth with the governance of autonomous agents? And then he layers on top of it and the post quantum threat to our cryptographic infrastructure.

This point about the agent driven growth, I think you will see an increasing and a need to have the greater focus around the governance and the control around those agents. Because, you know, I think whether agents are being developed by the employee or they're being developed by the enterprise. In one of those, you know, fifteen, 600 agents that I mentioned, Michael, that are being deployed, that we think will get deployed at scale by the end of this year, early next year.

I think the point about governance and control is still going to apply. Um, I think what's interesting with, um, you know, quantum coming through is I think there's been a heightened, um, concern amongst executives and focus from leadership on the the cyber threat presented by the frontier models falling into the hands of bad actors. So that, by its nature, I think, has has really dramatically heightened this focus on on on cyber, cyber protection, cybersecurity. I think that was already there with generative A I anyway, because of this democratization and the way in which the agents can start to pull and use data.

I think certainly my company I worked in before, which is one of the largest B to B platforms, data in the world. We were incredibly focused on making sure that the data we had rights to use the data, cause we have a lot of client data that we didn't have rights to use, and so we were very focused on that anyway. I I think with what's happened, you know, with the launch of some of these frontier models, I think that's taken that to a whole, a whole different level.

I think the point on quantum, I think, look, I think on, you know, we've been looking at as an organization, you know, we're along with, I think, Google, we're probably one of the leading organizations researching quantum in IBM. So we've been looking at quantum safe, quantum encryption for a while. What I'd say is, I think this is very, very much on the minds of a lot of governments. I think the advantage at the moment is, you know, production and producing quantum computers is still very difficult.

You know, we've obviously recent announcement that we're going to be one of the first, um, quantum chip manufacturers in the world. We're building a a a site in the United States. But the the the access to the computer technology for the for the foreseeable, for the next few years will be limited because of the nature of the computers, which is good from a cyber security point of view. But I think a lot of organizations, because of the way in which this technology will pose a threat to so much of our existing encryption technology.

I think many organizations are already starting to sort of think about this and to look about this. They were doing that anyway. I think what's happened with what's with Mythos and a couple of the models that have been announced very recently. I think this whole thing about protection and being ready, I think, is just it's just gone up two or three notches on the at the board level. Because for many management, they're getting asked this by their independent board, their non exec directors are saying, what are we doing about, whether it's protecting from the frontier models or what are we doing about, uh, quantum encryption?

They were asking it anyway, but I think it's now gone to a different level.

So, from a board standpoint, it's taking issues that were seen as important, but maybe simmering, and now bringing them up to a stronger boil.

I served on two or three boards in in a prior life. I think these issues are always there, but to be honest, as a as a board and a management team of running a large organization, you you naturally focus on the top two or three. Yeah, you always focus on the top two or three. Um, I think this is a great example where it was there, it was the board were aware of it, they might ask questions on it.

I think what we're seeing now, not a lot of non execs wanna know. How are we using A I? how are we protecting, you know, data from A I? And given what's happened in the last, you know, twelve weeks, a lot of boards are asking, um, are our systems safe? are our systems safe? You you will. We expect you to have done the preparation, we expect you to have run. You know, some of these models on our software, on our open source or our applications.

Tell tell us where the deficiencies are and tell us how you're going to patch those deficiencies so we don't have a cyber threat. That's what the board, you know, the boards are asking more and more, I think, to to the to the management team. And obviously that invariably then becomes, you know, top of the to do list for um, the C I O, the C I Os. You you see this at the moment. I mean, the level of interest we've had since we announced Lightwell, and obviously Lightwell is a number of other, you know, firms working with IBM on that, not just the IBM consulting.

But the level of engagement and interest we've had from boards, especially in financial services, especially in banking, especially in insurance around this topic. Ha, you know, has seen huge increase in the last twelve weeks.

This is from Braden Kelly on LinkedIn, who says, What is your view on what an A I soft landing might might look like versus a jobs apocalypse, or, uh, labor for great labor force contraction scenarios as a result of A I?

There's invariably going to be some hype, there's going to be some, you know, uh, stories, rumors, and it's going to cause, you know, anxiety. I if I, you know, look at it from the work that, you know, we're seeing and we're doing, I think a couple of things. So one, I think, um, you've got to look at this from a geographic point of view. Because obviously there's a number of markets, number of markets around the world which have got incredibly tight labor markets.

And therefore, where there is any, you know, um, redeployment, redevelopment, there is clearly an opportunity there to have people quickly redeployed, realigned, either within the same organization or actually within the economy. You know, if you look at, say, Japan, for example, even China, I mean, the the the extent they've got really tight labor markets. I think when you get into some of the other, you know, parts of the world, I think there's a point here around, um, training and development.

You know, I was I was involved in some work quite recently in a in a session with the British government, and, you know, it made the point that, um, you know, we need to be thinking about a level of A I skill and knowledge for all of our students coming into the workplace. At the same time, we need to be thinking about how are we equipping all of our workers, um, you know, to have a level of A I literacy and understanding.

There's obviously a role to play there for organizations like mine, but there's equally a role to play there for the government in terms of how they incentivize that and how they, uh, encourage that. The other thing we shouldn't lose sight of is, um, you know, as, you know, these uh technologies get deployed, um, you know, the skills that sometimes they need are not necessarily the skills that you think they would need. So, here's a very personal example.

My eldest daughter just graduated from university about eighteen months ago. She's an English major. She's actually now in the A I division of one of my competitors, and she's doing sort of like consulting work, using a lot of the A I tools. Um, eighteen months ago, she hadn't even used some of these tools, but she's joined this company, great company, great training, she's now doing that. Now the technology is is allowing and enabling her to do probably work that would have taken her longer to build up the experience before she can do.

And so, you know, I think the this notion that we will automatically see, you know, a jobs apocalypse, et cetera, I I think is too broad based. I think inevitably there will be some impact in certain activities, in certain roles. I think then the ability or the opportunity is, how do you redeploy, realign those skills and those people? Um, again, I give you a personal example from my organization. We've identified fifteen, 20,000 people from a skills point of view, that we look at these skills and we say these skills are in declining demand from clients.

So if we do nothing over a period of time, those skills will invariably, demand will invariably shrink and we'll need to, you know, shrink that group of people. But actually, what we've been doing is making that very transparent to people, making it transparent that they're in a skill set that is seeing less demand. And then laying out the personal, you know, learning plans or or development plans for them to basically be retrained in some of the skills that where we see more demand or increasing demand that we can't meet.

And so I I think the critical thing about this is this is not about individual companies. It's going to require, I think, a collaboration, Michael, between government, corporate, corporate and the education establishments in a particular market. And we've got to accept some countries have a different set of challenges than others in terms of how they where they are today, but also what they're going to need to address going forward.

This one is from Greg Walters, and it's a pointed question actually about the consulting industry. And he says, Will there come a time when a person asks the A I the same questions they would have asked a team of consultants? And get essentially, uh, essentially, $100,000 worth of work for almost nothing.

I think certain advice based activities, let's take an example from my former life. There are certain questions you want to ask about tax, tax rules, tax laws in a particular jurisdiction. Will the tool be able to give you an answer to that question? Yes, it will. Will the tool be able to sign a confirmation from a regulatory point of view that it did do that? No, it won't. So I think there's going to be examples where the tools can do that, there's going to be some that will and not.

But bear in mind, you think about broadly about consulting, yes, you want to get to an answer, but often you want to take a set, a management team or a group of leaders on a journey to get to the answer. The A I is not going to take you on that journey. It might give you the destination, but it's not going to tell you how to go there. And often that's the change management process you need to go through to actually reach the right conclusion, but also have that stick from a change management point of view in the organization.

So the mechanistic answers will eventually get taken over by A I, but not the taking the folks on the journey as you described. Uh, question from John A. K. Akinmade, and he says, two thirds of C I Os are accountable, but not in control. That's an IBM stat. Where should they start proving how A I is operating or proving it is operating responsibly? And very quickly, please.

I think the key key thing here is around the governance and it's around the observability and the transparency that they have and they can see. If you like, the agent landscape across the organization, which agents are being used for what act, what activities, by by whom in the organization. I think when you have that observability, transparency as a starting point, you're in a far better point to be able to exert influence and control. And then also work out which agents, you know, agents, like anything, can be retired, they can be, you know, combined with other agents.

That observability and transparency is how you get to that, uh, that that op that, uh, optionality and that ability.

You described pilots earlier, a year ago it was all about pilots. Can you describe what are the attributes that make successful pilots work versus those that quietly die and go away?

I think it's a couple of things. So one is around, um, could you see business value or benefit coming from that pilot? Did the employee or client satisfaction show that there was benefit from again from, uh, from doing that? And then a point of view around this sort of increasingly, when you go to scale, what's the cost and return of using that agent for that set of activities in terms of what you did, you did before.

I think a lot of proof of concepts, uh, failed, Michael, because they either failed because they never really had the data to be able to drive, uh, drive the agent. Or effectively, they developed a proof of concept that was trying to solve a problem that, when you got into it, was not that material enough a problem to justify the investment.

So selecting the right problem and ensuring you're set up to succeed, meaning that you have all of the data that you need to get it done.

Exactly, exactly. And look, I think a lot of clients have gone from, you know, twelve months ago, eighteen months ago, people were doing like, where can we apply agents into this workflow? But frankly, there are tools that will do that now for you. They will basically generate the agents based on your workflow. So that activity is gone. What people are now doing is saying, of those twenty agents, if you like, in finance, what are the top two or three that are going to have the biggest impact?

And now I'm going to focus on those those top two or three rather than playing around with all twenty.

Andy, enterprises sit on decades of legacy systems. How do you recommend that CIOs rebuild that old technology and those old processes without disrupting the company in order to adopt A I?

The the quick answer is, you know, with great care and consideration. So look, I I think what we're seeing with a lot of the technology, you can take a process and you can get agents, and you can have the prototype agents literally, literally in a matter of days. Um, you can now do what takes a lot more thought, and this is where I think the C I O C T Os need support and help explaining this to the business.

What takes a lot of thought is how you then embed that into the legacy workflows in a way in which you don't create unintended consequences. And I think that still takes time because you've got to build in security, et cetera. I think the other big theme that's happening, though, and this goes to the heart of, I think, to your point around legacy systems. I think there's a lot of organizations are looking at some of their legacy applications and saying, this legacy I use for a system of record, and I use it for a system of engagement.

Actually, I still want to use it as a system of record because it's functionally, it's, you know, it's efficient, it can process a lot. But actually, I really want to rethink how my my employees or my clients engage into that system of record. And I think that's where you're seeing the real power of tools like Copilot, you're seeing the power of tools, you know, of like Watson Orchestrate, in that you can have a different user experience, but still access the source system.

So you're keeping the legacy, what you're doing is you're wrapping, uh, new technology, new ways of interacting with that technology through the engagement, the engagement layer. And I think we're seeing a lot of that focus from organizations. So they're preserving, if you like, some of the the legacy, the system of record piece, but they're really reimagining that, uh, engagement layer.

So essentially, as you say, you're, you're protecting that system of record and the storage of the data as your underlying core, and then you're allowing changes.

but that, that, that, exactly, but that engagement layer as well, you know, I think what we're really seeing now is in that engagement layer. You can embed the data context, you can embed some of the workflows that were there, you know, before. And you can also tailor some of that to the individual based on the agents they then use to access some of the some of the data. So that that engagement layer, uh, I think, is where I think there's some really interesting work going to take place in the coming, um, in the coming years, with, uh, with organizations as they start to think about bringing that engagement, data context, decision rights and and the use of some of these

large language models and some of the other A I tools into that engagement layer. Then interrogating this system of record,

When an agent makes a costly mistake, who is accountable, the vendor, the model maker, or the C I O?

I think it's going to depend on the example, but let's be honest, I think in some cases, um, let's take if you're using A I as an advice agent, for example, then it. But the the person who's providing the advice is going to be responsible for the for the agent, because that's their agent. They've done it, they programmed it, they've processed it. I think if these are agents provided out of the box from the, um, the vendor, then obviously the vendor's going to be, uh, responsible.

So I I think the answer to your question, I'm afraid, is contextual, depending on the, uh, on the agent and who produced it and the role it's playing.

So eventually there would there will be this, uh, shared, uh, set of accountabilities.

One of the interesting areas, uh, that's emerging, is obviously, um, effectively assurance of agents. So that how do you have comfort that they are? They will consistently perform the same task and give you the same output again and again. And obviously, in in certain activities that you need, um, you know, a a precise output that's incredibly important. And actually, if it's regulated, it's unlikely the regulators will still will allow you to do that. At this stage, they'll probably still want a human in the loop.

Where it's not, you know, It's more more probabilistic. Then actually that's probably fine. You'll be able to use those agents because you're still requiring on the end user to use some judgment. About what the output was.

What is the one thing that you think you believe C I Os should be doing now?

I think it's just reinforcing. With the business that A I is a contact sport. The C I O can deploy the tools can deploy the models but the business needs to be engaging needs to be using it. And needs to be getting the value out of that and then that needs to be being fed back to the C I O C T organization. So that they can then you know make the appropriate adjustments. But this is a this is about.

You know for me it is about a contact sport you've got to have the the. The end user engaging and using the technology. Uh and then. Create that feedback loop to the rest of the organization.

Well, it's been a a great conversation. Andy Baldwin is the senior vice president of consulting off. Offerings and growth for IBM consulting. Andy thank you so much for taking your time and sharing your expertise with us today. I'm very grateful to you.

Thank you very much. Thanks for having me.

And thank you to everybody who watched and you guys who asked such excellent questions. Before you go. Go to CXOTalk dot com and subscribe to our newsletter so we can send you notifications. We do these conversations all the time. And you should join us. Thank you so much to everybody and thank you Andy. Have a great day.