Proving AI ROI in Pharma Marketing - with Kellen Winters of Novartis

Welcome everyone to the Emerj AI in Business podcast. Today's guest is Kellen Winters, Director of Customer Success and Business Operations for AI at Novartis. Kellen joins us today to discuss why proving the value of AI investments begins with measurement, establishing a data-driven baseline and anchoring each use case in the business problems where work stalls before any technology is selected. Just a quick note for our audience that the views and opinions expressed by Kellen on today's program are her own and do not reflect those of Novartis or its leadership.

According to Nielsen, 91% of podcast listening happens alone, indicating deep, undistracted attention, ideal for complex B2B messaging. To learn how leading brands and AI startups connect with enterprise AI buyer audiences at scale, download our media kit at Emerj dot com slash AD one. That's E M E R J dot com slash A D one. Now the conversation with Kellen. Kellen, welcome to the show.

Thank you so much for having me. I'm really excited to be here.

Yeah, you planted a seed in my brain last week, and ever since then, I cannot stop thinking about the amount of money moving through a marketing organization's vendor and contractor line. Before anyone actually asks, listen, is this working? Um, what, what feedback do we have on this? And and it almost felt like, okay, this is invisible spent until someone really looks into it, which is a very scary concept. Scary concept. So before I break my brain about this, I'm gonna ask you to dissect the problem first and tell us how would you describe the challenge in large pharma marketing organizations in terms of proving their return on investment for their AI and technology parts of that?

Yeah, great question. And again, it's so awesome to be here with you all to share some of the ideas and things that we've been seeing across the enterprise over the last two to three years. I've been working in this space and I I think that this is not an old problem that we're having. So when we think about technology investments and we're using these within enterprises, it's thinking through how are we actually driving a return on an investment in that technology?

And I think with the rise of AI, this has become much more on the forefront. So this has been a problem for, I would say, quite some time. But with these increasing investments in AI and the increasing pressure to embed AI in existing processes across any enterprise, agnostic of pharma, I think that we're starting to see and have more of these real conversations.

And I'm sure with starting those conversations, resistance is a big part of that, especially walking into legacy organizations. Well, I would imagine. Would you say the resistance is more about the money spent, or more about the admitting that it wasn't monitored?

I think it's it's a little bit of both. I don't know that there was necessarily that pressure within any components of the business to really sit down. And of course, I'm not speaking for any enterprise, but with a lot of the clients that I work with, that pressure to really show the return on investment on the spend was never really hammered into an organization. But again, I think with the pressure on AI, for the pressure on how to think about it and how to effectively apply AI, is just going to continue to increase as AI becomes more embedded within any existing process, um, across enterprises.

Is this a pharma specific blind spot, or have you picked up on this across different industries? Because something that our audience members might not know about is that you also con consult. So this is not just looking from a specific industry's perspective. Is this an entire world thing or just a pharma specific blind spot?

Yes, and we started to see this really early on, cause before I started my own consulting work, I was working at Microsoft for about six years. So when we were taking Copilot to market, one of the biggest components of customer feedback that we were hearing was like, hey, I don't necessarily care, and I mean that in full honesty, we don't necessarily care if I'm saving people five minutes of time a day. So really show me on paper something that's tied to my existing workflows, my existing business that justifies this huge spend that I'm gonna put on my organization.

And so we really had to go back to the drawing board and say, hey, how do we make this real for an enterprise? And and today you see things like the business value dashboard came out of a lot of those conversations at Microsoft that we have that really allows customers to plug in their existing workflows, what KPIs those are tied to, to really understand if the AI is driving the enterprise value that our AI vendors are touting towards us.

And also, are we choosing? are we even choosing the right workflow to align AI to? So I think we see a lot of people snapping to AI for everything. But then the real question is, is are we choosing the right workflows? Are we aligning it with the right technology? And and how do we scale that to ensure that we're really driving R O I?

So walking into a conversation with leadership, trying to, like you just said, justify how it's going to impact workflows in a positive way. What is that trigger? What does someone on the leadership team really need to hear? Is it one thing? is it many things? Is it specific to the industry?

Um, I think it's a series of conversations, like with a lot of my customers, what we really do is just sit down and really take a look at the reality of the business. So, for example, within Novartis, if I give you just a snapshot within the last year, we saw almost 200% increase in folks wanting to use AI for things like agents. They had net new use cases, they wanted to use net new tools, they wanted API plugins, so there was all this internal demand and ideas that folks had that they wanted to snap AI to.

So what we did is sit down with our leadership teams to say, hey, where do we really see things bottlenecking? Where do we see things stalling within your existing processes? And is there an opportunity for us to tie AI to those broken workflows, if you will? So from there, what we were able to do is kind of stack rank the top seven or eight different areas where we saw those opportunities. Really whittle it down to two where that we could really show and measure the incremental value over time.

And then from there, we're able to kind of figure out how we're going to build and scale for these two particular workflows that we wanna drive that R O I within the organization.

That is the perfect spot for us to bring in the next question and next phase of this conversation, and that is about how to decide how to prioritize. Because I think knowing that you're overspending is one thing, but changing the way that a team decides what to work on next is a completely different muscle. And you have some experience in building that muscle for existing teams that are so used to to the muscle that they had.

And this is not just thinking about spreadsheets or adding more spreadsheets to the conversation. What has to change in how a marketing organization evaluates and prioritizes its work before it can start measuring any of this in an honest way?

That's a great question. I think first and foremost, it comes down to measuring anything at all. So I've worked with a lot of organizations where there is no measurement system, where we're not thinking in a metric metrics first organization. And coming from a place like Microsoft, I mean, we live and breathe by the data. So when I started working with other companies and seeing like, okay, we're not really measuring anything or the efficacy of any of our workflows at the moment.

We're kind of just relying on blind faith and snapping tech to those problems, not necessarily the most effective way to do it. So I think the first thing that we really need to coach organizations to do is is really be data driven, truly data driven. And there's a lot of challenging aspects of that, because there's disconnected systems, especially within regulated industries like pharma. But it's how do we bring that all together to understand where we're starting, and then how do we strategically prioritize these use cases based on where we see things breaking or fragmented along the way?

And then in terms of picking a workflow to start with, how do you make that decision? How do you decide this is where we're going to start? This is where we're gonna kick it off? That's a scary concept.

Yeah, generally, I think if you talk to, you know, a C M O of an organization or a C T O, they'll generally kind of have a rough idea of where things are breaking. And again, that's why the data is so important. So are we investing a ton of, are we putting a ton of money in things like headcount or additional resources to solve a problem? Are there very manual processes that we're conducting through our organization today?

Are there broken and fragmented processes across the organization that could help us work better together more across functionally? And I think a lot of those conversations happen with us at Novartis just by sitting down and having real conversations about where we think things can improve. And then the tech comes, right? I think it's all about starting from a business lens and from a business problem perspective. And generally, that's a more comfy place for people to start than to start thinking about how we create a system of agents and all these other things.

So I think it really starts with your bottom line and then really examining where things tend to break or bleed. And I think that's it's quite easy for an executive to start to point those out, and then we kind of go from there.

Saying that, you know, starting at the bottom line, it sounds like, and this is probably a theory in my mind, I wanna double check in with you. It sounds like if you really start with the problem in those conversations first, not necessarily the s the solution, not coming with, oh, look at what this a I agent could do. And walking in with the we really have a problem that it will minimize the resistance more. Has that been what you've seen happening, or how do we limit the resistance?

Yeah, absolutely. I would say that starting with that business problem first and foremost, not only is it gonna be easier for an executive to digest where things are breaking or bottlenecking within their existing processes or their exhausting resources. So having that question from the business lens has always been kind of the way for us to understand. Then, all right, well, what technology do we really need to help augment that? And with A I, I think it's just become more important because the level of investment is is so great, right?

So I think there's just additional pressure there for folks to get it right. And it's hard because this is the first time for any of us to be working with this technology. So it's a bit of a experimentation phase, but we're learning so much as we go, which is really, really exciting. Because you could really prove that value if we approach this in the right way.

An interesting encounter that I had with a listener recently was on a similar topic where they said no picking the first workflow to deploy a A I in or to pilot this or whatever. It normally goes, the priority framework normally goes to. It's the person that is the loudest in the room, the person that complains the most, the per person that's really verbal with their their concerns and their problems in the workflow. How do we prevent that from distracting us from the silent complainers?

And what's actually really fragmenting systems and making things impossible or unnecessarily difficult?

Yeah, it's a great question and perspective. I think, you know, and it wasn't necessarily the loudest people, but at Novartis again, we saw such an increase in the demand from folks all across the organization that had really meaty problems. So again, that goes back to okay, us as executives and leaders within this business really need to set the precedent of what is the most important workflow or where, or what I should say, are the primary workflows that are breaking.

And then, you know, once we get that demand coming in from all different sides of the organization, it becomes a little bit easier for us to kind of prioritize what's gonna be the most important and effective for the organization. But simultaneously, we don't wanna shut people out or say that your ideas don't matter. So what we're actually working on now is an intake agent for all these different ideas coming through into the company. For us to not only kind of take a look at what is coming through that top of the funnel, but also make sure that we're not missing anything that's jarring or tripping people up, or we're not missing any of the core priority workflows that we need

to really look into. So I think it's twofold. It's again always resting on those priority workflows that we've determined alongside of our execs. But then how do we make sure that we enable the broader organization to think in an A I first way and for them to continuously think that way? Cause we're not gonna see everything and hear everything, so we need to have that pulse at the same time.

The concept of having an intake agent is so genius. I imagine it as the suggestion box of 2026, like that's what the future of the suggestion box looks like.

Yeah, and we started with just like a raw Excel sheet. So again, when I started within the org, it was really just people feeding us ideas again. We're getting emails, pings from everybody who has these ideas, and we stood up a quorum where we would meet every single month. And now this has kind of transformed into an agent. Because not only again, to your point, is it empowering people to really think in an A I first way, in tandem with all the great training and enablement that we're doing.

But also then we have a kind of data backed way to evaluate anything that's net new, that's coming in that we may have missed, and also a way to track and measure that R O I and progress of each of the use cases that we are prioritizing along the way. So it's not only just the intake piece, but once we start to work through the P O C phases of this, we're actually able to track that incremental value over time, which is gonna be tremendous for us because we finally have real pen to paper.

Because the other thing we were seeing is everybody is gonna say initially that they're gonna save your $5 million with their agent. But I think this is more of a real way to incrementally measure the value that our A I systems are bringing us over time.

It kind of sounds like the spending problem and monitoring the spending and monitoring the return on investment is not something that starts the moment a deployment goes live or something is actually already working. It starts right from the very first suggestion, um, tracking it from there. And I can imagine that marketing leaders listening to this recognize exactly what you're describing, and they're feeling the same pain points. So if you were talking to another peer at a big legacy organization, sitting on a pile of vendor and A I spent that they can't really account for and they don't really know where to start, what would you to tell them to do?

First, to sort out the current mess and from here on forward, to prevent it from just happening again.

Oh wow, so I would say the first thing is to really think about how to create a data first and a data driven organization. I think that living and breathing by those numbers is probably the most important thing for folks to think through in this new age of technology and all these different investments coming from many different angles. I also think with respect to your vendors, okay, this is another interesting one. It's it's thinking through, okay, how do I create guardrails for those vendors?

So we understand what they're using A I to create versus not, and that also will impact your spend. I think then evaluating your business cases, what's duplicative, what's driving the most spend again, sitting down and having that conversation with your peers to determine that is probably the most important thing in creating a plan on how to track that spend actually regularly and track the value over time. But then also just figuring out how to be agile and all that, and it's a lot of work, it's not easy at all.

It's a very iterative process. One of my mentors at Novartis said to me in the most earnest way that we're always gonna be a little bit behind. And I think that's a very real way of putting this. And that's not bad, right? I think that allows us to learn over time and just to create a little bit of a better system around a very legacy process. And I think A I is forcing companies to do that in a way that we weren't doing it before.

One last one before we wrap up, Kellen and I. I'm sure that this is not a one size fits all, but generally speaking, how long does it take us to get to that point where we have a a process in place that works and everyone is on board?

I would say starting from zero, I would say it will be like, it's, I don't know. It's taken us roughly nine months to a year to really start ironing everything out and getting what we need. And then in addition to that, I I don't think it's a one and done. It's gonna be iterative over time. Um, in terms of the savings that we're trying to project, we're saying through twenty thirty. So we have pretty ambitious projections with the savings across all my clients, but that's kind of the baseline that we're trying to project.

I I would say give it nine months to a year initially, and then, you know, think through the next five. How are you gonna transform your org through the next five years with this technology? What is it gonna be? And and how do we really prioritize that and monitor the value realization in a real way over that time?

I'm sorry, I know I said last question, but I'm gonna give. This is the last, last one. At what point during cause, obviously nine months to a year is a long time and we would want to have some kind of measurability before we reach the end of a year. At what point during the process of having all of this sorted, will we be able to start measuring results?

Great question, and I feel like my first response probably scared people. But I would say, you know, if we think through our build process, generally, I would say if we're running a P O C market for ninety days, what is this thing delivered in ninety days? And I think that's why the importance of data, your K P I s, how we wanna measure this for the growth of the business is so so important. Because transparently coming into that partnering with a technology partner, sometimes, you know, if we're not tied to a K P I, we could be taken advantage of in terms of spend, things like that.

So it's really important to think through what exactly you're trying to drive. And if the tech can really prove that out within the first ninety days of piloting this thing.

Well, Kellen, some great practical insights and guidance for our audience members. I'm sure they're going to love listening to this episode. Thank you for joining us today and for sharing so much.

Yeah, thank you so much for your time today. I really appreciate it.

Wrapping up today's episode, let's look at the three key takeaways from our conversation with Kellen. First, measurement has to come before technology. Organizations without baseline data on how their current processes perform cannot credibly prove AI value. Building a data driven foundation is the first priority. Second, AI priorities should start from the business problem, where processes bottleneck, rely on manual effort or absorb headcount and be narrowed to a few use cases whose incremental value can be measured over time.

And finally, every pilot should be tied to an existing K P I and expected to show results within roughly ninety days, which guards against inflated vendor claims and unchecked spend. According to Nielsen, 91% of podcast listening happens alone, indicating deep, undistracted attention, ideal for complex B two B messaging. To learn how leading brands and A I startups connect with enterprise A I buyer audiences at scale, download our media kit at Emerj dot com slash A D one.

That's E M E R J dot com slash A D one. For further executive level analysis and to join our network of leaders delivering workflow impact with A I, visit Emerj dot com. On behalf of the team at Emerj, we'll see you on the next episode.