製薬マーケティングにおけるAIのROIを証明する――Novartisのケレン・ウィンターズ氏と語る

Welcome, everyone, to the Emerge AI in Business podcast. Today's guest is Kellan Winters, Director of Customer Success and Business Operations for AI at Novartis. Kellan joins us today to discuss why proving the value of AI investments begins with measurements, establishing a data-driven baseline and anchoring each use case in the business problems where work stalls before any technology is selected. Just a quick note for our audience, that's reviews and opinions expressed by Kenan on today's program are her own and do not affect those of Novartis or its leadership. According to Nielsen, 91% of podcast listening happens alone, indicating deep, undistracted attention, ideal for complex B2B messaging. To learn how leading brands and AI startups connect with enterprise AI buyer audiences at scale, download our media kit at emerge.com.ad1. That's E-M-E-R-J.com.ad1.

Now the conversation with Kelly. Kaelin, welcome to the show. Thank you so much for having me. I'm really excited to be here. Kaelin, you planted a seed in my brain last week, and ever since then, I cannot stop thinking about the amount of money moving through a marketing organization's vendor and contractor line before anyone actually asks, listen, is this working? What feedback do we have on this? And it always felt like, okay, this is invisible spent until someone really looks into it, which is a very scary concept. So before I break my brain about this, I'm going to ask you to dissect the problem first.

And tell us, how would you describe the challenge in large pharma marketing organizations in terms of proving their return on investment for their AI and technology part of that? Yeah, a great question. And again, it's so awesome to be here with you all to share some of the ideas and things that we've been seeing across the enterprise over the last two to three years I've been working in this space. And I think that this is not an old problem that we're having. So when we think about technology investments and we're using these within enterprises, it's thinking through how are we actually driving a return on an investment in that technology. And I think with the rise of AI, this has become much more on the forefront. So this has been a problem for, I would say, quite some time. But with these increasing investments in AI and the increasing pressure to embed AI in existing processes across any enterprise agnostic of pharma, I think that we're starting to see and have more of these real conversations.

And I'm sure with starting those conversations, resistance is a big part of that, especially walking into legacy organizations. Well, I would imagine. Would you say the resistance is more about the money spent or more about the admitting that it wasn't monitored? I think it's a little bit of both. I don't know that there was necessarily that pressure within any components of the business to really sit down. And of course, I'm not speaking for any enterprise, but with a lot of the clients that I work with, That pressure to really show the return on investment on the spend was never really hammered into an organization. But again, I think with the pressure on AI, for the pressure on how to think about it and how to effectively apply AI, it's just going to continue to increase as AI becomes more embedded within any existing process across enterprises. Is this a pharma-specific blind spot or have you picked up on this across different industries? Because something that our audience members might not know about is that you also consult.

So this is not just looking from a specific industry's perspective. Is this an entire world thing or just a pharma-specific blind spot? Yes. And we started to see this really early on because before I started my own consulting work, I was working at Microsoft for about six years. So when we were taking Copilot to market, one of the biggest components of customer feedback that we were hearing was like, hey, I don't necessarily care. And I mean that in full honesty. We don't necessarily care if I'm saving people five minutes of time a day. So really show me on paper something that's tied to my existing workflows, my existing business that justifies this huge spend that I'm going to put on my organization. And so we really had to go back to the drawing board and say, hey, how do we make this real for an enterprise? And today you see things like the business value dashboard came out of a lot of those conversations at Microsoft that we have that really allows customers to plug in their existing workflows, what KPIs those are tied to, to really understand if the AI is driving the enterprise value that our AI vendors are touting towards us.

And also, are we choosing? Are we even choosing the right workflow to align AI to? So I think we see a lot of people snapping to AI for everything But then the real question is is are we choosing the right workflows Are we aligning it with the right technology And how do we scale that to ensure that we're really driving ROI? So walking into a conversation with leadership, trying to, like you just said, justify how it's going to impact workflows in a positive way. What does that trigger? What does someone on the leadership team really need to hear? Is it one thing? Is it many things? that's specific to the industry? I think it's a series of conversations. Like with a lot of my customers, what we really do is just sit down and really take a look at the reality of the business. So for example, within Novartis, if I give you just a snapshot, within the last year, we saw almost a 200% increase in folks wanting to use AI for things like agents.

They had net new use cases. They wanted to use net new tools. They wanted API plugins. So there was all this internal demand and ideas that folks had that they wanted to snap AI to. So what we did is sit down with our leadership teams to say, hey, where do we really see things bottlenecking? Where do we see things stalling within your existing processes? And is there an opportunity for us to tie AI to those broken workflows, if you will? So, from there, what we were able to do is kind of stack rank the top seven or eight different areas where we saw those opportunities, really whittle it down to two, where we could really show and measure the incremental value over time. And then from there, we're able to kind of figure out how we're going to build and scale for these two particular workflows that we want to drive that ROI within the organization.

That is the perfect spot for us to bring in the next question, the next phase of this conversation. And that is about how to decide how to prioritize. Because I think knowing that you're overspending is one thing, but changing the way that a team decides what to work on next is a completely different muscle. And you have some experience in building that muscle for existing teams that are so used to the muscle that they had. And this is not just thinking about spreadsheets or adding more spreadsheets to the conversation. What has to change in how a marketing organization evaluates and prioritizes its work before it can start measuring any of this in an honest way? That's a great question. I think first and foremost, it comes down to measuring anything at all. So I've worked with a lot of organizations where there is no measurement system, where we're not thinking in a metrics-first organization. And coming from a place like Microsoft, I mean, we live and breathe by the data.

So when I started working with other companies and seeing like, okay, we're not really measuring anything or the efficacy of any of our workflows at the moment. We're kind of just relying on blind faith and snapping tech to those problems, not necessarily the most effective way to do it. So I think the first thing that we really need to coach organizations to do is really be data-driven, truly data-driven. And there's a lot of challenging aspects of that because there's disconnected systems, especially within regulated industries like pharma. But it's how do we bring that all together to understand where we're starting? and then how do we strategically prioritize these use cases based on where we see things breaking or fragmented along the way. And then in terms of picking a workflow to start with, how do you make that decision? How do you decide this is where we're going to start, this is where we're going to kick it off? That's a scary concept. Yeah, generally I think if you talk to a CMO of an organization or a CTO,

they'll generally kind of have a rough idea of where things are breaking. And again, that's why the data is so important. So are we investing a ton of, are we putting a ton of money in things like headcount or additional resources to solve a problem? Are there very manual processes that we're conducting through our organization today? Are there broken and fragmented processes across the organization that could help us work better together more cross-functionally? And I think a lot of those conversations happen with us at Novartis just by sitting down and having real conversations about where we think things can improve. And then the text comes, right? I think it's all about starting from a business lens and from a business problem perspective. And generally, that's a more comfy place for people to start than to start thinking about how we create a system of agents and all these other things. So I think it really starts with your bottom line and then really examining where things tend to break or bleed.

And I think that it's quite easy for an executive to start to point those out. And then we kind of go from there. Saying this, you know, starting at the bottom line, it sounds like, and this is probably a theory in my mind I want to double check in with you. It sounds like if you really start with the problem in those conversations first, not necessarily the solution, not coming with, oh, look at what this AI agent could do and walking in with a, we really have a problem, that it will minimize the resistance more. Has that been what you've seen happening or how do we limit the resistance? Yeah, absolutely. I would say that starting with that business problem, first and foremost, not only isn't going to be easier for an executive to digest where things are breaking or bottlenecking within their existing processes or their exhausting resources So having that question from the business lens has always been kind of the way for us to understand that, all right, well, what technology do we really need to help augment that? And with AI, I think it's just become more important because the level of

investment is so great, right? So I think there's just additional pressure there for folks to get it right. And it's hard because this is the first time for any of us to be working with this technology. So it's a bit of a experimentation phase, but we're learning so much as we go, which is really, really exciting because you could really prove that value if we approach this in the right way. An interesting encounter that I had with a listener recently was on a similar topic where they said, no, picking the first workflow to deploy AI in or to pilot this or whatever, it normally goes, the priority framework normally goes towards the person that is the loudest in the room, the person that complains the most, the person that's really verbal with their concerns and their problems in the workflow. How do we prevent that from distracting us from the silent complainers and what's actually really fragmenting systems and making things impossible or unnecessarily difficult? Yeah, it's a great question and perspective. I

think, you know, and it wasn't necessarily the loudest people, but at Novartis, again, we saw such an increase in the demand from folks all across the organization that had really needy problems. So again, that goes back to, okay, us as executives and leaders within this business really need to set the precedent of what is the most important workflow or where or what, I should say, are the primary workflows that are breaking. And then, you know, once we get that demand coming in from all different sides of the organization, it becomes a little bit easier for us to kind of prioritize what's going to be the most important and effective for the organization. But simultaneously, we don't want to shut people out or say that your ideas don't matter. So what we're actually working on now is an intake agent for all these different ideas coming through into the company for us to not only kind of take a look at what is coming through that top of the funnel, but also make sure that we're not missing anything that's jarring or tripping people up or we're not missing any of the core priority workflows that we need to really look into.

So I think it's twofold. It's, again, always resting on those priority workflows that we've determined alongside of our execs. But then how do we make sure that we enable the broader organization to think in an AI-first way and for them to continuously think that way? Because we're not going to see everything and hear everything. So we need to have that pulse at the same time. The concept of having an intake agent is so genius. I imagine it as the suggestion box of 2026. Like that's what the future of the suggestion box looks like. Yeah. And we started with just like a raw Excel sheet. So again, when I started within the org, it was really just people feeding us ideas. Again, we're getting emails, pings from everybody who has these ideas. And we stood up a quorum where we would meet every single month. And now this is kind of transformed into an agent because not only, again, to your point, is it empowering people to really think in an AI-first way in tandem with all the great training and enablement that we're doing,

But also then we have a kind of data-backed way to evaluate anything that's net new that's coming in that we may have missed and also a way to track and measure that ROI and progress of each of the use cases that we are prioritizing along the way. So it's not only just the intake piece, but once we start to work through the POC phases of this, we're actually able to track that incremental value over time, which is going to be tremendous for us because we finally have real pen to paper. Because the other thing we were seeing is everybody is going to say initially that they're going to save your $5 million with their agent. But I think this is more of a real way to incrementally measure the value that our AI systems are bringing us over time. It kind of sounds like the spending problem and monitoring the spending and monitoring the return on investment is not something that starts the moment a deployment goes live or something is actually already working.

It starts right from the very first suggestion, tracking it from there. And I can imagine that marketing leaders listening to this recognize exactly what you're describing and they're feeling the same pain point. So if you were talking to another peer at a big legacy organization sitting on a pile of vendor and AI spent that they can't really account for and they don't really know where to start, what would you tell them to do first to sort out the current mess and from here on forward to prevent it from just happening again? Oh wow So I would say the first thing is to really think about how to create a data first and a data driven organization I think that living and breathing by those numbers is probably the most important thing for folks to think through in this new age of technology and all these different investments coming from many different angles

I also think with respect to your vendors, okay, this is another interesting one. It's thinking through, okay, how do I create guardrails for those vendors so we understand what they're using AI to create versus not? And that also will impact your spend. I think then evaluating your business cases, what's duplicative, what's driving the most spend, again, sitting down and having that conversation with your peers to determine that is probably the most important thing in creating a plan on how to track that spend actually regularly and track that value over time. But then also just figuring out how to be agile and all that. And it's a lot of work. It's not easy at all. It's a very iterative process. One of my mentors at Novartis said to me in the most earnest way that we're always going to be a little bit behind. And I think that's a very real way of putting this.

And that's not bad, right? I think that allows us to learn over time and just to create a little bit of a better system around a very legacy process. And I think AI is forcing companies to do that in a way that we weren't doing it before. One last one before we wrap up, Kevin, and I'm sure that this is not a one-size-fits-all, but generally speaking, how long does it take us to get to that point where we have a process in place that works and everyone is on board? I would say starting from zero, I would say it will be like it's, I don't know, it's taken us roughly nine months to a year to really start ironing everything out and getting what we need. And then in addition to that, I don't think it's a one and done. It's going to be iterative over time. in terms of the savings that we're trying to project. We're saying through 2030. So we have pretty ambitious projections

with the savings across all my clients, but that's kind of the baseline that we're trying to project. I would say give it nine months to a year initially and then, you know, think through the next five. How are you going to transform your org through the next five years with this technology? What is it going to be? and how do we really prioritize that and monitor the value realization in a real way over that time? Sorry, I know I said last question, but I'm going to get – this is the last, last one. At what point during – because obviously nine months to a year is a long time, and we would want to have some kind of measurability before we reach the end of a year. At what point during the process of having all of this sorted will we be able to start measuring results? Great question. And I feel like my first response probably scared people. But I would say, you know, if we think through our build process, generally, I would say if we're running a POC, mark it for 90 days. What is this thing delivered in 90 days?

And I think that's why the importance of data, your KPIs, how we want to measure this for the growth of the business is so, so important. Because transparently, coming into that, partnering with a technology partner, sometimes, you know, if we're not tied to a KPI, we could be taken advantage of in terms of spend, things like that. So it's really important to think through what exactly you're trying to drive and if the tech can really prove that out within the first 90 days of piloting this thing. Well, there are some great practical insights and guidance for our audience members. I'm sure they're going to love listening to this episode. Thank you for joining us today and for sharing so much. Yeah, thank you so much for your time today. I really appreciate it. Thank you.

and be narrowed to a few use cases whose incremental values can be measured over time. And finally, every pilot should be tied to an existing KPI and expected to show results within roughly 90 days, which guides against inflated vendor claims and unchecked expense. According to Nielsen, 91% of podcasts listening happens alone, indicating deep, undistracted attention, ideal for complex B2B messaging. To learn how leading brands and AI startups connect with enterprise AI buyer audiences at scale, download our media kit at emerge.com slash ad1 that's emerj.com slash ad1 for further executive level analysis and to join our network of leaders delivering workflow impact with ai visit emerge.com on behalf of the team at emerge we'll see you on the next episode Thank you.