自律稼働するAIを信頼するために本当に必要なこと――CVS Health のYogesh Bhagat 氏に聞く

Welcome, everyone, to the Emerge AI in Business podcast. Today's guest is Yogesh Bhagat, Vice President of Digital and Technology at CVS Health. Yogesh joins us to discuss what it takes for customer-facing AI agents in healthcare to move from pilots to autonomous operation and why no agent should launch with autonomy, it has to earn it by giving consistent answers however a question is asked. Just a quick note for our audience that the views and opinions expressed by Yogesh on today's program are his own and do not reflect both of CBS Health or its leadership. If you offer AI products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title and the right type of organization and of course readiness. Emerge attracts VP and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional IT. To learn the exact strategies we use to help leading AI brands and startups

connect with their ideal enterprise AI buyers, visit Emerge.com slash AD1. That's E-M-E-R-J dot com slash A-D-1. Now the conversation with Yogesh. Yokesh, welcome to the show. Good morning, Shalinda. Happy to be here. So excited to talk about my experience about AI. Yeah, and I think today is going to be a practical conversation, and we're going to start at quite practical as well, because I think so many people will have some kind of phone story, an IVR story, right? You phone in, and it says press 1 for this and press 2 for that, And eventually you're shouting at the phone saying, I want to speak to a human. And we've seen teams switch out things to replace it. We have seen so many teams having to go through these processes.

And some succeed, some fail. And this is something that you've been through more than once now. So when you're building AI that sits right in front of customers, where does the real friction show up for us before that system even goes live? That's an interesting and loaded question. Friction has a different meaning for different kind of industries. For me, in healthcare, friction is not about a user experience issue just. It is about access to care, affordability, and patient safety. Now, coming to the point of friction, before even something goes live, I think there are two fundamental things which we need where we face friction. One is governance model, and second is run cost. So let's take a simple example of pharmacy management system. We receive a prescription. We edit it. We go through a verification process, fill the bottle, dispense it to the customer. Now, this involves hundreds of tasks. Every task has a different level of complexity. But when you are trying to build an AI agent around it, the level of governance doesn't change.

You need people from security. You need biasness validation. You need to have observability. then you need to make sure that answers are consistent. It's more like a deterministic, not the probabilistic approach. So all those things creates a governance model. And the natural tendency for us is, hey, let's bring experts from every field and every agent which is going to go live is going to go through this governance cycle. That is called governance by process and which will slow down and which will create friction for you. What we have done is, hey, we don't need our engineers to learn how to build observability. We don't want every engineer to focus on security. We don't want every engineer to focus on guardrails. First thing we said is we need to go governance by rigor. And what it means, we actually build an enterprise AI development platform,

which brings security, guardrails, and observability by default for creating any agent. Now, pretty much you have taken a lot of tasks which humans would have done, being done by some automated process, and it is validated and it is tested. So you validate the platform, not every agent which goes. So that's the first step of a friction, which you can remove by having the right tools and building the right processes around it. Now second step is if you look at these processes in any system, complexity we talked about. Few complexities like reading a prescription from a fax might be easy to automate, create an autonomous agent. Whereas agents where you going to touch patient eligibility patient co amount or dosage those are difficult because that directly impacts health So what we decided is we are going to go in a very different approach

Nobody will have a right to launch an autonomous agent. You have to launch an agent and earn the respect. How do you earn the respect? You need to demonstrate that every action you are taking or Every response you are giving is consistent, even if asked in a different way. And you need to have a way to measure that. Once you are able to do that, you have proven that you are no more a problemistic agent. You can run autonomous. So that's our gut rail. I'll give an example like DOV check. You can do it. Once we validate that it is always doing it, we remove the larger model out of context, We remove human in-loop and we know that it is always working fine. So these are the small things which runs in governance. Second, I talked about our run cost. Let's understand, all of us build any use case for having a ROI work for us. I'll say everyone tells me, MIT has published a report where 95% of AI project fails.

But that's a theory. How do you attack that problem? You need to think while solving any business problem, there are things you are going to automate. You are going to run safety check with models. You are going to run judge models for result validation. So it is going to cost you. If you don't think about that cost up front, you can't solve it. And how we did it is when we looked at our IVR calls, which is million plus call every day. How do we solve it? we hosted 25 SLMs in our data center working in conjunction with the LLMs. What it helped us? It reduced our costs by 5 to 6x. In addition, it also helped us to meet the sub-second response time which any IVR system would require. So I think these two are the main friction points before you even run the things. And if you don't care about it,

you might move fast, but that fast will have a roadblock very soon. That makes sense. And thinking about, you know, with the governance, you said that testing is going to earn us the right to automate. But how much of the friction comes from testing? Because like you mentioned, not everyone's going to ask the same question in the same way. The date of birth question is a great example. If you ask me for date of birth, some people will answer it date, month, year. Others will say month, date, year. So is there a lot of friction that comes in with that testing? It's a great question, Nancy. See, I think testing itself or while you are building an agent, no longer a testing is a traditional way. Nothing fails in the same way. If you go in a traditional software, it fails in the same ways every time. But agents doesn't fail. Same way, it's a probabilistic system. You can get a failure from multiple points. So what you need to do is build an agentic AI testing approach as part of your development lifecycle itself. What we have done is every agent we build, our developer platform also builds your agent to test it.

I'll give you an example. We say, hey, agent, you are an X person, old age person, and you have X number of basically healthcare issues. And you're asking a question to an IVR. What kind of questions will you ask? And when will you get frustrated? How will you show your frustration? So agent is generating those test cases, and we solve that problem before it goes into a production. Second step is about judge models. What you do is your main models, which are generally a smaller models, are giving an answer, and then you are cross-validating the output just by larger models. So as you see, as you see maturity, these judge models will go out of life because your smaller models would have learned everything. That's how I think you get to reduce the testing effort and overall fuel on the right. Wow. And that is exactly what I was going to ask next because I want us to get into the point of, okay, you know, we've tested. It's taken real-life calls.

How do we know whether it works the way we should? And you've already started answering that. And when we're saying the big models and the smaller models and, you know, taking out the large ones and then leaving the small, are we also thinking cost there? I'm assuming that bigger models mean more money. Frontier models, if I would have gone frontier model, then I would have not been able to make it live. It would have only been a pilot project, would have never gone live. But I think in case of IVR, it is beyond cost. Because in IVR, as we are talking even, if I don't speak for another five seconds, you will feel like network is not working and you will hang up on me. So you need a sub-second response time, and that sub-second response time is where smaller models comes. Even sometimes you will replace few things with ML models So that the learning which you built and you remove use larger model only for reinforcement learning or feedback loop So how we do is everything we do have two reviews. One is a judge model and second is a human in loop, which is an escalation path

for us. These two reviewers do not just act as a security check for us. They act as the feedback loops so that we can train our smaller models constantly and it is improving constantly. This is where you are measuring their deterministic nature and that's how step by step you will remove larger models from real environment and you will only use later on for reinforcement learning. How do we know that the caller is not feeling that switch to a larger model? Is that something that happens during the call or is that something that we sort out way before we get to that point? So there is a clear escalation path which you need to build. And like example, I'll tell you, we have a color path way to know that there is combination of frustration, combination of a repeat of a question and different scenarios, which let lower model know that if the question is getting repeated twice, then I need to escalate to a larger model and validate my response.

And if that also says it's not what response is right, then there is an escalation path to a human in loop. Biggest thing here is you need to make sure that there is no biasness while you're doing it, because this is about access to care. So it's not just about escalation path, but also about ensuring there is no biasness in this escalation path. Yeah, no, absolutely. When we're saying there needs to be a human in the loop and there's a feedback loop going back to the human, who's the human in charge? Is it going to be engineers? Is it the contact center floor? Is it compliance? Who gets to be involved in that? The human in the loop is the agents who are finally picking up the customer calls. What questions were asked from them? What were their responses? Why were we not able to answer? And that recording also goes as a feedback loop. So in this case, it's a real human who is solving final escalation path. Great. So it goes back to, I would say, contact center floors,

and I guess then certain scenarios would be used for training purposes as well and giving feedback to the engineers as well as to where we need to maybe up governance or change the way it interacts. Am I getting the right idea? Absolutely, because once you take that transaction, you put it into a reinforcement model of models, engineers look at it? Do they really need to change the rag or do they need to change some kind of a prompt? What is the real need? And that's where the engineer comes into a picture. It makes a lot of sense. So we're getting to the part of the conversation where we get to tell our leaders, thinking about replacing IVRs and thinking about this entire process, they need to know where to start. They need to know what to look out for. So for a leader about to hand a major customer facing process to AI for the very first time, what would you let them or want them to have in place before they trust it to run on its own? What do they need to get to make sure that they earn the right of automation? I'll say first, we already talk enterprise AI development platform, which allows all engineers,

your average engineer to best engineer to build AI agents in one consistent way. That's a table six. You need to build that. second within that platform you also need to build a testing approach deterministic testing approach doesn't work we already talked about it i will not repeat it third is ability to replay the transaction or observability that's a crucial piece when you are talking with millions of customers any one escalation comes you need to know out of 100 agent, which agent answered, which particular task was deterministic in nature, where the path went wrong. And if you don't know how to do that, and it takes many, many hours to just find out one use case, then you are not ready for a go live. So you should be able to find out gap in your process in seconds, not in even minutes. So that's a key piece. Then you need to know customer experience.

Are you working for customer-facing system where it is a sub-millisecond response is required? Like, example, IVR versus a chatbot. Chatbot is a place where you can get two seconds because you can say, I'm triangulating, I'm doing this. People are so used to chat, GPT, Claude that they are okay with the time. If you know that, then you need to pick right models which can answer or meet your things or you can have engineering solutions like async way of working or whatever you like there are different ways to solve that problem then economics i will stress this part a lot of time because everybody is tilted i will apply a one cloud and have open ai one model frontier model and the problem will be solved but it will not be solved in an economical way That's a crucial key to success. If you have not solved about that problem, then you are going to face issue after pilot.

So that's a core, I believe. Would you advise our listeners to start with their main business problem or their main friction point, or would you advise them to start on a narrower path? Allow things like a customer phoning for store hours or just to follow up on whether a refill is already done. Start with narrower things before you expand. Get the testing done on those first. It's an interesting thing. It's a very debatable piece. We started the same thing. We were replacing the old legacy IVR, which used to say, press one. It's a pre-structure. And we said, hey, should we replace one? and then do step-by-step versus all at once. And what we learned is experience is so much broken between if you try to move from conversational AI to old IVR systems, they will not be able to communicate in the same way, the tone which you want to set for consumers, you'll not be able to measure. So our approach was go for full-blown, but go with a smaller set of our customers.

Learn from a smaller set of a customer. Give them a complete experience. And then build confidence. Build your learning models. Build the feedback loop in that smaller set of customers. Improve your containment numbers later on. First, focus on experience. Because what mistakes you can get into is, day when you launch with, let's say, 2% of your customers, you want the containment of that AI system to be better than the traditional IVR. That's a default every business user wants. No, that's not what you should focus on. First, focus on learning, observability, how you are going to improve the customer experience. Then, automatically, you will see the containment and satisfaction of a customer will increase. So, my suggestion will always go with it. Go small, but with full AI experience. Otherwise, it is difficult to change behavior of a customer. I like that. I think before we wrap up, I think I just want to recap for our audience that I think what stood out for me was when you said no one has the right to go autonomous.

We need to earn it. And the testing and everything that goes on behind it is super, super important, like you've just highlighted. How do we know that it works? We need to make sure that we do have that continuous feedback loop, that we are continuing to guide the agent and get it to a point where we know that we can trust it. And then let's just say start small. Start with a small sample of customers. Take them through the entire journey. Learn through that. It's part of your testing process. And that's how, like you said, how we'll gain confidence. And confidence goes into trust. And that's ultimately what we're paying for. Hakesh, anything that you'd like to leave our audience with? I would leave it with one thing. Autonomy is not earned, not launched. You need to really earn it in that way. Safety layer should be so cheap that nobody gets tempted to replace it. so make sure your economics play in your hand and safety is always first exactly and we'll leave it just there thank you so much Yogesh thank you

wrapping up today's episode let's look at the three key takeaways from our conversation with Yogesh First, autonomy should be earned rather than launched, with agents running on their own only after they give consistent, measurable responses however a question is asked. Second, governance scales when security, guardrails, and observability are validated once at a platform level rather than reviewed agent by agent. And finally, run costs have to be engineered from day one, because pairing smaller models with larger judge models and human escalation keeps safety affordable and responses fast enough for the customer-facing channels. If you offer AI products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title and the right type of organization and, of course, readiness. Emerge attracts VP and higher-ranking enterprise audiences who are already convinced that they need to move beyond traditional IT. To learn the exact strategies we use to help leading AI brands and startups connect with their ideal enterprise AI buyers, visit emerge.com.ad1. That's E-M-E-R-J.com.ad1.

for further executive level analysis and to join our network of leaders delivering workflow impact with AI visit Emerge.com on behalf of the team at Emerge we'll see you on the next episode