AIが暴いたWebセキュリティの欠陥「PHP2Shell」と、音声だけで学習した新しい音声モデル

I just think it's incredible that someone who wrote some code to not have to write any code for his personal homepage, you know, has exploded into powering 70% of the Internet. All that and more on today's Mixture of Experts. I'm David Zaks and welcome to Mixture of Experts, IBM's podcast in which we bring together a panel of IBM experts to talk through the week in AI news. My guests today are Ash Minhass, a practice manager for technical content, Judith Aquino, staff writer at IBM Think, and Jessica Nahoulin, master inventor and lifecycle management team lead for IBM Business Automation Workflow. Later in the show, we'll also be joined by Q Gabriel, who is going to report from the field, and he is the IBM field CTO for data. He is at IBM Tech Exchange, which is our annual conference for builders, engineers, hands-on demonstrations. So we're looking forward to that. First, we're going to talk through three AI news stories. So we're going to talk about Desta, a new model from Block that was trained on voice conversation, no text.

We're going to talk about a new benchmark called cultural riddles that is meant to stump AI models with riddles that are culturally specific. But we're going to start by talking about a giant bug. First story is about PHP 2 Shell. This is a big bug that a couple security firms found. It's part of this broader trend of AI discovering security vulnerabilities. This one is particularly large. Judith, could I start with you? Could you help us get oriented? Like what are the two or three most important things we should understand about this PHP exploit? PHP is a widely used programming language that powers about 70% of all websites. And these vulnerabilities can allow an attacker to remotely take over a web server. But besides that, the problem is organizations that are running older PHP versions can't just upgrade and patch because that could disrupt other legacy systems built around it.

the somewhat good news, as our friends at Security Intelligence mentioned, is that the exploit chain isn't exactly easy to take advantage of, but it still shows how vulnerable we are, as given the fact that these security flaws were found with AI models, which are making it easier to find vulnerabilities to exploit. Ash, can you help me understand the scale of the problem? Why is this? Obviously, bugs are discovered every day, every week. Why is this one that began to sort of trickle into front page consciousness? You know, there's so much of the Internet that's powered on web servers that run PHP. And so this particular vulnerability allows an attacker to be able to remotely execute code on the server that is exposed to the Internet or, you know, other aspects of, you know, access that it may have. And so when you think about sort of a typical web architecture, you're going to have things like databases attached to that server in order to be able to query something and provide information to whoever the user is on the Internet.

And so it's the fact that you can take control of that thing and then have network access to networks you shouldn't have access to. You have access to credentials that you may not or shouldn't have access to. And you're able to just, in essence, use that computer as you would like to in any nefarious way that you can think of. So in essence, it's like having the keys to like someone else's like hotel room, for example, OK, long after, you know, you know, you should have and being able to do whatever you like there. And yet, I mean, that's ultimately the crucial vulnerability that's exposed here is like all these websites out there can have this problem. Now, you know, you'll find that like in large organizations, there'll be architects who will have architected their web servers to kind of like be segregated from a network perspective. Like they'll have them out in a DMZ. They'll have like web application firewalls in front of them and so forth.

But none of those defenses are going to be able to protect them from this kind of issue. And then, you know, if you think about other people who have PHP servers, you know, they might be smaller businesses or individuals. They don't necessarily have that complex a network topology, and therefore their servers are just basically sitting ducks on the Internet. So I think what I hear you guys saying is that there are at least two reasons that this is really, at a headline level, bad. One, PHP, it's not like most bugs. It's like this one software program is affected. Oh, Microsoft Word has this, you know, little problem and they'll push a patch and it'll automatically update and then it's no big deal. But this is actually, I gather, a bug that exists in this fundamental language itself, like not a product, but a language that undergirds this. It's like it's like a bug in the infrastructure almost of the web. Jessica, you am I right that you have I mean, you're a master inventor. You have many patents.

You build things. I'm gathering you've probably built websites that use PHP. How are you actually as a practitioner who's, you know, maybe put things out in the wild that are affected? How are you thinking about this right now? I think the way everyone needs to maybe change their mindset now is we've built like websites, solutions, whatever small businesses like Ash mentioned might have put up. We've built all these solutions. We've put them up there. We've put them on the web, and we've moved on. They're running well. We can drop them out of people maintaining them or even looking at them because they're all fine. Now we have AI, and with AI, we're able to, at this rapid pace, both discover vulnerabilities, attempt to exploit them. And now we have to revisit all these things we've somewhat abandoned and look at, you know, what dependencies do they use? was that built with PHP? Is that using some open service software that has only one maintainer and they only touch fixes once every two months?

We had to look at essentially our full stacks and see how we can modernize and evaluate what we use. So it's honestly, so that's from both small individual businesses to big enterprises, that's a huge cost. Before modernizing was almost like this nice to have. Now everyone is forced to modernize so that we are ready to pick up the latest versions of these fixes and these dependencies without breaking our infrastructure. I think one of the things that interested me about this story is the story of how the bug was discovered and how it was disclosed. And if you really look at the timeline, it's pretty complex. there were well first of all the bugs were sort of half understood or something like the bugs were understood in previous years but it wasn't thought to be quite as severe um then this summer uh independently a firm called whiz and a firm called california um uh started directing their ais to

sort of uh look poke at php and see if they could find uh flaws and uh they sort of simultaneously discovered these flaws with and California began talking to one another they began talking to the entity responsible for maintaining PHP code which is something called the PHP foundation that foundation it turns out despite managing the code that undergirds 70% of the web has one full-time employee meanwhile Amazon put out a honeypot server even before the the the exploit was released, somehow someone attacked the Honeypot server with this exploit, suggesting, what? I don't really know. What are your thoughts about that, Ash? One of the reasons why so much of the internet runs on PHP is probably WordPress. Okay? WordPress runs on it. Joomla runs on it. Okay? You know, I've had personal experience setting up all of these things for, like, friends and family members, as Jessica was alluding to, like, in past

lives. Um... The alarming thing, as you said, is that there's one full-time employee for the PHP Foundation. And I think that one of the things that we take for granted as a society of open source software is that if it works, we just use it. And there's an assumption that it's someone else's problem or responsibility to maintain and react to situations such as what's happened this week. And I think that there needs to be a rethink around how people who are consuming open source code are investing back into that ecosystem because quite simply, you know, there's time, money and resources required to solve these things. And with the advent of generative models and AI agents and finding vulnerabilities and exploits quicker than a human being can without an equal level of investment in that time and money and resource it will be very difficult for us to sort of like try to close that gap between discovering something and being able to resolve it the exploit in the parlance the common vulnerability the bug is called PHP 2 Shell But the language itself PHP

originates with someone invented it how many years ago, Ash? Rasmus Lerdorf made it, and he made it in 1994. And, you know, it obviously got a lot of traction, and he's a very, very competent programmer, but at the time he would joke and say, I'm not a real programmer, or I just threw this thing together and moved on, and he actually wrote it to program less. And so I just think it's incredible that someone who wrote some code to not have to write any code for his personal homepage has exploded into powering 70% of the Internet. Right. I mean, I sort of feel like it would be like waking up one day and being like discovering that all the concrete that New York City is built on was actually just like, you know, baked in someone's backyard, and he was playing with his kids' Play-Doh, and he thought it'd be cool, and everyone's like, give me some. And then they built a city on top of it. And I don't know, like, my analogy is probably imperfect there,

but broadly, what does it mean when our digital infrastructure, like, if you start drilling down into the archaeology of it, often it's just something clutched together, like the kernel of it is something clutched together by some dude in 1994. Like, is that a problem we should be worried about? I think it's a big problem. So, like, I'll speak from personal experience here, okay? And, you know, when I say this, there's probably so many technical members in the audience who are going to listen to this and go, yeah, I know exactly how that feels. The world of technology is layers upon layers upon layers of abstraction, okay? What we do is we take complexity, we turn it into an abstraction layer and then we build on top of it and as the consequence of that i just think as a society we've never ever stopped and taken sort of a first principles approach to how we make software we just go oh in order to really need to make that thing i just have to import this library and i have to import this dependency and i have to import this thing

and like i don't really need to know how all those things work i just need to know if i import it and put, you know, the import name and run, it will do what I want it to do. And there's not enough, like, sort of, like, a sort of mental bandwidth there around what is the thing that I'm importing and where did I get it from and why should I use it and, like, you know, is that the right thing to use and, you know, can I trust it for the thing that I'm going to do next? I do wonder, like, running with the physical analogy, like, presumably when they decide to, presumably if I dig into the archaeological layers of this road outside my home, like, the further down you go, you'll find less sophisticated building materials, but presumably when they modernized it, they made structural engineering decisions to make sure that the whole thing is fundamentally sound. And, like, do these, could you tell me a little bit, just as somebody who's new to this world, about, like, practices of code modernization and, like, best practices of just sort of making sure that the kernel of a thing is safe? Jessica, do you have any thoughts about that? I would say there's the ideal world and then the reality.

So there's a lot of best practices around exactly what you described. You know, a lot of people do want to re-architecture these conflicts, like 30-year-old systems and see how we can do better. But then the reality is then there's timelines, there's resources, there's maintenance. And then just sticking glue and tape and everything else on top, it just becomes easier. and then you kind of have a little cupcake on top of your little broken bucket or whatever you got going. And then that's kind of the reality. But I do believe that AI is also giving us this opportunity now that now rewriting code and understanding code where the experts aren't even there anymore because, again, it was created maybe by some team 50 years ago and you don't have them to speak to. Now all that's so much easier with AI. So I do think, like, this news article exposes an issue that now we are finding issues fostering or ever,

but we're also capable of fixing them fostering or ever. In a moment, I'm going to move us on to our second story, which is a really interesting one about an AI model trained only on speech. Before we do that, if you happen to be listening to this, watching this on YouTube, it's really a big help if you could just click like, subscribe, maybe drop a comment, tell me what you think of my outfit today. It helps us, you know, engage the algorithm and grow the show. So thank you for that. All right, our second story today. This one, I really nerd out on this sort of stuff. Model architecture, new modalities for models. So this is about something called Dessa. It comes from Block, which already this is interesting to me. Block is Jack Dorsey's company. Block makes Square, the payments processor, Cash App, the other payments processor for cooler people, and also owns Tidal, the Spotify competitor. But if we know one thing about AtJack, he's always tinkering, always got things up his sleeve.

Apparently, five, six years ago, he bought an AI, frankly, it was kind of like a deepfake firm. They got famous for making a deepfake voice of Joe Rogan that just sounded just like him in 2019. So they've been cranking for six years, apparently. And they announced that they have created a model that was only trained on voice conversation. So unlike the LLMs that we're familiar with, the GPTs, the Clauds, those basically are really, really scholarly. They read a lot of text. this model did not read any text it only listened to human conversation and uh nevertheless was able to uh is able to hold a conversation and to do so in a way that uh well why don't let someone else explain that judith what what is so unique about desco were you able to listen to any of the the demos or were you able to yeah why don't you tell us what's kind of uncanny and cool about it first

thing is that it can listen and talk at the same time. So versus other models where they're silent and wait until you're done talking, from the demos, you could hear it interrupting people, even with just like a laugh or like the things that we tend to do when trying to show that we're engaged in a conversation. So that immediately jumped out at me. One of the coolest things it can do is say, hmm, while you're talking, right? Yes. So apparently linguists call this back-channeling. And, like, you know, if I'm talking to anyone and trying to show them that I'm engaged, I'll just go, hmm. And the fact that it can do that in, like, a socially graceful way is a massive technical feat, apparently. and it really comes down to training on that audio. So far, I mean, we should note that it hasn't been publicly released yet,

and we haven't seen really any research or much about the methodology behind it. So we're still just going on what's on their demo website. Yeah, Ash, was that curious to you? Because there's this, there wasn't a paper. However, there's also this this kind of model that can talk and listen at the same time is called fully duplex. I think it sounds like a real estate term, but it's about the I guess the dual channel and the model that can listen and talk at the same time. But there is a fully duplex. There's a duplex bench that they could have said that we scored off the charts in it. They did not share that score. They did not share any numbers whatsoever. But Ash, what do you think about this? And, you know, where does your skepticism lie, given that the details are scarce? The fact that they're saying that they didn't train it on any text suggests to me that the amount of world knowledge that this model has is more limited.

Because, you know, if you're training it on conversations itself, then you need lots and lots and lots of conversations. Right. And so as we would find, you know, with early like transformer models. I mean, this isn't a transform model, but it probably would exhibit the same sort of like behaviors, is that it probably would hallucinate a lot, I would say, because it doesn't have as much grounding. The thing that interests me more than anything is that we have all of these benchmarks for all of these models. Like, you know, you've got like MMLU and you've got like these coding benchmarks and so forth for, you know, the frontier models and any other model that they can run against it. I don't know if there is such like an audio benchmark. I mean, like, how do you benchmark and test this thing? OK, so even if they did want to release some thing in their research, it's like, what would be the benchmark that they would be using to do that?

It like I think right now one of the things that Dessa has to contend with is how do they show the world controllability of it evaluation you know how does it handle the hallucinations I mean I think all of those things are still open questions Apparently in someone demo it really likes to call people Reinaldo whether they named Reinaldo or not So it works really, really well if your name is Reinaldo. It aces the Reinaldo talking to Reinaldo benchmark. Maybe that's how they train it. You've just got to, like, I don't know, open up an app or a browser and just keep talking to it for the next couple of years. Eventually, it convinces you your name is Reinaldo, and then it works perfectly. Frankly, I was I was surprised to see AI news coming out of block. They weren't on my radar as a frontier lab, to say the least. But they've done something clearly impressive in the demo. But what's your reaction? Honestly, first, very impressed, because it's the first time I've even listened to an agent speak in that duplex wave.

But honestly, echoing Ash in that, the first thing I, as soon as I read that they, how much tech they essentially trained this model on, And my thoughts were, well, then it's everything I dislike in a model. It's inaccurate. It doesn't know the right answers. But the very thing we choose our model on, it fails at, right? Or I assume so. I have to believe that, you know, you're getting hallucinations. It doesn't know the answer anymore. So what value are you getting? So one thing when Ash was reading and he talked about the hallucinations, I was thinking, Maybe that's actually a good thing here because now you're getting like the randomization of a unique human personality in some senses. So it's also quite interesting because if I was to describe this to you, I'll be like, it can speak to you, but it can now interrupt you. That sounds great. I want I totally want to be interrupted by my LLM while I'm trying to ask it something, you know.

Um, so I guess a lot of skepticism on, um, its performance and like how much it knows, but I think it's unlocked this whole new avenue. And it's like, how do you even grade that avenue? Like Ash was saying. So, um, I was listening to him and I was thinking there's whole new criteria here because now we're not even grading it on, because usually when we want to grade our typically like Claude or our love models, we're grading it on how accurate is it? How right is it? how consistent is it reliably able to answer something. But here, maybe we want to grade that if I go, what does that movie, like it catches what I'm trying to think and it can interrupt and jump in. How do you grade that and evaluate that? It's a whole new criteria because you're trying to kind of evaluate nuances and human speech and expressions and just even the noises we make. Well, it would be interesting to see what happens for now. It's just a research preview and a kind of private beta for security testing.

So we'll see if this becomes a product, and it'll be interesting to watch. For our third story today, our third and final story, we're going to talk about a pretty unusual benchmark. So we were just talking about benchmarks in the last segment, and just for those who don't know, a benchmark is basically kind of like an SAT test for models, right? So people devise these tests, whether it's a difficult math test or a difficult reasoning test. You test a bunch of models against it, you see how they fare. And this one is about culturally specific riddles. So that actually has two components, and I think it's worth taking them one at a time first. So let's start with the riddles component, which is kind of the more interesting half of it to me, actually. It's not okay, but we definitely know that these large language models that are trained on huge corpuses of mostly English and Western languages, on basically a text of the Internet, which is not representative of all the countries in the world,

we've known that these models might have a tendency towards their training set. But this theme of testing models on riddles, why and how did this emerge? because this actually is a practice that even precedes the launch of ChatGPT. So, Judith, I asked you to read up on this history a little. Could you brief us a little bit about why we're playing like Riddler to Batman with models at all? One of the reasons is riddles are, especially for models, can be incredibly difficult to solve since it's not just about pattern matching. You need creative thinking and wordplay to solve most riddles. And so when we're testing the effectiveness, efficiency of a model, riddles are a really good benchmark for them. Speaking just to the riddles component of it, like if you think of the canonical riddle for me is the one that Oedipus solves from the Sphinx,

what walks on four legs in the morning, two legs at day, and three legs at night. The answer, man or humankind, because, and it requires stretching your brain a little, You're thinking about legs in a different sense. You're thinking about times of day in a different sense. The baby crawls on four legs in the morning at the beginning of life. The adult stands upright. The elderly man or woman walks with a cane, night being the nighttime of life. Right. It's this metaphorical use of language that initially the presumption was that our C-3PO wouldn't understand that, Ash. Right. so Ash when you add in this level of cultural specificity were you able to dig into the paper at all and what do you think happens when you both kind of brain tease a model with respect to this riddle like stretching of language and then also kind of demand of it cultural specificity what do we learn about models here I know you love benchmarks by the way I do indeed so first off you can actually go

to Cohere's cultural riddles paper website and you can do a quiz. So you can kind of do a quiz and test your knowledge against an AI. Thankfully for me, I outbeat the AI. I think I got a score of like 80% or something. It was actually pretty fun. I recommend you can go and like anyone wants to, just go to their website and do it. I hope that they're collecting that as training data. What this exposes is far bigger than a riddle or, you know, like cultural awareness, what this exposes is something that even OpenAI probably put some research out about, I think last year or the year before, is that we just, like, the entire space of benchmarking needs to evolve, right? And let me explain why I say that, right? All large language models, the benchmarks that they get benchmarked against, you if you get the answer right you get a point if you don't get the answer right you don't get a

point right but also if you don't know the answer you don't get a point okay so the models are incentivized to guess right as opposed to because you know they think about it and they go you know if you ask the model what's my date of birth okay mathematically there's like a like there's like a If you just think about, you know, the day of the year, OK, there's like a one in 365 chance that it's going to get it right. If it says I don't know, it's got zero percent chance of like getting a point. Right. And so it's incentivized to like, oh, I'll just make one up because mathematically, I'd rather take that chance and have like, you know, a point zero, zero, zero, whatever percent likelihood of me getting this right versus not getting anything at all. I think that we need to rethink benchmarking in totality in regards to how we evaluate these models. In regards to the actual question that you asked me, I think the great thing about culture and society and so forth is it's like a moving goalpost.

Our cultures evolve, it doesn't stay still. OK, and so it's very, very difficult unless you have super fast training runs and you've got all the right source data to go into training these models for them to be able to respond appropriately. Last question, Jessica, I'll keep it open ended. What interested you most in the cultural riddles study? What interested me most? I guess it was, I was thinking back to a conference that actually ran this week from IBM Tech Exchange. I'm not sure if you caught this. So Tim O'Reilly is one of the speakers and he talked, actually he commented on, he made a parallel between words and essentially the context they hold and magic. And let me explain that. So essentially, a word and your prompt, it holds a representation that then once you type it, it expands into a request, right? And essentially, now we're able to package our perfectly written, you know, requests and share them in the tech world, right?

Share these skills, share these agent workflows. And that's like casting spells in a way. And it really makes you think about the power and the context that every single word has. So, for example, if you say, write me a PRD, that means a product requirements document, that three-letter acronym, it expands into this whole structured text. And from company to company, what a PRD looks like might be different. And I think that very interesting because it almost like what happening here because every word from culture to culture might have different kind of a different sense of interpretation of what that word looks like And then the entire request changes completely So it that takeaway of really understanding the context of the languages and who your audience is, right? So if you say you support people from all around the world or certain locales, actually test with the languages that they'd be using

and how they make sense of the world eventually. I love that you brought in the spells. I think the ultimate benchmark should be expecto patronum. That's what I'm saying. And when a model nails that, we're in trouble. Or we're very safe, maybe both at once. Thank you so much for covering these three stories. We have a special bonus segment that you teed up perfectly. We're going to bring in, I guess, to talk about Tech Exchange in just a moment. Okay, we're joined by Kiyo Gabriel for a special segment. Kiyo is the IBM field CTO for data, and he joins us from Atlanta. On the day we speak, the 29th, this is the last day of IBM Tech Exchange. And Kiyo, you've really sold me on this as being a really exciting conference. You said it's full of boring stuff. But actually what you said is boring. It's like the story is that the boring stuff is becoming really exciting and maturing. I've been going to like all the big conferences for the last three years.

And the hype thing, there's a wave. There's waves. It's like at first it was like vector search and retrieval augmented generation. And that was AI, LLM model training. Then we got into, you know, over the last year and a half, it's all agents. And then, you know, some coding agent stuff. But what was really awesome about Tech Exchange is we're starting to see us getting our hands around things like agent identity. orchestration, you know, process orchestration, monitoring, and all of these things are, they're old, but they're also new. Like the way that we have to do these in an agentic world has changed in many cases. I would go so far as saying fundamentally. And one of the things we introduced was the Streamhouse, which is really just giving a name to a new data architecture that some of the like, the biggest companies in tech have been using for many years. It's just that if you didn't have those like big, big, you know, AI native or tech

native companies, so like, you know, we talk about like Uber and Netflix, if you didn't have their data problems, you didn't need this class of architecture. But now if we look at sort of the way that the volume of agents that there are and the way that the data exhausts the agents create, we're seeing companies that are used to have much lower data needs. Now they're going to need that kind of platform, and that's what we're bringing to market around the Streamhouse, just as one example. So lots of, it's sort of like the old is new, and we're getting into the maturing cycle of AI. You were speaking about how, especially in the opening moments of sort of, you know, displaying some of the things that have, some of these mature technologies, people were getting really excited. Let's start with Streamhouse, since that's kind of your wheelhouse to double up on the house metaphor. What is Streamhouse and what has really matured in the last year that you've been able to present something really new and exciting to IBM clients and other folks? I think that we've seen a lot of companies gravitating to a lake house. We went through Warehouse and then it was Data Lake and then we went to Lakehouse.

These are all different ways of sort of housing data, right? So a data lake is you just kind of dump it all in and worry about it later. A warehouse is something else. Streamhouse kind of brings some ideas together. Is that right? Yeah, I mean, the warehouse is like structured archival. Data Lake is throw everything in the trash bin. And Lakehouse is trying to recognize the strengths of both of those. Like, I want to be able to have structured and structured, and I want to be able to separate compute and storage. Those are sort of the pillar value props of the Lakehouse. The challenge is that now we're starting to see, hey, I like that scenario. The reality of my business is that my data estate is distributed, which you could also refer to as fractured. And so we need a way to pull all that together to bring context to agents so that agents can act on it quickly in an operational context. And so how do you do that with a lake house that was really not designed for an operational context? And what we're showing is that with this sort of shift left technique of saying,

taking a lot of the way that we handle data in a lake house and running it in Flink, So that's a processing framework in the streams, in streaming platforms, that suddenly now we can do a lot of that and deliver the operational needs of agents in real time. Obviously, IBM in the past year or two made this big acquisition of Confluent to enable a faster moving data. So data is where it needs to be for agents and others. So bringing all these long term themes of agents together and it's sort of this kind of like archipelago of ideas, it sounds like it's converging to a really exciting moment. What is one concrete kind of cool application that an IBM client could not do a year ago that now they can do now that Streamhouse is a thing? Yeah, that's a good question. I'm going to twist it just a little bit. I think, and like I did the keynote, I talked about this there. It's that historically streaming architectures have been hard to build. the way that you have to think about stream processing, especially what's called stateful stream processing,

it'll wrap your brain around an axle. And now with AI, it's sort of, it's interesting timing because AI is creating this problem that allows, that requires more companies to use this new architecture, but it is also the solution. I can write stream processing, stateful stream processing jobs using AI effortlessly. I can take jobs that I've defined in the lake house and say, hey, just rewrite this. So the important thing that this unlocks is that most people, because streaming architectures were hard, they would gravitate to what they know, which is a request response style architecture. Most of us, we came up through, we were educated in that. Almost every web app anybody's ever built, most of their mobile apps ever built, they're kind of layered on, hey, I'll check for data every now and then or I'll query for it and I'll update it. It's very request response, right? Streaming architectures are better because it allows me to, with lower resources, be able to make my apps more responsive. They get updates right away.

And the whole thing starts to feel more organic, but it's harder. And that's why fewer people have done it. Well, now those barriers have come down, and we can now, you know, say, the request response architecture, I understood it. That's why I built it. But now I can actually build what I really wanted, which was a better solution for many problems in small and large businesses. We just couldn't build it, but now we can. The advent of agents are pushing more and more businesses into a corner where maybe they weren't, maybe you didn't have a big data problem or a big concurrent user problem previously, and suddenly with agents you do. And now you need to think about, rethink, okay, what is the right solution for having this new data problem that I didn't have before? And the same thing actually starts to spiral with identity. Identity is as old as computers. You know, I sign in. But now what happens when you have these agents that are autonomous, they're not people, and there's, you know, maybe an order of magnitude more of them, and I need to make sure that they only have very short-lived credentials and things like that.

IBM has fantastic solutions around this. But they're different. The problem, it's like it was around before, but it's manifesting anew with agents, and it's causing us to approach the solutions differently. What feedback have you gotten? Have you talked to any clients or friends of IBM who are excited to try this out next year? Yeah, I had a great meeting with a major insurance company, and they're looking at this and they're saying, this is the solution that we've been looking for actually for years. And they're starting to see how it can suit them. I think this is an interesting situation because typically in most enterprises, the operator, the people who are responsible for keeping things running, in many cases, they really don't know the specifics of what's going on in the systems that they run. And that creates this interesting tension and barrier between the people who build the stuff and the people who run the stuff. But now with AI, it's also really empowering both of these groups.

You know, the people who were just operators and they really don't know, now they can say, hey, I can actually be a better service to the people who are building. Now with AI, I can actually understand how my systems are being used. And likewise, the people who are building can say, okay, now I understand how my software leverages the underlying systems and where I may be causing problems for those groups of people. And I think that it's really cool for people to start to see, wow, with AI, it's not just about, oh, I can make cool images and I can write code. transform the way that you actually execute your job and the way you actually work with other people. It's very transformative to relationships and the organization and to me, I think people are starting to see that and that's really exciting. Well, that excitement is coming through. I wish I could have been there in Atlanta with you. Thank you for this support. Enjoy your last day there. Safe flight back. I want to thank both Kiyu and our other panelists Ash and Jessica and Judith for joining for this

week's mixture of experts. Tune in next week. We'll see you then. .