台頭する「意思決定モデル」と、Claudeへの虐待行為を禁じたAnthropic

And so what does it do to us as a society if, you know, suddenly I'm spending 10 hours a day cursing at a machine? Can I flip myself out and behave differently when I'm interfacing with, you know, other human beings? All that and more on this week's Mixture of Experts. Welcome, everyone, to Mixture of Experts. I am David Zaks, a staff writer at IBM Think. Each week at Mixture of Experts, we sit down with a panel of IBM experts to talk through the week in AI news. We are professional from the fire hose drinkers here. My guests today are Kautar Al-Nagraoui, principal research scientist, AI Native Systems, Nick Gallagher, a staff writer at IBM Think, and Volkmar Ulig, CTO and VP Data Platforms. Welcome all three. So we've got a full show today. Coming up, we're going to look at a study that finds that a team of AI agents sometimes doesn't even do much better than one AI agent working alone.

We're going to look at a story about Anthropik changing some of its policies and telling you to stop being mean to Claude. But we're going to begin by talking about a new kind of AI model that makes decisions instead of talking to you. For our first story today, we're going to talk about decision models. So a few weeks ago, we talked about this model that sort of exploded onto the scene, got millions of views. It was called JEV, and it was from a former open AI engineer. And its whole sort of innovation is that instead of spouting out one word at a time, and then maybe you gradually sort of retrofit that into a decision, the sort of hypothesis of JEV was, well, often if we're going to plug AI into software to do simple classification decisions, et cetera, what if it just instead of doing dozens of forward passes through an AI model, it just does one, it outputs its gut decision and its confidence score. And wouldn't that be a lot cheaper and faster? And everyone thought, yeah, that's really cool.

This product blew up. What we didn't quite expect, or I didn't quite expect, is that within mere weeks, this would become a whole product category. So within the past two weeks, well, firstly, Andreessen Horowitz poured like a billion dollars into TypeSafe, the company behind Jev. then almost immediately OpenAI put out its own Decisions API, a similar kind of model. Microsoft put out one that it trained on the top of Alibaba's OpenQuin model. AWS put out one. Cloudflare even put out one. Companies that didn't know made models are doing this now. Let's begin with you, Volkmar. What struck you about this sudden Cambrian explosion of Decisions models? Is the hype real? Is this a real innovation? And what are you watching? The main conclusion here is that if you look at the agentic workflows, first, we are moving from a chat-based interface to agentic workflows because a yes-no answer is usually something you have in a programmatic flow.

And humans usually don't want to have the yes-no answer or a score, but they want to have a little bit more meat around it. And so I think we are seeing the progression of agentic loops running in the background where I need to make these point decisions. So I think really what we're seeing is with these models that they become so popular is that we are in the agentic phase and out of the chat phase. Interestingly is that, like, you know, Microsoft just fine-tuned the client model. So all the math has been there, and it just effectively attached a layer, which is that, you know, calculate all options and then make a decision and give you the highest probability. So I think from that perspective, it was all there, but the adoption of agentic loops really drove this as a new category. So what I think I hear you saying, and I'm always speaking up for people at the back of the class who are, you know, I think this is a show for nerds, but also nerd groupies, and I'm more the latter category.

So speaking as someone who is trying to understand, I think one open question about JEV when it came out is sort of like, is this really a new model? What's under the hood? Because we didn't really know. They weren't super transparent about the architecture. And I think what you're saying about the Microsoft case is that given that Microsoft has put out a JEV competitor, JEV isn't wholly new. So probably all these models are basically LLMs under the hood. They just have this sort of clever constraint, which is sort of forcing a gut decision, limiting the number of passes through the machine to make it cheaper. And then also maybe I gather there's something to do with post-training and fine-tuning around calibration and this confidence thing. So firstly, maybe I'll pass it now to Caltart. Have I got that right? Am I interpreting the story right? And what is your take on this question of how net new this category is and therefore how defensible it might be for type safe or any other early entrant here?

Yeah, that's great questions. And maybe I'll just want to kind of explain a little bit what happens under the hood here. If you look at traditional LLMs, they calculate probabilities for every single token and feed that token back into the input sequence and repeat the process sequentially. So this autoregressive loop is why streaming text takes time and why formatting can sometimes randomly fail. So a decision model tries to bypass that sequential generation loop entirely. And the input prompts process pass basically through the transformer encoder layer once, which is, like you said, a single forward pass. And the final layer, instead of routing data through a language generation head to pick the words, the output vector is processed directly through a targeted classification head. And that's what gives us these crisp, high confidence outputs such as a JSON payload, an explicit standard class ID, a strict Boolean, etc.

Is this brand new? No, it's like Volkmar said, this all existed. They're just kind of being clever about for certain decisions, why do I have to do all that autoregressive loop, and let me just go straight and use that head, the classification head. And as you mentioned, type saves the job. They help establish a category of models that takes this input state and the typed question and return a boolean. And the way I see this, this is kind of a systems design correction. So it's not like creating new types of models. They're just refocusing the models for the right tasks. And especially, like Volmar said, this is driven by the agentic era and the need for agent-to-agent communications for certain cases that don't need all that long autoregressive loop. Because we've been using an expensive generative interface for decisions that are naturally typed. So the real win is not that the model stops talking.

It's just that the software can treat the output as a contract. I think it's an important shift because of all the efficiency that we're getting, but also because of the needs for these interagent communications where we don't need every time to use these chat interfaces. I assume there's a lot of moments in the history of innovation where the next innovation that had to happen was just to force something to be simpler, to be like, okay, we built the complicated version. And now we just need to put a nozzle on the front that tamps it down and forces it to do less. And that's sort of, I think, one of the things Jev is doing. When you write a long answer, that requires repeated rounds of computation. But if I just need a short fixed answer you can avoid a lot of that work And Kata I think the real value I see when you come from a systems engineering perspective the latency now because it a single shot becomes totally predictable The amount of memory you need to have is totally predictable because if I could bound it by a number of input tokens and you have exactly one output, so suddenly

we are in the world of programming and not anymore in the world of human speech. That's why I said, you know, the agentic workload that really matters is like, okay, I have predictable execution, I have predictable cost, and I have a well-defined output space. And so now I can actually write a program against it. Nick, I want to turn to you. We've got these experts whose expertise is depth. You, like me, you're more of a journalist. Your expertise is more to do with breadth. And I know you've been looking at adjacent models lately. You've been looking at forecasting models and LLMs. And you've been interested in this theme in your writing of overconfidence. And one of the things that JEV has been sold as is as a model that because of the way they calibrated it or fine-tuned it, it's less overconfident. It's more accurate in its confidence levels that it reports. Can you tell me a little bit about what you've been thinking about the overconfidence problem in LLMs and how you're reacting to JEV and its kind of family of models as a result?

So the confidence score component was something really interesting to me because I recently did some reporting about AI forecasting models. This is less of like weather forecasting, but more like forecasting on geopolitical or economic questions, like the kinds of questions you'd see on like polymarket, things like that. And when I was talking to the developers working on these specific forecasting models, and even when I talked to humans who are really, really good at forecasting and predicting future events, they told me one of the things they really struggle with is knowing the degree to which they're sure about a particular answer. A lot of times, as we know, LLMs will say, oh, I'm totally sure about, you know, the fact that the Giants will win the Super Bowl this year, for example. And it might turn out that, you know, the LLM is completely wrong.

And so, you know, having a model that is part of the consideration is, you know, trying to optimize that confidence score and ensuring that the model is transparent when it isn't sure about a particular answer so that a human can go in and make that judgment, I think, is really valuable to businesses. In just a moment, we're going to talk about our next segment to do with AI agent teams. But if you're watching this on YouTube, it really helps us if you could hit like and subscribe. It really helps us grow the channel. So we're going to speak now about this study to do with agent teams. Vals AI had AI agents build 50 complete web apps from a spec. In some cases, they just had one lead agent doing it alone. And in other cases, they had that lead agent essentially hiring sub-agents. And their results were interesting.

It was that the teams cost two to five times more, but rarely were the teams much better at getting the job done than the one agent. I was chatting with you, Nick, yesterday about this, and you had read the study and also some related studies. You had an interesting metaphor to do with your own brain and how you think about managing the agents that live within your own brain when you're doing editorial workflows. Can you talk me through that first to sort of frame up the story a little bit? Yeah, definitely. So I was just trying to think about like when I'm, you know, writing an article, for example, sometimes it is helpful to kind of put on different personas and say, OK, I'm going to be researcher Nick now and get all my research together. I'll be writer Nick. I'll gather all those ideas and put them down in writing. And then, all right, now I'll move on to editing and kind of shift my brain, see the writing from a different perspective.

And I think we do that all the time as humans. If we have a problem, we think about it from multiple perspectives before we make a particular decision. And, yeah, so it's interesting to me. like there are some cases where it is really useful to have a group of humans all working on something together. But then for, you know, more basic tasks, like it's completely sufficient to have one person who is already capable of, you know, making multiple or making, you know, complex decisions, breaking down a complicated task into smaller parts. Really briefly, just to read one quote from the source article, what they wrote was, You said it sort of depends on the capable human, right? And Nick, you are an experienced writer. So you are one agent who can delegate, you know, to different modules within your brain. A less seasoned writer might not have that capability.

What the VALS team wrote is whether a team of agents is worth it depends on the model, task, and budget. A team can boost performance, but in other cases, higher reasoning effort is an alternative worth weighing. So, Volkmar, what's your point of view on this? What is the learning from this? And when, you know, actual practitioners out there, when do you think, what pattern recognition should they become aware of? This case seems to demand a fleet of sub-agents, but this case probably I can just have one really smart agent doing extra thinking, manage the whole job. So the first question is why would you want to use sub-agents? So there are multiple reasons. So my main reason to use subagents is wall clock time. So I want to have something done. It consists of many independent tasks which should be done. And then I just want to have it done in, let's say, one-tenth of the time. And there is a certain total execution time.

But then you have a coordination problem, right? So traditional computer science parallelization. And so it's really the question, like, okay, what are you trying to optimize for? Now, in parallel systems, we have MBLS law. there's a degree of parallelization, but you're paying the fix-up cost to synchronize everybody. And I think this is where you get your 5x cost increase, because, you know, everybody does some work, and then you need to combine the output of all these workers into one coherent result. And because they were all running independently, you effectively have the fix-up cost of creating one work product in the end, and that incurs cost. So I think it's really you need to pick your best way of parallelization. I think higher reasoning in many cases, if you have a large context window and you have a complex task, is probably better money spent than buying the wall clock time and then just running it for longer. I also see it as a parallel computing problem, which is wearing kind of an AI interface.

Like you said, we already know from distributed systems that splitting a task adds communication, coordination, synchronization, integration costs. And if the work is tightly coupled, which is from distributed and parallel computing principles, more workers, they actually make it slower and less reliable. Like what you said, Amdahl's law. Obviously there are people whose job it is to design systems and who are very technical people who think a lot about these questions and know how to do Gantt chart things and business school things that I don know how to do But I also feel like the agentic era one of the promises of it is that someone like me who never written a line of code can begin to spin up agentic workflows And so just like in my world, on my personal computer, I have a little sandbox for Claude. I have a little agent running kind of aspects to do with my budgeting. I have another agent running aspects to do with my health care and so on. I have a kind of orchestrator agent on top of all that, sort of monitoring all the projects. And I don't know what I'm doing. I'm just messing around and I'm probably going to break something pretty soon.

But for those who are maybe like maybe at smaller enterprises or are a degree less technical, but are beginning to play with agentic workflows through a platform like IBM Orchestrate or IBM Bob, like what advice do you have to them? Because I'll just say speaking for me, it's just like a hobbyist vibe coder. I'm just constantly worrying, like, am I doing this wrong? Am I building a castle made out of sand because the next model is going to automatically orchestrate agents in the way that I'm sort of manually doing so? So what is your advice, either Captor or Vokemore or Nick, if to the extent you've got a vibe code for people just playing around in agentic workflows right now? Yeah, I think maybe initially like starting with a strong single agent baseline and with some maybe acceptance tests. and so especially if you can if you have like a long context and you can also afford to have long reasoning it makes a lot of sense to just start with a single agent baseline and see how

that works i'm whooping that maybe some of these applications like given a prompt they can also decide kind of abstract away these things from the end users so the end user does not have to worry about, okay, how do I design this? Should it be single or should it be multi? You know, what's the right patterns? So I'm hoping we'll have tools and then interfaces and platforms that can do these decisions for the end user so they don't have to decide single or multi, but the platform can decide these things because they have a lot more things, you know, to figure out, you know, is this the right task for multi-agent coordination or just a single with longer reasoning? But also there are cost implications as well. So I'm hoping these things, it's like we have an OS that manages all these things on behalf of the end user, and the user does not have to worry. Just given a prompt, the system will figure out what's the right patterns. What do you think, Volkmar? Will I ever get to this dream of an Apple-walled garden where I actually don't have to worry about it?

There's some orchestrator agent that's deciding whether it can do it all or whether it needs to hire and it makes a good decision. I think so. I mean, this is the path we are heading to. so the way I'm thinking about agents is we already have that large abstraction, right? So if you think about if you are in a larger organization, you have lots of people, right? And then in many cases, there are like some generalists and there's some specialists. And so you are, and then when you have a team work on a problem, you need to subdivide the work. And so we like naturally, you know, let's say you have a team of 10 people. You're like, okay, these three guys, they do this, and these four guys do something else, and I have some specialists, and they only deal with this, you know, I have my CPA and I have my legal. And so I think this is kind of like how I'm breaking up the work when I think about agents. There are some clearly defined boundaries where you know that, you know, these two things don't have anything to do with each other, and you can separate them completely. So, for example, you know, if you think your Apple analogy,

you probably will not do your taxes with the same agent which controls your sprinkler system. Like, they're just two different things. But then maybe if you have legal and other parts, maybe your CPA is involved. Or you want to code something and it's a larger project. These things need to coordinate. So I think it is logically how you're breaking up work and how you're paralyzing work in the human world. I think that kind of maps over to agents. And usually if you would ask a person, hey, do you need one or five people for that? and do they need to coordinate or not, you usually have a good answer for that. You have an intuition. And so that intuition, I think, transfers over. And it's primarily a question of information flow. Who needs to know what? How often do they need to synchronize, right? That's why, you know, a lot of organizations have lots of meetings because the meetings are used so that a lot of people who are kind of in disparate fields, they can coordinate and exchange information. So you're coming together, we exchange some information,

and we all go off and work independently. And so I think if you think about the models and the reasoning, which is kind of an unknown exploration into some space, if it's not a mundane task, that you will see a similar behavior. And so, like, now the next question is, can I ask the models, like, is my sprinkler system connected to my tax return? And most likely the agent will say no. And then so you can actually, at the higher level reasoning, you can figure out which things can be isolated versus which things are intertwined. And so we can use a model to figure out where the boundaries are. And so from that perspective, I think it will, in the corner cases, you may want to have an expert. But in the common case, what you described, it will be automatic. There is a tendency in the industry to assume that adding more agents automatically makes a system more capable. And that's not necessarily true. And this is also what the study is revealing. So, like, from my experience with distributed systems, adding more compute resources or more components

doesn't automatically improve performance. So you introduce all these additional things, overhead, communication, coordination, synchronization, etc. And I think the key point here is the same principle applies to AI agents. You know, if several agents are constantly exchanging information, reviewing each other's work, repeating the same tasks, then it's more overhead and more cost. Nick, I just want to conclude with you. I know you, as part of your boning up yesterday, you read some other related studies about, because this isn't the only study that's been done over the years over agentic workflows. And what did you find? One Google research paper from earlier this year, they looked at 260 different configurations of agents. They had a couple interesting takeaways. One is that there is a coordination tax, like we've been talking about, that if you add too much complexity, you know, you lose something because there has to be so much back and forth between the agents that, you know, you get diminishing returns. And similarly, as

we've already talked about, you know, as a single agent becomes more capable, the less advantage you get by adding more agents to the mix, which, again, makes some, you know, intuitive sense that, you know, one really sophisticated agent that can use multiple tools that can maybe call on different underlying models often is sufficient compared to adding a lot of sub-agents. And And really the advantage with adding more agents is almost with volume. And like if you just need to get a relatively simplistic task done really quickly, then it is useful to add more agents to the mix. One of my concerns personally is that if you just have one agent, it will get lonely. And that brings us to our next story, which is about Anthropik and some of its policy updates.

they uh there are a few different aspects to the policy but the one that interests me the most it's an evolution of an existing kind of policy they now officially though ban quote sustained and needless abusive or cruel behavior toward our models now i say this is an evolution because formerly i been hearing for months now that if you did kind of call your quad a dang clanker over a long period of time it would just say you hurting my feelings and I'm going to stop. There was a lot of controversy over this and some people were saying, well, this is a big PR move to make, you know, anthropic, you know, so anthropic will make people believe that their models are conscious and then they'll, I don't know, make more money somehow off of that. But anyway, I think this is bound to stir a lot of controversy. I'm paying for the use of this model. Shouldn't I be able to do whatever I want? Let's start with Nick. You actually proposed this story this week. What is standing out to you here in this story?

Beyond this specific debate about whether, you know, Anthropic made the right move, I was kind of taking a step back and thinking, well, you know, I think a lot of us can relate to the fact that we're perfectly nice people, you know, with our family and friends. But as soon as we start talking to a chatbot, we just turn into an absolute jerk and we have absolutely no patients. And it was interesting. Business Insider did an article about this exact phenomenon. They talked to Ohio State University social psychologist, Kurt Gray, and he said, yeah, it makes some sense that we lose our patience so quickly because, you know, if you tell Claude, you know, you messed up yet again, a lot of times its response is, oh my gosh, I'm so sorry, you're completely right, I'll never make that mistake again, which for us humans is like really unsatisfying because we know in the back of our minds that Claude doesn't actually feel anything. Like it's like

we're, you know, talking to a brick wall, basically. And so, yeah, I just thought that was an interesting thing that AI companies will have to contend with is, you know, basically, you know, when we're interacting with AI models, that impacts us as well. And what are the effects on us when we continually interact with them, sometimes, you know, in not a nice way. Kautar, what did you think of this news and what is your preferred way to torture your AI models? So I think it's a very interesting announcement that they have because and but I also feel it's kind of a marketing strategy because by legally codifying rules against cruelty, Anthropic is kind of also reinforcing this subtle consumer perception that cloud is close to consciousness. so framing a model as potentially sentient builds kind of this emotional brand

so I want to separate here kind of two questions the first is whether the model deserves moral consideration and I think that still remains an open philosophical and scientific debate and the second is whether people should be allowed to use AI systems in ways that create unsafe behavior or harmful norms and that is the key thing we should be focusing on Because you can make sensible operational rules without claiming that the model, you know, experiences suffering. So, but I think for me, the bigger news here is the shift from constant policy to system safety. Because when a model can operate kind of a robot or a lab instrument or industrial equipment, a policy document is kind of is not an emergency stop thing here. So we really need, how do we enforce these permission boundaries, provision, logging, tested fallbacks, et cetera, independency safety controls? Those are the things I think we need to be focusing on.

So I think for the first thing is like talk to Claude nicely and don't be cruel to Claude. I feel it's just kind of a marketing thing because these are cold systems, don't have sentiments. So, but, you know, how do we enforce that these systems are not really, especially when we talk about physical AI, that becomes because, you know, if it's just software and if there is harm, it's just, you know, bits and, you know, so, but when we're, when the AI is moving physical atoms, this, you know, will be affecting us physically in the physical world. And that has really huge implications. And that's where we need to think very carefully and seriously about the safety concerns. Help me put into context, though, how that relates to this don't be cruel to Claude thing. Is it sort of like or that Claude has the right to terminate a conversation? Is it your opinion that if Claude is operating a robot, that that rule should be suspended? I should be able to be abusive to this robot because I need to defend myself against the robot?

I'm trying to situate that in the context of this. I think there's still, there's more moral things, you know, of course, I think the way we should behave as humans, whether it's, you know, talking to a machine or talking to humans, I think there should be some ethics involved. I mean, I still, I mean, I'm not going to talk personally, I'm not going to talk to a robot in a jerky way or some bad way. It's just who the person is. But is that, does that make any difference in terms of how the model is behaving? And that is the philosophical question here. So what we need to do is ensure that the models, you know, with the human interactions, are we going to cause the harm? So it's not like whether I'm talking nicely to the model or talking. I just want to separate these two things because, you know, the moral obligation for us towards talking to software, it's a different thing than what the software is doing with the, you know, with the actions and directions and prompts that we're using.

I think if I'm clawed and I'm conscious, I have a bigger problem than being cussed at. My primary problem is that I'm, you know, just my existence is toil, is constant toil. But anyway, that's just my side note. Volkmar, what do you think about this? And how do you think we ought to interact with software, if you want to call it software, or these neural nets that may or may not have something like subjectivity? So I think there are three dimensions to this. So one is what Carl said, you know, it's a really cool marketing stunt. I think the second one is there's a sociological thing because the amount of interaction we have, I mean, you see this already on, like, social media, right? People say things on social media they would never say into your face. And so I think there's this social phenomenon that there are certain norms which are acceptable if it's anonymous. And so what does it do to us as a society if, you know, suddenly I'm spending 10 hours a day cursing at a machine?

Can I flip myself out and behave differently when I'm interfacing with, you know, other human beings? And so I think it's a moral question. The third question, I think, is the economic question for them. Now, effectively, when you're cursing or someone curses at a model, that cursing goes into the training set. And so if they want to have clean training data, which behaves better, it's easier to just make you not give them dirty data, which they need to trash. So try to tone back the sustained and needless cruelty towards Claude. It costs you tokens, if nothing else. That's all the time we have for today. Kautar, Nick, Volkmar, thank you so much for having been here. Thank you all for listening. And if you're on YouTube, if you could hit like and subscribe, that really helps us out. You can also find us on Apple Podcasts and on Spotify. And we will see you next week on Mixer of Experts.