AI開発者とドメイン専門家をつなぐ――小さな特化型エージェントの未来

There's going to be a future where our agents are tuned to perform one small task and perform it really, really, really well. And in that case, you could actually sort of measure the efficacy of how good it can do whatever that thing is. That's literally my buddy Raul just told me like, oh, man, I left my stuff running and I got a 1K bill I was not expecting. Like software factories are hard, man. if you throw something at a big model and it does it like is there really the need to go and then scope down like you're talking about if you've got these thousand agents isn't that so taxing on the human brain to figure out like where's that like issues agent i can barely keep track of my freaking chats that i have right now and i don't have these thousand agents it's a good question because of two things. One. All right, Justin, I'm excited for your talk at AgentCon and

MCPCon in October in San Jose. This is the North American talk. What are you going to be talking about? So the title of the talk is The Future is Domain-Specific Agents. And it's basically that. There's this sort of rustling in the trees right now about this thing called domain-specific agents, although everybody has a different term for it. So Jensen calls it super agents. Jensen calls it super agents uh elon has called it like specialist agents um but the idea is that there's a huge unlock where you have small agents that are around a very narrow set of tasks so where our models right now are being trained to be sort of god models that have you know they know how to code but they also know how to like create a water filter to survive an atomic apocalypse for some reason like they can do both of these things there's going to be a future where our agents

are tuned to perform one small task and perform it really really really well um and and in that case you could actually sort of measure uh the efficacy of how good it can do whatever that thing is however small it is like maybe the entire agent's job is to like check gmail and that's all it can do so it's even more fine-tuned than what you'd think of like a rock bot or something like that um and if you could do that if you could have a little agent that was just like it was perfect at checking gmail like that's all it has to do how hard can that be if you could do that then maybe you would want to share that agent with other people right and you could post it online in you could have it on a thumb drive you could have it however you want and then that agent could work with another agent and these things could compose a little bit like we do with skills right now, but with agents instead. And the only reason that this doesn't exist is, well, first of all, it's sort of a higher, it's sort of a higher order thought process

that we haven't quite, you know, like we're unpacking this as we're all learning together. But the only reason it really doesn't exist is that there's no standards around it. So if I create that incredible Gmail agent and I give it to you, you can't use it. So we have to solve the standards problem. So my talk is going to be a lot about that. Oof. Oof. All right. So thousands of agents that are hyper scoped in a way. Yes. Or thousands of agents that have very, very tightly scoped tasks that they are trying to do. It feels like we are going a little bit back in time and just like machine learning models used to do that, right? Yes, machine learning models did used to do that. The cool thing now is that you don't have to train the model. So that the GEPA paper or JEPA, however you choose to pronounce this word, was really instrumental in setting the stage for some people. But it basically says that it is more effective to train your prompt than it is to train your model.

and there's a really weird relationship between a model and a prompt and the tools that are given to it that is not very well explored right now. And you've seen the labs hint at this from time to time. When, like, for example, OpenAI recently said, hey, actually, it turns out that our model performs way better on this test if it's not using this old broken down harness. If it's using this harness, then it's way better. Well, that's actually true for literally everything. Everything is like that. You can have an unbelievable output from one harness and a terrible output on the same prompt with the exact same model, and the only difference is the harness. Now, I've got tons of examples of this. So what you're searching for is to find effectiveness. You want your agent to be very effective.

And once you've locked in that effectiveness, it almost doesn't matter what model it's using or how old it is or something like that. So our coding agents right now, everybody is being trained that we just like, oh, we see a new model release and we just switch our coding agent to the latest model and boom, we're good. That actually fine at these like really large scale models It like kind of can figure it out although you are losing something in there But with small models this is not possible So we really need a type of specification that says this exact prompt with this exact model hosted on this exact provider, this entire chain of things, has provable efficacy for doing the task that it's made to do. And then when I share that with you, you're getting the exact same efficacy. Ooh. And so you see standards as almost being like metadata that you can pin to an agent so that it knows everything that it needs in its harness to be effective?

Yeah, not only that, I think that there really should be a runtime for agents. So imagine in the same way that we have a runtime for Node, and Node can run in Bun, or it can run in Deno, or it can run in Node QuickJS. There's lots of different runtimes that you can run JavaScript in or the browser. We need runtimes for agents. That's very similar. But all of those languages or all of those runtimes are complying with the same standard specification. So there's going to be variations in them. The V8 runtime is just really fast, you know, and it's faster than other ones. So you might want to use that runtime to run your, you know, mission-critical application. But that's not the only thing that matters. And so I could see an ecosystem of runtimes that exist that are all able to run your agents in a guaranteed way that have different pros and cons.

But that is the world that I am 100% convinced is going to emerge. People don't believe this right now, but I'm 100% convinced that eight months from now, this will be a no-brainer for everyone. Dude, I'm so glad you just said that because I already see the clip forming in my mind. as a lift of just like we start with that and then we go like elon calls it this and jensen calls it that and then boom like i i have the voice overlay of justin is bringing blah blah and then i talk about what you're doing and then we rock and roll with a few more clips of like but we need standards and that's the key here and then there's your little trailer that we've got so note to editor That's how I'm seeing it in my head. And all right. But there is there's a few cool things that I just want to like satisfy my own itch for when I when we talk about this. I've found that you get to a certain place and you need this out of, or you take this step of going very, very small scope with small models or smaller, like, scoped models.

It doesn't necessarily have to be a small model. when you have like a forcing function right it's not necessarily you don't necessarily think like when you wake up in the morning like wow cool let me go and make a thousand super small agents that just do one thing really really well yeah and you take like a microservices paradigm to your agent building because if you throw something at a big model and it does it like is there really the need to go and then scope down like you're talking about this is a great question too there must be because on multiple vectors on one there's the efficacy itself and you're right that a big model is generally going to have better efficacy than a small model on any given domain. But that's not true on any given domain if you've had like a really reinforced prompt

and model. Like, for example, my coding agents have like, I don't know, half a dozen to a dozen skills on a few MCP servers. And so every time it needs to do work, it's sort of wading through this ocean of stuff. And it has tons and tons of different tools. But if I were to create that Gmail agent, it would have Gmail related tools. Like it would have a tool to be like check latest messages. Right. And like that's all it can do. And so you actually can squeeze out way more value from a smaller model very quickly. And this is actually why I don't think routing is going to end up being the answer. I don't think model routing will end up being the answer because even the big models don't really know how hard a problem is on the front side. You know how hard a problem is on the front side because you are faced with it just like you said. At some moment, I, like, have a need. I need this thing to, like, do a background job. I just, like, I just needed to go check issues on a five-minute loop and then notify me or queue up something else.

That's all I need. and it turns out i just burned 60 of my fable weekly allotment on that exact problem this week dude that's literally my buddy ro just told me like oh man i left my stuff running and i got a 1k bill i was not expecting like software factories are hard man dude they are especially you it can get away from you quickly so imagine if like when i hit that exact problem Then I tackled it I was like oh my gosh this is ridiculous There so many advantages if there standards First of all when I say okay I need to build that agent my agent knows exactly how to build it I mean, what I'm not saying here is like I have the spec and I have the runtime. So that exists, which is why I get to play with these things all the time and I can see that they're very effective. But I can just spin up any of my coding agents, Fable or Codex or whatever. And I just say, like, hey, I need you to build an agent that does, like, this discrete task.

And then it just goes and does the discrete task. It creates the agent with the precise tools and everything. And now I can deploy it on the cloud. I can give it to my coworkers. It's, like, a very distilled thing. I'm specifying absolutely everything down to the model. There's all kinds of problems that come with this that you have to solve, which is why it's sort of an unsolved problem at the moment. But it will get solved. I think I've heard this called agents as tools. Would that be the same thing? Yes, it could. It could be that. In fact, that's like a way to solve the problem. Yeah, because what I'm understanding from you is more like, OK, there's the and you're saying specifically maybe it's even not agents as tools. because the model router of the like big brain that goes and chooses and kicks off tasks to sub agents isn't necessarily the best way of going about this especially if you already know hey i just want this thing to run uh every five minutes and it's not like it's this very complex

thing and i can hyper optimize this thing that just needs to happen it grabs issues or it tells me what my latest unread messages are you know like those things can happen on a very frequent cadence and you don't need a big model for them and you know that it needs to happen so that's right i just set up you set up the specific agent to do that but where i am kind of still questioning is how this looks in practice if you've got these thousand agents isn't that so taxing on the human brain to figure out like, where's that like issues agent? I can barely keep track of my freaking chats that I have right now. And I don't have these thousand agents. Such a good question. It's a good question because of two things. One is like this distinction between sort of a coordinated with sub-agent structure, like a tree structure of agents or graph engineering is kind of what people are calling it now. Right. But but basically a tree

of agents where you have some sort of a coordinator sitting above a bunch of other ones. That's a great architecture for all kinds of things. So if you were to take this same idea and try to build like a Hermes open claw style personal assistant, that's what it would look like. You would have like one that you're speaking to. It has a lot of sub agents. They're all going and doing their different tasks. It would look like that. But that is by no means the only mechanism by which agents are going to find penetration. So, yes, it can do that. But also it can just be this like tiny little back of house thing that's running for a butcher shop. And it's like figuring out how many pounds of meat to order for the next week. That is like super realistic, like a very realistic tiny little agent that would cost them absolutely nothing to run every week. and this is like an advantage that i actually have not living in the valley right now is you know everything when i go to the valley pretty frequently and when i'm there it's like

oh my gosh this is amazing there's so much hype and excitement and then i like come out out to i'm in virginia i come out to virginia and it's like nothing nobody zero penetration crickets yeah absolute crickets i could walk around and be like what are you guys doing with it with agents these days nothing no yeah what are you talking about chad gbt yeah yeah exactly But like this is this is sort of, in my opinion, the missing link. Like it is the thing that will that will turn this into a main street tool is being able to say, like, actually, we've got this this very narrow implementation that meets your needs precisely rather than a two hundred dollar cloud code sub or something. you know where i have heard and i actually was just thinking about how strong of an opinion i have on this that it won't happen so potentially we can talk about why it will happen right now is since 2022 or whenever agents started getting popular the first thing that a lot of people went

to try and build was an agent marketplace and I am convinced that is never going to happen it's like mean girls like stop trying to make fetch a thing because it is not going to be a thing in that again like me as a person I'm not going to go sift through a bunch of agents that random people built on the internet and see if it might potentially work for my use case but then I got to tweak a few things or I got to tell Claude to tweak a few things instead of just why don't I just tell Claude to start it from scratch and exactly what I need and I create that agent and then if I can distill it down and make it a smaller agent like you're talking about I see the value in that but an agent marketplace I 99 sure that never going to be a thing I think it already a thing all right so good you you got the counter I think it already a thing everyone just has their own marketplace And the only place that we shared any sort of agreement on this right now is MCP and skills

But MCP and skills are effectively just like really bad agents because they're just trying to convert the agent that you're using into one that can do these other things, which has this like incredible upside, which I'm not going to try to take away from it, which is that now I get to still use the same user interface or something like that. Yeah. The downside is that it's all context injection. It's like a giant rag pipeline just getting shoved into the brain of this agent. And, like, not only do the agents get worse when you do this, they get dumber on, like, completely related tasks. So, like, if you were to take that same model and the same agent, like Sol, and give it the job of a skill exclusively, it will do it way better than if you have that skill get injected on top of its pre-existing everything. Right? So, the argument here is not that, like, people are going to need to go out and, like, assemble these things carefully themselves.

they're going to use them very much in the same way that people use skills right now. Which you probably have already done this. Somebody somewhere in your life has been like, dude, you've got to check out a skill. I know, skill, skill. Superpowers. Shout out to superpowers. Exactly. It's always superpowers. Or the grill me. Or last week it was show me. It's true. I have downloaded a whole lot of skills. You've already done the thing you said you're never going to do. I did. That is a marketplace. You're right. It is a marketplace. But imagine if what you were installing was, like, actually effective agents. So there's no reason why the interface for domain-specific agents couldn't be your existing agent. There's no reason why that's not the way it's exposed. It could be exposed that way. But it just needs its own guaranteed way to execute and run and so on and so forth. So, like, that could be one output. But I'm not even really interested in, like, what that final interface is. I'm interested in trying to say we need a way to share and distribute these things.

And, like, that might be the winning outcome. Maybe the winning outcome is every human on Earth has their own website with its own chatbot on it. That sounds very unlikely, but I don't know. Or, you know, maybe GrokBot ends up becoming, like, the thing, and everyone has a GrokBot. Yeah. Because they're doing it too, right? Like, they have their own marketplace. If you use GrokBot, they have their own marketplace. You log in there, you find their plugins. Like you want to use Google, it's got to be one of their plugins. It's like all, I mean, most of them are MCP related, but like it's still this like pre-approved marketplace of things that plug into Grotbot. Actually, as soon as I created my Hermes agent, the first thing I did was like go out and scour the internet for what are the best Hermes use cases and skills that I should have with my agent. yeah and so if you look at it like that oh man it's true the use case i know like already over time but the use case thing is actually one of the most interesting problems too is like when

you go and you talk to people about like hey what would you use a personal agent for yeah if somebody's not using it already or even if they are using it they're like calendar maybe like uh yeah maybe my email uh i had to do some digging it was exactly that this is why domain specific agents are the win because people in different domains know how they would use it in their domain right like when you're at work and if you're not working on agents which is sort of this vicious, you know, what is it? Something eating its own tail. Self-licking ice cream cone. If you're not doing that and you're like a nurse or you're, you know, some sort of doctor, like you immediately are like, oh, my gosh, I have so many use cases where I could have it, you know, be doing the transcription or the notes or it could be doing like there's so many different use cases that we in the tech world are just like sort of immune from.

but the domain experts need to make the domain specific agents so they need a way to just do that yeah they need a way to do that and then i think where i've seen it fall down in the past is that they don't realize what what the possible is and so like the transcription for example it's like oh it can do that oh well yeah like i would love to get transcribed all of these different things that happen right and then you start you start to unlock these doors and you realize wow that's that's uh that's possible okay well yeah and then i want that agent too and so then there's this merging of the two worlds of the tech where we know everything it can do but we don't necessarily have the use cases and the folks that have the use cases who want to know like what can it do yeah this this we need to figure out a way to empower that well i'm so excited for your talk you've got me convinced i'm gonna be there enjoying it and hearing about this whole world of domain specific agents and i'll see you october 22nd and 23rd in san jose

for agent con mcp con looking forward to you and everyone else