Grok vs. Dot vs. Muse vs. Claude Duet：パーソナルエージェント対決で最強はどれか

I told you all I have personal agent fatigue, but I am here for you. So today I am going into the personal agent minds and I am running the very first How I AI personal agent benchmark, which goes across four agents and nine tasks to tell you what personal agent is actually the best. I am going to try to run this in the most unbiased, equal way. I will tell you how I set up the benchmark, what is in the benchmark, how I ran the benchmark, and we're going to look at the results and at the very end get a summary of what agent I think does best on which task. This was a really fun and also hard one to run. I spent definitely several hours today figuring out how to run this and getting it done. I hope that you all find this helpful and I am going to be along the ride with you because I really haven't looked at many of these results and so we're going to look at them live. We're going to see how it goes. And I'm going to tell you which personal agent is the

best for you. This episode is brought to you by WorkOS. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch, it's a massive lift. That's where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features, so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using WorkOS to move faster and meet enterprise demands. Join them

and hundreds of other industry leaders at WorkOS.com. Start building today. Okay, before I go into the results of the benchmark, I just want to tell you how I came up with this bench. So the first thing I did is I went into all my agents. I went into Dot, GropBot, News, and Duet, Claude's Duet. And I said, I'm going to run a benchmark. Give me like the top tasks that you do for me. And so each of them gave me their top 10 tasks. And then I put them all in a Claude and I said, okay, come up with a way to run a personal benchmark against this, picking the tasks that were most common across the agents. Because to be honest, I use different agents for different things. And so I wanted to find sort of the most cross-cutting use cases as well as use cases that would test different components. And then the way I was going to run this is I was going to have one operator agent, which was going to use computer use to run all the other agents and then keep the benchmark going and so on and so forth.

The one problem with my operator agent plan was I also needed to run DOT and Codex is my favorite computer use agent. So just to be very clear, I had Claude run dot and then I had Codex run Claude, Grokbot and Muse. So there's like a little bit of diff in the operator agent, but I thought this was pretty clever. The other thing that I wanted to make sure was that all of my agents had access to the same things. And this is a really good tip for you all that are suffering like me, where you have all these agents and they all have access to like a couple different email addresses, a couple of different connectors, like one's connected here, one isn't connected there. So one tip I have for you all is I had Codex use browser use to check that all my agents were connected to roughly the same stuff and had access to the same skills. So just to set the baseline, we picked a couple of tasks across the top ones from my agents. I used Codex to make sure they all had access to the same things, although I forgot one of my agents, Muse, and that they all had

access to the same skills. So they were like as baselined as humanly possible, although of course they all benefit from memory and all the different things that are in each of those agents. So then I had this even kind of like plan. And then what I did is I had eight, actually nine use cases that I was going to test each of these agents against and created very specific lightweight prompts for each of these. I didn't want it to do that like sort of AI prompting where it does like a paragraph for you. I was like, keep it in my voice, do exactly what I would do. And so I plan, there's actually nine here, I'll walk you through them, run sheet of tasks with the prompt, if an agent asks for something, what you should say and how you can nudge it. And so the tasks that I came up with were drafting a reply in Slack to my friend at Lovable that asked me for something this morning. I'm cutting a short form video, fixing a bug from Sentry, building a how I

AI thumbnail, triaging my inbox for to-dos, building a family newspaper, explaining a physics question answer to me because I'm going to explain it to my fourth grader, going shopping for shoes, and then making a chat purity watch video. So those were the tasks. And then as I said, I had quad computer use run dot, and then I had codex run computer use against quadduet, GrokBot, and news, and then I had it run and time all the things and do all the results. Before we get into the full results, just a couple observations. One is, man, I'm now going to have computer use run all my agents running other agents like the parallelism of this exercise was thrilling to the heart um two I did forget to hook up Muse to a bunch of business tools I just don't

use Muse for business stuff and one of the things I noticed is as I was like the bench was running I was like oh I forgot to hook up Slack Slack isn't in the connector list and so it was just kind of annoying so it's very clear that Muse is sort of like the one off the side that's not really meant for business necessarily it's really more of a personal agent and then cloud computer use is just hot garbage so I was like super frustrated having cloud computer use but I needed something other than codex to run codex and so we get what we get we don't get upset but I did not love cloud computer use. So let's dive into the results. I'm actually recording my audio right now. So as I talk through the results and tell you what I think, we're going to put that into Claude at the very end. It's going to score all these things and come up with which one I like the best. So the sort of live human in the loop scoring that I am going to go through all these, tell you what I think, and we can see if you agree as well. So let's go for it. All right.

So this is how I, personal benchmark, Dot, Grok, Muse, and Claude. Call out here. Muse and Claude could only get seven of the nine tasks done. We will see if that was human error or agent error. But Dot and Grok completed all the tasks. Okay, so how we ran it, again, the prompts were just one line. They were very human-driven. They weren't those, like, big AI prompts. The agent did not drive itself, so the agent was driven by an operator agent. I had to like babysit things. I had to nudge things. Computer use broke, all that kind of stuff. So it's kind of in the loop. So we did a little zigzag and then actually judge line is fake. I am judging it not blind. I'm going to judge it live on this episode. And you all are going to come along with me on my judging journey. Okay, here's the entry scoreboard. And then we'll do the final score at the end. Just a couple things to see right off the bat. Muse is very fast. Muse was the fastest in four out of the nine tasks, even though it had to be steered the most.

Grok and Dot also had a couple tasks where they finished very fast, and then Claude was overall and in incremental tasks the slowest. I called this out in my recent Claude Duet review, just super slow, so I, like, forgot what it was doing. Kind of frustrating. And as you can see here, this is really funny. Claude asked for the most permissions Like Claude was the most timid so you can see it asked four times for permission where all the other ones only asked once So very interesting kind of meta view on safety and stuff like that here with Claude. So let's look at minutes per task. So you can see Dot and Claude are generally the slowest, and then Muse is consistently the fastest. Muse was the fastest on building a thumbnail, going through my inbox, building a family newspaper, and building a launch video. Grok was fastest at drafting a reply of me and fixing a bug.

Interestingly enough, it was the fastest at coding. and then Codex was the fastest at making a video and actually shopping for shoes for me though we'll see how good of a job that it did. Dot was also the slowest on a lot of tasks and then Claude for whatever reason was very slow on replying as an email. So you know this impression that I had in my Muse review where I said Muse is very fast actually held up in this experiment and then my impression that Claude was very slow also held up in the experiment so while you all may think that my reviews are not scientific they are in fact backed up by this very tremendously accurate data science okay so let's go through the tasks so the first one was replying to me it was replying to a slack it was replying to a slack to my friend at lovable who asked me to do a talk so I will say all of these were fine um Muse couldn't do it because it didn't have access to Slack again

because I don't really hook it up to Slack that's just not what I'm using it for all of them were great all of them said yes I will do this event but I did like Dot in particular because it asked for me to be paid and it asked if it was remote and if you don't know me I like to be paid to do things and I like to not leave my house. So it did feel like ChadGPT, Kodak's Dot knew me the best and knew about the things that I cared about. My schedule is busy. I need to get paid if I'm using my time, all this kind of stuff. I promise Lovable, I'll do it for you. I'm happy to do it. But it was pretty interesting to see Dot was the only one that did it. All the other ones were fine. I would call this, you know, quad is the shortest that actually did the most inaccurate response. So if I'm just eyeballing this, Muse didn't do it. Quad did short, but okay.

I would say Grok is most in my voice and Dot did the most accurate job. So again, I think any of these tools is fine. They all did fine. This is not a very hard task. Okay, now here's the fun one. I had these agents make videos for our Monday episode with Blair from Reddit. And so it actually, all it did was go into my Dropbox, find the raw video, pull it down, cut it, and make the video. Claude could not do it. Duet could not do it. It would not download a video. So Claude totally face-planted on this one. Gets a big fat donut on Claude Duet for video editing. but Dot, Grok, and Use did them. And we're just going to go ahead. Why not? Let's watch a little of them live. Not listening. What is your prompting strategy? How do you get it back on track? I don't yell at them. I know some folks do. But what I do is I take a day and then I get more vague.

And we all have bots with projects and comments. Okay, that was Dot. What is your prompting strategy? How do you get it back on track? So I don't yell at them. I know some folks do. But what I do is I take a day and then I get... That was Grok. Groff has bad overlays, but it did the like zooming in, zooming out videos better. Let's look at Muse real quick. What's your prompting strategy? How do you get it back on track? I don't yell at them. I know some folks do. But what I do is I take a day and then I get more vague. And we all have bots with projects and context. And I pop out of a project and I describe the thing large, like a movie director. So here are all the facts. And then here's all the intents. And then I pull that back into the project. and then it knows enough to apply it to what our goals are. Okay, so this is what I will say. Very, very hot takes. Again, Claude didn't do it. These are all, like, kind of impressive in that I didn't touch them. The design of the one that I like the most is Muse. So Muse is very simple. It's got the How I AI tag in the front. The caption layout is ours.

There's no funky stuff. The cuts are punchy. They don't do the zoom in, zoom out, but it's okay. The design of Grok is terrible. And it's got this, like, overlay on his forehead, which is very unattractive. But it did that zoom in, zoom out. And then I would say Dot looks very good. It looks just like the Muse one. But it, again, just doesn't have those punchy cuts. So I want to give it to Muse. I'm giving it to Muse. I think Muse would do the best. And then I think, you know, Claude failed. Grok probably the worst. And Dot, second choice. So, again, I didn't really give it much to work with. I gave it access to the transcript or raw video. I didn't touch it, and all of these cut. I think with a better skill, it could have done a better job. But I was actually really surprised by Muse because I have not had Muse do this task before, and it really nailed it out the gate. So I don't know if you all know this, but I've been using Muse a lot for social, specifically posting social. but I may start to think about Muse for creating these social posts as well because it did quite a nice job.

The other thing to call out is Grok did the worst and took the longest. So again, I don't think it's really optimized for this video cutting use case. And then Dot was, it can't tell how long it did. I think it took about seven minutes, was good. Not great at this short. Okay, the next one was fixing a bug. So classic coding use case. Again, another one that had one of these agents fail. Muse did not do it. I don't have GitHub access to Muse. I don't actually have any business system access in Muse. So Muse did nothing. I asked it to maybe assign it to Devin in Linear, and it thought Devin was a person. Devin is not a person. Devin is an agent. So Muse fell off the board. But let's look at the other ones. They all took about 10 minutes to get this done. and I actually had Claude Opus 5.5 go grade the fixes and we'll just look at them right here. So Dot Codex got closest to the actual fix. So that one did the fix that we thought

was the most accurate for the bug. Claude Duet, though, found a second bug that we didn't find, which we really, really love. Grock did the worst job, which is at loosened validation. So it was just like, we're going to solve the bug by not validating this bad issue, and then use didn't do it. So I'm actually going to give Claude and Dot the wins here. I'm going to give Claude the extra thumbs up, because not only did Claude find another bug and fix it, it also identified that Dot had already fixed it and said, I don't need to ship any more code. So I would say for coding, Codex and Claude or Dot and Duet are truly kind of the best. I actually don't code that much with Grok, so I'm not surprised that this wasn't as good. I really only do my coding in Clawed and Codex. So I'm not surprised these worked, but I was surprised at how bad the coding from Grok was. So kind of my instinct was right on these.

Opening I, Anthropic, much better at coding than GrokBot, although GrokBot's good at a lot of stuff. This episode is brought to you by Firecrawl, which gives your AI agents data from the web and beyond. Your agents can search and scrape the web while accessing Alexandra's rapidly expanding library of over 100 million index sources and more than 200 data providers, all through one connection. Connect Firecrawl to ChatGPT or Claude to find people in their work emails, pick your fantasy football lineup, or analyze trends for your next marketing campaign. Firecrawl is building the library for super intelligence, bringing more of the world's knowledge into the tools you already use. Go to firecrawl slash howiai to get started Okay next one Let look at episode thumbnails So we did episode clips Now we do episode thumbnails And I will say we did the pre work here So our thumbnail

templates are actually like quite lovely. So it's actually pretty easy to use the Sigma MCP and create new thumbnails. So I'm just going to do a hot judgment here. Muse did the worst job. muse pulled our existing thumbnail and then did this really ugly one where he's like blair is like peeking through creepy curtains we hate this i'm sorry blair i would never do this to you um and so news is gonna fail here no good um grok though dot and claude did perfectly fine jobs although I will say Dot and Claude just put new content in existing templates that we had so didn't really get super creative I would say Grok actually wins this one in that it gave me really good new ideas for these thumbnails and headlines ones that we hadn't done that I also think look really nice actually maybe we should test these while they're live they have like

that YouTube vibes for them. And so this is like a fun tip for you all is I have consistently tested Grok, Dot, and Claude Duet on Figma. And Grok is definitely the best at Figma. Grok is like 100% the best at using Figma. I don't know if this is John at SpaceX size, like Figma bro, magic touch. But if you're using Figma, use Grokbot, man. It is so good. So that is my preference. I'm not surprise this one grok takes the cake for figma and maybe i will have it create the thumbnails for this episode okay something that might not be obvious from the high level benchmark that i want to call out about how all these agents did things is they all have different interfaces for giving you to do's and so while they all found the to do's fine i suppose and they all gave me reasonable things to focus on. I am going to say that Grok did the best job because it gave me a Grok document.

And these just came out, I think, last week. And they're so nice. It's just this like beautiful list of things that I need to look at with little check marks, super organized, really nice to work with. And so, you know, all of them did a fine job finding the tasks. I would just say Grok gave those tasks to me in the best format. Muse only had access to my Gmail but found all the things I need to do as a parent ahead of Halloween week which is a busy week for parents and then Claude wanted to put my to-dos in its little like task follow-up list so I could track them. Dot did the worst job here for sure it just gave me a message with like five things I should focus on so Dot gets the thumb down for me on like the personal agent personal agent I would rank these as grok best interface best job this is what I use grokbot for my cheese's staff is in grokbot I'm not

surprised that it does a really good job there used it a perfectly lovely job with the information it had claude was fine but again I just don't like that I have to be in the claude app for this and Dot really, really failed here. Okay, family newspaper. So this is a really interesting one. This is a use case if you've used my GrokBot TradBot that I really like where we print out a newspaper in the morning for the kids to look at about what's going on for the week, what our schedule is like, what events, what news would be interesting. Claude did such a trash job that I feel like I should actually call it out. It made like a markdown file. It made me so, so, so sad. And Claude only had access to my work calendar. And so it was just like, I can't do anything. I made you an outline. And I had this experience a couple times with Duet where like it couldn't do the job and then just gave me like a markdown file that was like, this is how you can do the job.

This is not what I want at all. So Claude definitely failed the family newspaper test. but Dot, Grok, and Muse all did different jobs and I was shocked sadly shocked how poorly Dot did like look at this design it is trash trash and again these are not my children's names I just like had it swap out different kids names but this is so ugly like what model thinks that this is a good idea. This is terrible. Grok did a really nice job. I actually run mine consistently through Grok. So you can see here, it's got big week at school. It's got a don't forget list. There's Bay Area sports. There's Halloween stuff. There's nice weather across the top. But Muse, per my earlier review, just absolutely crushed the family newspaper. It generated this nice little image. And I do have three boys, and they're adorable like this.

And so it has this beautiful image at the top, a headline. It did take two pages, but I'm okay with it if it's really cute. And it has our schedule on one page and then a cute day-by-day weather, ideas, inspiration on the second page. I just think it's really cute. I think if I tuned it with a skill, it would do an excellent job. So if you're looking for family newspaper, yes, I love TradBot. please download it for your GrokBot, but maybe you tell your muse to do it because it does a really nice job with family-oriented artifacts. Okay, one kind of like side task I had these agents do, which I occasionally do on the go, is take a screenshot or take a picture of my kids' advanced homework, which I have forgotten how to do, and ask it, how do I explain this to my kids? So I I haven't done physics in a long time. Like, how do I explain a velocity over time graph to my kid? Now, I do know how to do this, but we just used it as a good example.

I just opened up the workbook. Okay, so I picked this, like, explain this little exercise and asked them all to complete it. They all got it right. The answer is C. They all explained it fine. Who won, though? Muse won. Why did Muse win? It took approximately a half a second to get this right. And so this was one where like the information was correct across the board. Muse was so lightning fast and we didn't record it here because it was so fast. It was so fast and gave me the answer. And so I've actually been moving a lot of my chat GPT like googly question and answer style chats to Muse because it does such a good job and it's so fast. So like sort of my go to like explain how this works. What is this? Look this up. I've been moving those a lot to Muse, again, because of this experiment where it's really, really fast. Okay, another personal use case I did here, shopping for some autumn shoes.

I don't know what's going to make me cool. I'm an old lady. But I want those like Bradadidas Tokios. I hear that a slim shoe is really in right now. But I'm cheap. So I want to find a good deal. And so I asked Grok, Dot, Muse, and Claude to do this. Claude totally got blocked by almost every shopping site. I don't know why. It was, like, consistently blocked over time. It also looked for white shoes with red stripes instead of red shoes with white stripes. Like, I don't know why it did it. And it found the most expensive ones. And so, like, Claude really failed this personal shopping task. Grok found great ones. So, Grok did a really, really good job, as did Muse, although Muse needed a little bit of steering. So, Grok was super fast and did a good job.

I have Grok do a lot of this, like, price comparison shopping for me. And this has been my experience where it like really good at using navigation shopping and price compare Dot did not do a good job here either So while Claude did like a bad job Dot like barely tried Dot did not do comparison It didn't look at different colors. It didn't like say like, I can't find red, but I can find maroon. So I would rank this as Grok Best, Muse almost as good. Claude not good, but at least it tried. And Dot did not try at all. So if you're using shopping, I would say Grok and Muse are your buddies. I'm already laughing because I can tell one of these are too variable. So the last one, a launch video for a chat PRD feature called Product Intelligence that I'm working on and launching. And you can just see here, they just really went lots of different directions.

So I have a skill for this. And so it should be able to follow the skill and do what it's supposed to do. but we're just going to look at the differences between these. We're going to start with dot, which is not good. It's like a PowerPoint. It's no, no, no, no, no good. I mean, this is very soothing, but this is terrible. Okay, so no, fail, I hate you, dot, bad video. No good. Kind of like the newspaper. It's like, what are we even doing here? Grock? Crushed, though. Look at this. Now, it actually used my skill. So where – oh, there's like a little error here. So the layout isn't perfect, but it at least used my skill and looks like our video is looking, but the layout isn't very good at all, and there's a lot of overflows. So better but not great. Muse ate it. Muse doesn't have access to my repo, so I did not set up our sweet news.

Reproduct Decision has a paper trail. It is just scattered. then clerkly link shows what how confident together evans first inference stayed hold on i'm pausing because although it hadn't access to nothing it made up a bunch of stuff um and gave a nice little uh british accent overview uncertainty out in the open Ooh. So I would say it's bad, but bad in an ambitious way. So I'm going to give Muse a couple extra points for effort, a couple of points for effort. And then Claude, I don't know, this is why I used Claude to do this. Claude did a very, very good job and the most good job. Your customers are telling you exactly what to build. It's just scattered across support chats, call notes, bug reports, issues, and pull requests. Meet product intelligence. From ChatPRD, connect the tools where customers talk and your team ships. So what I will say is, like, Claude nailed the value prop.

It used our branding. There were no layout issues. It actually, when asked and used the 11 Labs MCP to come up with a voiceover, I would have loved for it to be a female voice or even my voice that I have recorded in 11 Labs. But Claude absolutely crushed this task. Duet crushed this task. and so what I would say is Claude for video Claude for video Jen it's just it's just the best maybe not funny TikTok style vertical video but like these product walkthroughs those sorts of things I actually didn't think Claude Duet would do a good job and it did a really good job Muse was terrible but like terrible because of me Dot was terrible because of Dot and Grof tried Like Grok followed instructions, so I can't give it a hard time. So that is the final kind of like my analysis. Let me pause. I'm going to throw this transcript into Quad, and we're going to get the final ranking and scoring.

All right, party people, here we go. Here are the results of the very first How I AI personal agent benchmark and Grok won. GrokBot. GrokBot's the best. And we love her. She's the best. So A plus SpaceX SI team, you cooked, the potato cooked. GrokBot is freaking awesome. So GrokBot got 93% done, high quality fixes. I don't actually know how these scores work, but COD told me it's true. It won three tasks, including doing my thumbnail, doing a to-do list for my inbox, and shopping for shoes for me. So that just shows the breadth of the GrokBot. But as predicted, and this is why I tell people I use these two agents, Muse coming up with all the personal use cases. We gave Muse a break on the tasks that it couldn't do because I didn't give it GitHub and I don't give it access to work stuff, so it can't do those tasks.

But on the tasks that it had access for, it crushed especially creative ones. The short form content, it did the best. Family newspaper did the best. It's super fast at explaining homework. It is my personal agent. I'm not surprised to see it up here. Dot coming right up behind. I think Dot got kind of like a little sandbagged. So I would actually say that Dot scored closer to Claude Duet than it did to Muse because Dot really like got scored for writing a Slack like me. Like that's not amazing. And it did a good job of code and it did a good job of orchestrating all the other agents. So Codex does orchestrate all the other agents, but Dot's not Codex, but it totally failed on anything creative. It wasn't great to work with, so I would say Dot is a little bit more towards the Claude end of this ranking than the Muse Grok end of the ranking, and then poor Claude Duet pulling up the rear 79%, super slow. It did fix the bug the most completely, and it did my launch video the best,

but otherwise either did not complete or lost against all these other models, unfortunately. So Duet's got some work to do. Okay, so what are my takeaways? Pick your agent for your job, but if you're picking work jobs, probably give it to GrokBot. And if you're picking life jobs, probably give it to Muse. And OpenAI, Anthropic, get your act together. You're losing to these other agents. Access really matters, so make sure that you have consistent tool access across your agents if you're trying to benchmark them. I like speed, so fast. I used to say all the time fast speed's right, but fast is a very good part of an agent experience. Using artifacts, using different ways to communicate between human and agent, those like quality of life affordances that I call them really do matter. Grok's documents are very nice. And so the way that the agent is actually designed to present you information as a human is really important. number five safety styles differ claude's still a nerd it's still asking for permission so like

when you have a personal agent striking that right balance between permission giving and proactivity is really important i really like muses sidebar with all the permission asked so i can don't forget them um but claude was just way too conservative and then at the end of the day there are still two agents racing for my coding workflows, DOT and DUET, which means OpenAI and Anthropic, Codex and Claude. I think this race will continue because this is where a lot of the tokens are being spent, bought and used. But I would still go to those tools, but I actually wouldn't go to those agents. So what's interesting about this is while those two agents did the best job on coding, I would not use those agents to code. I would just use the underlying tools and models to code. So that is it. That is the How I AI agent benchmark. I am so happy to have done this. We now have a framework when new agents come out. I can definitely run them. And now I'm going to go back to hanging out with my GrokBot, hanging out with my Muse, and ignoring Dot and Duet.

Thanks for joining How I AI. Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.