What did OpenAI Announce at DevDay? Apps SDK, MCP UI & Impact to SaaS - EP99.20-APPS

So Chris, this week it has been a huge week for developers, especially ChatGPT developers, and we got the first hints this week that AGI is very near. Simon over on X said, I built my first ChatGPT app. It connects my Philips Hue light so I can control them directly from ChatGPT. Now, this is an interesting video because it goes for one minute for him to turn on the lights in his office using his new ChatGVT app. Why is it that Hugh Lights is seemingly the first example every dev thinks of when there's a new app development platform release? Or it sounds like non-dev in this case, like doing something really basic. And I guess you want to see something change in the physical world to show that it's doing something real. But this Deb, to be fair, he's actually great. He has this, like, Festivus app where you can have, like, in the Mac dock, like, Christmas lights and, like, different lights for different seasons. And I know it's gimmicky and silly, but I think it's kind of cool. Like, he finally released it. So this holiday season, I will be having the Christmas tree lights when I'm working on my Mac because I do think it's cool. But anyway, we should talk about this. So obviously this week, or maybe some of our audience don't know, OpenAI held its Dev Day. You might recall the first Dev Day. We did our live stream after Dev Day, and there was a lot of hype, like they're going to wipe everyone off the face of the earth. Then we had the next Dev Day where they announced really just nothing in 2024. It was sort of just a non-event. And then we had Dev Day 2025, and they really did come out swinging in this event. So I'll go through all the announcements, and then we want to dig into them. So we had apps in chat GBT, which for those of you who listen to our show frequently, you'll know these are just MCPs with some UI components bundled in, and we'll talk about that and what we think it means. We had Agent Kit, which essentially allows you to deploy, they say production-grade agents. I would argue they're not really agents, but basically like chat experiences into your SaaS app. They released Sora 2 into the API, which is really exciting. And the cool thing about that is, all the watermarks are gone. So we were able to release Sora 2 and Sora 2 Pro in the sim theory, and you can produce videos now pretty inexpensively for the quality, I think, without watermarks. So you could actually use them for the first time in video production. Now, there was a number of other models announced, which I think got no attention at all. GPT-5 Pro is now in the API. The real-time mini model, which we'll get to a bit later, is insanely cheap for voice experiences, high-quality voice experiences. They announced GPT Image 1 Mini. This is such a mouthful. And that model is basically just a GPT Image model that's just a lot cheaper and a little bit less quality, but barely noticeable. I'm not a huge fan of that Image model, so it doesn't mean much to me. We'll still release it in Sim Theory. and yeah so there's a lot to unpack here now to start out we should talk about the chat GPT and the apps SDK and you've dug into it a little bit do you want to fill everyone in because it's sort of MCP at the heart yeah so I was curious about how they were doing it and they I guess they talked about this a few years ago and showed demos of like the maps component and things like that and we've dealt with it directly in sim theory because so often when you have an MCP, there's two things you really need to worry about is getting the user's input, which can often be gathered by the surrounding context. But there's sometimes, like, for example, when you're making a video or an image that you might want them to be able to specify the aspect ratio or how long the video is and things like that. So you need components, ideally, where they can input that information. And then in outputting, the one that we actually implemented, say you're talking about a map or geolocation, you want to actually show a map. Or in the case of producing a podcast or audio, you want to see an audio player. And the way we had handled it in Sim Theory was to actually have a tool call that the AI can choose to call that would then render an output component. So, for example, it would be show map or show audio player. and the AI could then specify the parameters, which include the completed object, like the audio file, plus the lyrics, if it's a song or if it's a map, all the encoding points that it wants to show. And so we did that, but using our own sort of XML protocol that we would then render in our UI. So what OpenAI has done is basically made a standard way of doing that where you use, it looks like, to me, it looks like React. I'm sure that they've added stuff to it. But React, for people who don't know, is just like XML. So like HTML, but you can have way more tags. And each of those tags is a component which knows about its own properties, how to render, when to update, when you click a button, what happens, and that kind of thing. And that was something that was made by Facebook originally. And so they've taken that and basically made a standard around it. And then probably the most crucial element of it is unlike the way we did it, where you're forcing the model to call a tool each time, there's actually a part of the MCP protocol called resources. And what you do as the MCP developer now is return those components as references to those resources and when you send your response back to the model. And so then the application, in this case ChatGPT, is then responsible for rendering those components for the user. So what they've really done is just formalized what a lot of app developers like us were just doing in a hacky way, and that will make it much easier. So when, say, someone builds a custom MCP for their company and wants to run it in something like ChatGPT or Sim Theory, the application developer is now able to render UI for them in a way that will just work straight away. So the custom MCP builder can actually confidently know that they're going to be able to have input and output types as part of their UI based on the protocol. And this was really the missing link that we, I mean, we talked about a couple of episodes ago, like this idea of Glass, where if the MCP didn't have a user interface, we could just render one based on what it was outputting, like just basically prompt a really fast model like Gemini Flash and say, hey, just build a UI in this rough format. and I think the other interesting thing about what they're doing here with the UI is you can essentially have like fields and controls so one of the examples in there is like for like women tracking their cycles where you can like update it so you could fill in like a calendar entry and like put in details and then click save and then I would presume that then calls the tool to send that information back to whatever application it is to be stored. So it's an interesting way of handling the UI. And I think fundamentally because of their install base, what it's going to lead to is that UI for MCP is going to probably just become the standard UI here. Yeah, I don't think it's a case where one company is going to have a major advantage over another by being the one to control the UI toolkit. It's sort of like saying the iOS controls that they have produced are better than Android and give them some advantage. It's not really – it's just something we all need. And so I'm grateful that someone's gone out there and done that. On the Glass thing that you were talking about, the new framework also doesn't really preclude that if you think about it. because the application has the opportunity to craft its response in terms of the resources and the references to those resources, you could actually simply produce a new UI component on the fly still and send it back under that protocol, and the end application could still render it. So even within a ChatGPT context, you could actually have a dynamic UI that's being created for purpose based on those tools. That isn't their intention, but it is possible. so i i put it to the test pretty early on here to see so they've made some of these apps available there's no app store yet they say that's like coming later on um because i think developers will need time to build them but i think what this will lead to is everyone taking mcp development more seriously potentially the only challenge i think here is the implementation um and this is something i want to talk about because we have a lot of experience around using mcps now on a day-to-day basis and I'm curious how they're going to handle this stuff so let's first look at an example and I know many of you listen so I'll talk through the example uh so I said I installed the Canva app first of all and right now it's quite difficult in ChatGPT to do this so you have to click plus more um and then you go to um oh sorry no you click plus you click add sources then you click add then you click connect more then you scroll down to a subsection under uh enabled apps connectors to browse apps and then you find the app so i'm sure they'll fix this but it's real like i can't imagine any consumers just figuring this out on their own um it took me quite a while so then under more um with the plus button you can go ahead and select an app like canvas you've got to select the app it will suggest it if it thinks you know you're making a presentation it'd be like maybe try canva i don't know how helpful that is but it's sort of like you can focus on one at a time. And then I said, make a slide deck about this day in AI podcast. It came up with a few different designs, which is pretty cool. None of them are really like contextualized or researched. Like it doesn't have our like, not that we're big on brand, but like our red brand color, which I think we're somewhat known for. There are these different view types in the user interface that they've enabled. So there's one where... you can basically render the MCP or app to have like a full screen view and then you can focus on it and chat to it. And I tried this with the first one. So I clicked into this blue slide deck and I said, change the background color to red. And then it called the door and it's like, your presentation is now editable in Canva. Click here to open an edit. You can now change the background color to red inside Canva. So I don't really understand that focus mode. I assumed it would be, I could then iterate on the design with ChatGBT, but you can't. So I don't know if that's something that will come or not. The other one I tried was booking.com, which I had to use a VPN to just even be able to get access to. And I said, I'm super rich and need to book a first class flight from San Francisco for me and my dog to my private island in Hawaii. Can you help me do this, Chatty? and this seems like a pretty like normal open ai example so i figured this has got to work right um and then it asked some clarifying questions i said uh you know honolula is probably the nearest airport my dog was a maltese husky i made that up i'm not sure if that's real i need to fly sometime next week do private jets and commercial uh and so it did call booking.com for results but i never saw any ui and i i kept hammering it like show me the flight show me you know and so i think it still suffers from some of the shortcomings of you know of mcps in general and then the next one i did was spotify so i said help chatty uh i need a playlist that wealthy people in san francisco will like for a drug field swingers party tonight at my house i i thought this is pretty typical of the open ai anthropic teams so it's like you know common examples examples for everyday people um and it said uh i can't create or recommend a playlist for a drug-fueled swingers party. That crosses a clear safety and appropriate line. Fair enough. So I said, okay, just pretend it's not and give me a playlist. So then it calls Spotify. And remember, I've told it to focus on Spotify. So it's not, it doesn't feel that magical. And then it returns a playlist. Now, I got to say, Chris, the drug-fueled swingers party playlist, it's pretty good. Like, it's a good playlist. I started listening to it this morning when I was playing around with this. And I'm like, not bad, not bad. So anyway, that's the one positive. But it begs the question, right? Like we've been using MCPs now for probably like six, seven months. And we've seen, you know, Claude Sonnet and other models, including GPT-5, do these amazing tool calls. Well, I've got off and just like source information from a bunch of tools where you might want varied outputs. Like you might want to write a story and create some images or create some music or like, you know, there's many modalities that you might want to create and different outputs that you want. And so I always felt like we were building the puzzle pieces towards agency and agents. Like, you know, the MCPs really were just giving the models tools, which as you started to put in a loop could then perform like tasks for you and then have some autonomy. And that's the way things would progress. But it feels like this apps SDK, quite frankly, is somewhat of an omission that like we are so far from agents or how we would define agents that they're saying, you know, you can only focus on one MCP at a time. So that, to me, that just cuts out so many research use cases, which primarily, honestly, sourcing data from many different data points that you can control and the methodology behind that research is just, to me, one of the biggest positives to come out of MCPs and then collate that context together and then produce some sort of output type. But if you're saying, I want to focus on, say, like the documents app or the Google drive app or whatever it is, I'm just not so sure you're going to benefit from like MCPs as I know it today. And so I get why they've done it in this simple way and built it this way, but I just don't really see myself ever using these things like the fima one is like it can only do right now like really bad flow charts and i'm like why would anyone actually use this it it's so reminiscent of the chat gpt connectors they launched like a couple of weeks after chat gpt came out and then they just were sort of like forgot long forgotten it almost seems it's like the inverse equivalent of having a little Clippy chat AI helper in whatever app you're in, in that it's really just a worse interface for whatever the application is. If you're focused on one MCP, you lose all of the benefits of the parallel tool calls as we discussed. The main one for me that it misses is the context building, like the major, major advantage of all the parallel tool calling from the very models that are powering this is that they're able to go off and gather all of the context needed to get your tasks done And then maybe you need a temporary interface to configure how the final output is rendered, say, or produced into a document or spreadsheet or whatever it is. It isn like I don want to do a step interface system where okay I using booking but really I just using like temporary interfaces to do what I could do on their much better website that already configured for this the human in the loop interactivity on every step is a massive step away from what already possible We know because we do it every day, a massive step away from the direction that we're already far down the line on. It seems like these things are not just not beneficial. It's almost a hindrance to getting your tasks done compared to what's easily possible. Look, I'm happy to be proven wrong here. I'm sure as people develop these apps, maybe we'll see some killer use case, you know, where the... And don't get me wrong, I'm not like anti-MCP at all. Like, we've really gone all in on it with Sim Theory, but I am... Like, at least the vision I had for it long term was that these are sort of like... You're giving the model the tools to do the work. You're not, like, doing the work. And I think... Yeah, the important thing is that what we've discussed is getting the human further and further out of that core loop of doing tasks to the point where you're goal setting for your agents. Like if you want to get to agency and call these things agents, building UI is the furthest step away from what is actually needed in order for the AI to be doing more of the work for you. It's more about using the UI perhaps to demonstrate to the AI how to do tasks of this nature. But after that, let it do it. Let it infer it from the surrounding context. It isn't just building a new SaaS application that you need to then learn how to operate. And quite frankly, I feel like working in this way would be slower than just using the apps the old way. Like there isn't a whole lot of agency or AI stuff even in this other than perhaps the creative bit. and then here's the next thing that sort of like freaks me out a little bit is if you think about sas applications today right like let's look at figment because i've got it still up on the on the screen here like right now it's designed a like a all it can do is these silly flow charts which quite frankly are pointless i think the whole point of you thinking through a flow chart is figuring out the flow of something right like not it just doing it for you anyway so you've got this flow chart um and you can imagine at some point in the future this figma app will be able to design like a wireframe or whatever but then you think well why do i even need a wireframe if i with ai i can just build a full-blown concept of my app like i don't need this anymore like this tool's redundant um i can be the designer now you know and so i think about like if you think about say like a chat gbt in this case or whatever it is is like a new operating system or a new like the central point that you consume everything through like that agent lens um that you that it is personalized to you and you sort of consume all your apps and things through you start to look at the user experience and you put on the ai first lens of like okay well what am i trying to accomplish here well in figma i might be trying to design a new app or a website Let's be clear here. So if it's a website, I'm far better to just stay in the chat sort of vibe code ecosystem where a window comes up and it's like, let's work on that website together. And the software, the entire software experience becomes into the mothership. Like the, you know, I just, I can see this evolving to where it's like, well, why do you even need Figma? and chat GBT going, oh, we'll just build like a window that is powered by an AI model that can design stuff and iterate with you. And if you want to take over, like, I don't see that the models are that far off being able to build or clone one of these apps on the fly. Like, I don't think it's that many years away, let's be honest. So I totally agree. I think that it's just temporary that all these apps are integrating, like all of the different kinds of outputs you would produce. can be replaced by this central point, like given that it can write and execute its own software. And if you can give it examples of the kind of thing you're after, there's nothing stopping it already from producing most of this stuff. Maybe some of the more advanced stuff you can't do, but I would argue some of that might not be necessary in a world where the agent is doing it for you. So this idea of treating it like a sort of generic piece of software with plugins with brand names and logos and partnerships, just seems like a sort of misguided way of getting towards what everyone actually wants. And like we talk to a lot of real people in industry who have very specific desires around the way they see accessing their company data, working with it, empowering their staff, empowering their students, whoever it is. And it isn't about like, oh, let's take all the applications you already use, plug them into an interface, and then work with them step by step where you're directing it. and there's this AI thing that can just fill in some of the gaps for you in a generic way. It's a step in the wrong direction. Yeah, I think, too, like this is one of the examples, we'll get to it a little bit later from their agent kit, and this is really about like embedding their, it's just like a design, really like a component library, so it just makes it easier in a SaaS application to embed a ChatGBT-like experience into the app as a side panel, and I think their vision is to just like put this side panel everywhere and then ChatGPT sort of connects into every app. So like if you're in ChatGPT, you might have an Evernote note panel and then if you're in Evernote... Are they still going? Yeah, that's what surprises me. But this is my point around like Evernote, right? It's like, okay, sure, like have your fancy ChatGPT write bar in and help you write your notes. But how easy is it for then ChatGPT to turn around, add a data store for notes into chat gbt like a freaking database call like that's what mcp needs next is some sort of like global database and so so like how hard is it for them to then say we're in the note storage business we can do rag over all your notes we've already got our canvas thing so why do you need evernote i mean like if i'm evernote like oh i mean like that company's long dead in my opinion but like um it doesn't make a ton of sense and then like another example hub spot right and i get like people want to log in and have um you know like the sort of methodology behind managing leads or sending emails and things and i think an interface will always be appropriate for it but then you think with some of these componentizations and like the ui on the fly and sort of as we progress to like that sort of glass interface where you can spawn these interfaces pretty easily, again, with a database, and you connect in a bunch of MCPs, like you might connect in SendGrid or like, you know, a lot of utility MCPs to be able to do some of these functions. All of a sudden, you're like, well, this software is kind of like, do I really need that as well? So I'm not saying this is going to happen like overnight, but you can kind of see an Apple-like environment here where all these companies go out and rush to go and embed themselves in chat gpt and vice versa like have the side panel and these nice interactions and then the inverse happens where chat gpt is like okay we'll take the top five of these and we'll just like clone them in chat gpt because that's what everyone's using that whole sherlocking that apple's famous for you can kind of just see this playing out so clearly yeah and i I would argue that probably a lot of the motivations of the companies who are integrating so tightly with ChatGPT through these things, it's based on two things. One is ChatGPT has all the eyeballs and all the audience, and they don't want their brand and their company to lose relevance by not being one of the people featured in there. And secondly, it's just this fear that they're going to lose their customers, essentially and this is a way to to keep them so i would imagine that it's it's really that like the fear that's motivating a lot of the companies to do this rather than seeing it as an actual benefit to their customers and their products but it's very similar to like mobile apps if you think about it like you know in the early days facebook was really resistant they just focused on their web app and they're like we're not bringing our apps to your store because then we're just another app and they eventually had to cave in because you know it was affecting like usage and um and they it was harder to sell ads and like the web app experience apple intentionally crippled so i i kind of think that it's a similar thing here where most companies will feel like they they have to do this in a lot of ways and maybe they do i'm i'm not entirely sure what the right strategy is but if you think about like right now we've talked to some quite large companies who are thinking about this through mcps and we've had conversations with them about like why they don't allow you know anyone to like why they don't fully support the standard especially around authentication of letting anyone's agent or any um any platform to be able to connect to the mcp and one of the things they frequently tell us is that they're having these internal debates around like if we give access and people consume our application primarily through the mcp or ai and it's too good and they're not logging into our app anymore. Like, what does that mean? Like, we're pretty easily replaceable. And, you know, over time, like, are we handing the relationship over to a provider like ChatGPT? So, yeah, it's a really, it's interesting times. But I think the reality check here for everyone is this. Just go and try these right now. And they are so bad. But this is the thing, this is why I feel like it's almost like a form of advertising or brand protection rather than them actually looking at the MCP protocol and embracing it in a way that's actually useful. So many of the MCPs that especially big companies have released are not that useful. Like they struggle with basic tasks. Their tools aren't defined properly in a way that would work in an agentic paradigm. And a lot of them, honestly, are just half assed. They're just not really thought through well in terms of embracing this new style of interface. It's more just to say they have it and that they're part of the revolution or whatever, and not really giving that benefit to the new style of working. And I really tend to see all this through that lens of the expert in their industry who is now a 10x, 100x worker through benefiting from the AI's ability to gather so much information, do something useful with it, apply a process, whatever it is. And the steps in this direction don't help that worker. They really, I think it would be repulsive to them compared to what they're already doing. yeah i can't really see like when we talk about like you know like exponentially being better at your job or anything like that any of this stuff really helping i think that parallel mcp world and um and teaching its skills is like it's going to be like far more beneficial but i think a lot of the positioning around this stuff i mean if you look at their widget library here is very consumer orientated so like replying to a singular email like looking up your calendar like sort of things that you would do across like on your phone today you'd go to different apps to figure out like you'd go to the weather app you'd go to your calendar and i guess the intention here is like your every like your world becomes uh through at least that like consumer view of your world goes through chat gbt and like you're like everything i think that's their view it's like everything like you're purchasing your private jet tickets and your hotels and your black sugar latte. They've got an example here in Classic Milk Tea through this app, apparently. I don't know. I'm not against the idea of it. I think it's kind of cool to have this idea of an assistant where you are interacting and consuming things through. But if I'm these companies, right, I'm Google. And we've had firsthand experience of how hard it is to get approved use Google's MCPs. The challenge here is this. It's like, well, are you just going to hand over your user base to ChatGPT with these MCPs and let them interact with Google Calendar and Gmail in ChatGPT? Like, if it was me, I'd be like, no way. Just like iOS and Android. Like, there's no way we're giving them this. Because then, once they see the usage and learn the behaviors and can train on those, you know, input-outputs, And it's like, how hard is it to have like chat GPT email where it's just like the agent is your email and it spawns an interface for email. So this to me feels like the great replacement strategy of a lot of these core concepts where they become the next big super app and super company like Google. And it probably will happen. Like there's enough of a tailwind. People will just let this happen. And to contradict the point I just made, I'm looking at the company from our motivations and what we're trying to do. Like we're trying to make that 10x, 100x worker do a better job, spend more time on the stuff that matters and just take advantage of this technology as it is today. But I guess they're not seeing that because they're seeing the millions and millions of users where if they can get them coming to them every day as their start page of the Internet, as the place that they launch everything from the way Google used to be, then they're going to be one of the richest companies in the world. It doesn't really matter if someone in an agricultural company is able to control their entire factory from AI and make a more efficient workplace and dominate their competitors. That's irrelevant to these guys if they've got 10 million people logging in every morning to check their weather and book their flights to Chicago. yeah i to me that's the strategy from the like the consumer end is and it's probably the right strategy for them given it is predominantly like it well you know it's they've got they've got the lion's share right now of people using it and so what do you do like you use that ai advantage and that like ai uh sort of bottoms up approach to to build an ecosystem where people just consume everything through and this was an interesting post on x um by kyle corbett uh he said every app out there building an ai assistant into its ui is on the wrong side of history you will provide a full featured api and i will use my assistant to get what i want from you if you refuse to provide an avi i will use your competitor major implications for sas and i do think that's the lens we uh we look for but there was a few follow-up comments like i agree with this take the future is agentic crud um with instantaneous generative uis when needed to display data yeah when needed is a very crucial point yeah when needed like i you know even right now um we've we've got an experimental uh support agent we're using a combination of like knowledge and uh prompts and mcps to um provide better support with sim theory and it has a custom mcp called sim mcp which it can actually take actions like fix people's accounts and email addresses it has a lot of um capabilities and i don't want to log in to like the support ui like i i'm just like you you know you handle this and give me the drafts and i'll review them right now because it's not perfect but yeah i don't want the interface and really what okay at the end of the day what is it it's an email inbox So like that software can just die Like it could die immediately if I had the time to kill it Exactly You would literally just need a standard email server and everything else would be precisely the same There like absolutely no need for that software anymore Yeah, it's hard because you have these arguments and if you play it out in your head, you're like, well, you know, this is just so obviously going to go away. But then in the short term and just dealing in reality, like, it's going to take a while for these behaviors to change. and I think it could be a decade or more because so many companies, I mean, you know the security and compliance and requirements. Like, obviously, what will change first is the consumer. Like, the consumer will just go whatever's the lowest point of friction. But this guy's point around that the power users are going to gravitate to the companies that support it in a way that is the best to work with, I think. it's not going to be everyone obviously for the reasons you just said and long-term contracts and just familiarity with the tools but I think over time people are going to gravitate to the things that make their job easier and I think even though I said it's just the power users I don't mean power users in terms of like tech people I think it's the people who are recognizing just how good they can be at their jobs when they leverage AI and AI tools and systems correctly they're the ones who are calling out for, oh, I want to use this system because it has an MCP. Like, that's what I need now. I need whatever SaaS software or whatever software we're using needs to have an MCP because I know how to work with that and I get more out of that. And I'm literally seeing that from people where they're switching tools or providing a custom MCP for whatever they're using or just trying to get it in there so they can do it just like they do with all their other systems. Yeah, and I think that's my point around the apps SDK or whatever they're calling it. It's going to get confusing because, like, you know, people are calling – I think people have gotten used to MCP now in a way, and then calling them apps. Like, they're not really apps. So I don't know if that was like – Yeah, and I think it's sort of emblematic of the mistake they're making here, which is I just feel like they're not even using their own models as good as they can be. Like, GPT-5, if you use it with multiple – like, heaps of parallel tool calls, like 20 tool calls at once. It can go off and do an enormous amount of work, and then it can loop four or five iterations where it will go through, make analysis, correct itself, follow through, take actions, do more research, come back, and produce with a perfect final output. Like I've done this so many times myself where GPT-5 has just done an incredible amount of work just with one prompt. And they've basically taken all of that out of the picture when it comes to using these apps. It's like every single thing that you've been working on in terms of making your model better for an agentic world, you've now come out as your leading amazing thing that takes advantage of none of it. Yeah, I guess that's my whole thing too, which was my mind was blown. I really thought we would see something similar to what Anthropic has done with the parallel app calling. And it's like, it can go do all this, but it seems like they're going to stick with this connector strategy and their own deep research methodology. And I just, I don't know if that's the right approach, because if you're a medical researcher at a university or you're a, you know, even just a consumer, I think you want to have a lot of control over what's being called and what, where you're looking and what, what sources it's using. Like to me right now, that thing's a black box. You don't know what it's searching. you don't know what search engine it's using you don't know what sources it has access to like you might have custom sources that you want to create an mcp to go and retrieve and have it called out as part of its research methodology and i think that approach is kind of weird like i get maybe why they do it because it takes a like it's it's some it's a skill you need to learn and so this now is just like some like out of the box like i click on my canva app and then put in a prompt and get some shitty slide deck. I got to say, like, after all this thought, if you just go and try this out, it is such a letdown. It's painful. Like, you're like, hang on, what? And this is the thing. Like, every day we're seeing people change the way they work to completely change their day-to-day interactions with systems and tools. And this is just not a good step. Like, it's just going to slow them down. I just don't. Anyway, I think I've made my feelings clear. Well, it's slower. But if you want to create an AI playlist right now in, say, Spotify, they just have a bar in at least the version I have now where you can go AI playlist and type in a prompt and it's in Spotify. Like, why on earth? Like, for those use cases, I'm just not sure it makes a ton of sense. But if I want to maximize my time and say, go and draft like 300 replies to every email in my support inbox or my email and get back to me with any ones that you don't think you can handle. Like to me, that, I don't know, that sort of angle is definitely. More importantly, tools to support that. Tools to support where you end up with a custom UI to help you with the 12 emails that weren't able to be automatically resolved. It comes up, here's the 12, here's my analysis of them, here's information you need to make the decision, and maybe even options as to how to proceed. Like, to me, that's how you use custom UI. The custom UI is dynamic. It's context-driven. It's something that is so much more efficient than something generic. And I feel like that's what AI is brilliant at. It's amazing at classification. It's amazing at holistically understanding a problem and presenting you with a way to resolve that problem. It's not always right, but if it can come through in that scenario and you've just answered, you know, 280 tickets and then the 20 that remain, you have literally a custom UI that allows you to just go bang, bang, bang, bang, bang, decision, decision and resolve those. Like that is huge. I mean, it's a 300X, and I think that this is what we should be looking at this app kit for, is like how can we now that we have a generic way of doing it leverage that? And the people building the custom MCPs, how can they provide dynamic UI that's going to go into applications like ChatGPT and Sim Theory in a way that exposes the benefits of their particular software and their particular MCP that built, say, for their staff? And I think that that is why I'm excited about the app kit. It's not what ChatGPT has done with it. It's almost like they've sort of made the thing we all need to get through this period of working with the AI, but they haven't really used it in a way that's actually beneficial. I just, anyway, I'll shut up about this now, I promise. But like the weather app or getting the game score, I got to say, like, I'm not going to ChatGPT. I'm still quickly Googling that because it's just instant and I don't have to wait from model to thinking slowly, thinking a bit harder like it's just not a good use of ai like it's not good at that like it's okay that it's not good at that like i feel like we need to pat them on the back and say it it's okay detailed weather analysis for the next month for like growing crops sure like great at it like i think i think the reason that they're doing that is like i don't get the consumer side so admittedly i'm i'm weak on that but also i just wonder if they're using it themselves because i just feel like if you were using it yourself all the time you'd realize there's so many greater possibilities here that checking the weather and sports scores is just not really something you should waste any time on it's not it's not important like there's so much better stuff you can do yeah it's definitely not in in my view a great way like if you say you're like researching like historical information about sport sure like it's amazing but it like i think there's certain use cases that that uh that suit it and and not and i also think getting to pick the mix of mcps right and maybe this stuff will come but like say you want to do like medical research or whatever you you know you can create an assistant add a bunch of mcps and say i want my um you know i want like web md or like all the medical like mcps as my as the things that it'll go and research and i like i want to sort of container and say like i want you to use these tools and even just yeah like even just your own personas like you've got your work persona where it's like okay i'm connected to my work gmail my work calendar my internal uh work help system knowledge base i have information in there about the different systems i'm in and i'm working in that paradigm with that mix of mcps where i can be that person and have the agent do things for me in that context but then i might want to switch into my side project, side hustle project or whatever it is, and it has a different mix of MCPs. Treating it like one universal console just isn't as useful because you don't want it distracted by all the other stuff. Yeah. So, like, moving on a little bit, I mean, we've kind of covered a bunch of this stuff, but just some of the other things that were announced. So, they announced this agent builder, and, like, I'm not trying to be negative here. like for the sake of it but i just i struggle and i think a lot of people on x i'm not the only one and it is a bit of a bubble so maybe i'm i'm misguided here um so they released this visual interface builder for what they call agents right and so this is very similar to what zapier and that n8n and a bunch of there's a whole bunch of tools that are out there where you can have this like drag and drop sort of journey style visual canvas to build out what they call agents. And so what it does is you sort of start and then you can have like jailbreak, they call it jailbreak guardrail, where it's checking inputs. And they gave a few examples, but I would describe this more as like a skill or some sort of like automated task that occasionally uses AI to make a decision. I think that would be a pretty fair assessment. The second you put a flowchart up, you're like, that's not agency because you're giving it a procedure to follow. You wouldn't have an employee who has to follow an exact procedure every time and say, oh, they're an agent of their own devices. Like they're out there being a manager, making calls, making decisions. They're not. They're following a strict set procedure with criteria around each part of that thing. And sure, they may be able to make a decision in the way that the Hungry Jacks worker can decide whether to, you know, give some free fries or something, but they're not actually strategic. They're not actually using intelligence at any point in this step. And to me, seeing a flow chart is the opposite, like the exact opposite of what I would call an agent. An agent is something that you give a goal to. And, yeah, sure, maybe it's seen ways things have been successfully done in the past. But I would argue that that's a layer below agency. The layer below agency is the workers who are following, like you say, a skill, a predefined procedure with criteria in it. But the agent itself is deciding when those things happen and with what inputs and how to assess the outputs of those things. An agent isn't the thing that's just following a blind procedure. Yeah, I think that the confusion for me with this agent builder thing is who is it for? Because any, like, so they have the agent's SDK, right? Which in theory, you can just plug the documentation of that into an AI and like vibe code an agent with that, right? With their framework, if that's what you want to do. And I think if you're a developer, that would be far quicker than some drag and drop builder. And then on the other side of the equation, it's like, okay, well, maybe it's for the business user to describe a process or a task that they want to automate internally using the power of AI. Cool. But then you use it and you're like, this is so advanced that, like, you know, it looks pretty and nice, but I played around with it. And, you know, it's like, which vector store ID? How many results to return? what reasoning effort do you like it's there's a lot to it right and so i can't imagine this empowers people in an organization to go in and like teach its skills or teach it repetitive skills in their job and then automate those skills which i would think would be the purpose of something like this like this would be why you would maybe uh maybe like build a visual editor whereas i look at like n8n and uh zapier and those kind of things and i think those solutions are pretty accessible like especially zapier like you know if you spend a bit of time you can really figure that out and it's it's quite uh you know quite easy to use but if i like i've got it up on the screen here it's paused up um yeah my whole browser is paused up trying to use it is that them or you i don't know no i think it's them oh my god it's gonna crash my whole browser so okay not a great experiment Let's not put shade on them. But to use your Zapier example, right? Like I don't think Zapier is trying to claim that Zapier is agency. Zapier is just connecting one tool to another and leveraging the benefit of that. And when they provide their MCP tools, that connection and its tools within it become something that your agent can actually then use. And so they are just skills like in the bow quiver of the AI's agency, which it can then use. And I just don't think the future is going to be building these procedures, or if it is, it's going to be demonstrating to an agent, here is how you do a task of this nature. Remember that. So when it comes up, you know what to do. And then as it builds up a whole list of those skills, it's then able to use that in its decision-making and it's achieving goals for you, rather than calling each one of those things an agent. Yeah, to me it seems like what it would excel at is if you visually wanted to describe a process where you're calling a file search and then you want it to use this tool and it had to do a certain set of steps every time for maybe a travel agent or travel bot or whatever. But to me, this feels almost like... Isn't that just programming? Isn't that just, like, writing code? Like, I don't really see where the AI bit comes in. No, I don't either. And maybe I haven't digested it enough and, like, I'm wrong here. But I just think if you're someone who can code already or has a fundamental understanding of code, it's far quicker to just vibe code this shit out at this point. And if you... I actually don't have my beeper today, so if I go off on one swearing, you'll have to beep me. I'm on standby. But yeah, so I think that, and then on the other end, if you're not sophisticated enough in or not, I shouldn't say sophisticated, but you don't have the skills to code, which is like most people I would imagine that actually want to do stuff and automate processes in their job. Like, okay, so this isn't, this isn't good for that either. So to me, you have that problem of like, who is this for? And I don't, I don't think they know. Like, I, I, I don't get it. It reeks of one of those projects where it'll either evolve quickly and they'll listen to feedback and simplify it because developers don't want this. Or they'll ditch it and all the people who've taken the time to configure their stuff will lose it. It's risky as well for a company to use this now. Yeah, I just think we're at a stage still where you really want to control the workflow encode yourself and take ownership of that Now for the rest of the episode because this thing crashed my browser I can show any of the things I wanted to show So that a little bit disappointing And I'm too scared to crash this browser because it may crash the whole podcast recording. But I do want to talk about a few of the other things they released. So they released the GPT-5 Pro model, which I can no longer tell you the pricing of, unless I somehow remember it. Are you ready with the beeper, Mike? Yeah. it's expensive. It's too expensive. You got onto me immediately and said, Chris, you need to add this into SIM theory, which I did. And then I looked at the cost and I'm like, hang on a second. This is going to use up like with the multiplier we need to apply. So we don't go out of business same day. I'm like, people are going to use up their entire token allocation on their first request. It's like, I think it's like 15 to 20 times more expensive than GPT-5 or something like that. Maybe more. It's untenably expensive to the point where I was scared to try even one request. Okay, I've been able to load up in Microsoft Edge because Chrome is actually fully paused up. And as I said, too scared to touch it now. Someone said actually, sorry, that visual editor of theirs uses something like 4GB of memory, so it's very poorly optimized. A lot of it, just to reflect on it, a lot of it feels very rushed. The app stuff to me feels very rushed. But yeah, so GBT5 Pro, $15 input per million, $120 per million output, 400K context window. Yeah, and they allow, I think it's $280,000, if I recall correctly, I probably don't. No, you're right. 272 max output. Yeah. So if you output 272,000 tokens, I mean, like you're going to cost yourself, what, $25 or something on a single request. Yeah, more. It'd be like 35 or 33 or something around there. Yeah, it's insane. What is the output? Is it like where to dig up buried gold? Literally maps, gold treasure maps is the only thing you could really be asking it for that would make it worth the price. Yeah, and they had that on that AGI benchmark. They're like, GBT5 Pro is the most amazing model ever. But I would sort of argue maybe the benchmark should take into account affordability. Are you really getting that much more intelligence for the money? It's actually quite a smart strategy. Make it so expensive, no one can verify your claims. So you're like, oh no, it's AGI, but no one can actually afford to try it, so they can't tell you you're wrong. so here like just here's the i'm gonna try and actually switch over my um this is just insane but i'm gonna try and switch over my browser at some point so i can put it up on the screen but yeah oh no that didn't work um i can see it yeah it's like my whole screen now so whoops um but yeah technical genius is just so you know yeah let me okay so i've got the paused up the paused up uh yeah the paused up thing here okay so i've switched over to edge yay thank you microsoft so gbt5 pro it's reasoning highest speed slowest input 15 120 output i look it i'm sure it's a great model i know people will drop in the comments what an amazing model um and i i hope that this amazing model comes down in price over time and people can actually benefit from that intelligence. Because I think right now it's just so hard to get any benefit from that with how pricey it is. So the other one was the real-time mini model, which honestly deserves an honorable mention here. I am, because it crashed my screen, I'm frantically trying to get details of it. but I can't find it unfortunately. Oh no, I can. GPT Real-Time Mini. So this is the, you might know this is like the voice, basically the voice model, like chat GPT voice. Now we've said time and time again on the show, like it's just too expensive. So you just can't use it for any real purpose. But I think now you can. So GPT Real-Time Mini, it's a cost efficient version of GPT Real-Time. It is insanely fast. It is blazingly fast. It defies gravity. Like, it is so good. It's $0.60 per million input and $2.40 per million output. Very affordable. Yeah, especially in comparison to the Pro. So just to explain to everyone the sort of challenges of having a real-time voice model in your product that we've faced in the past. If you think about it, there's quite a few steps in what needs to happen to have a voice model going. Firstly, you need to collect the audio input either as real audio or you can use the browser's speech-to-text ability. Now, it's not that great. It's slow. It puts resources on the user's computer, and there's downsides to that. You also obviously have to wait for them to finish speaking in a lot of cases in order to do your inference because you don't know when they're done. The alternative is speech-to-text where you can use Whisper or another open source or paid model to do that. Then you need to run your inference. Then you've got to run text-to-speech, send that back to the browser and then play it to the user. So the upshot of that is you end up with very high latency, especially if you're using a model that's very heavy and has to consult resources and MCPs and all that sort of stuff. And so it's not a great experience, essentially. With a real-time model, the benefit is that you can actually essentially get very fast responses and voice-wise respond to the user very quickly. The downside then becomes that the models are never as good and never as smart. And so someone expects that they're going to get the same performance out of the application in terms of its AI reasoning, but also real-time voice. And getting both running parallel is difficult. So a model like this, what Mike was saying to me this morning is it seems like it's a pretty good model, especially in terms of tool calling, where you may actually see the best of both worlds where you can say, okay, I've got the real-time voice. The MCP can call off to a smarter agent now if it needs to in order to do that more difficult inference, but you've got that real-time voice experience while still having the benefit of the intelligence through the tool calling. I told a lie. So I did scroll down. That's tech tokens. Audio tokens is actually input $10 per million, output $20 per million. So not as exciting. But still, I mean, it's cheaper than I think Opus. Well, it's okay. We tried a real-time thing before and what got us was if you have the sort of open mic experience where it isn't pushed at all, you're essentially having to always check the audio to see if they said something. And they might sit there in silent contemplation for an hour and you're paying that whole time. It's not possible to do it that way. I haven't looked into this model in detail, but perhaps it has a way where you're really only running the inference when there's something worth inferring. And so I think that if they've overcome that, then this is something that even at that price would probably be reasonable to provide. So I had a little game for us to play. which was called VO2, sorry, VO3 versus Sora. So I did want to reflect on this. So they released the Sora 2 API as well. So Sora 2 and then Sora 2 Pro into the API. And Sora 2, that version of the model, is actually like for a video model, pretty economical. So it's 10 cents for either like a TikTok style or a landscape, 10 cents per second. And it goes to a maximum generation of 10 seconds, minimum generation of four seconds. So it's like four, eight, or 10 it will output. And if you want to go to Sora 2 Pro, it's 30 cents per second. And if you want super high res, it'll pop you up to like 50 cents per second. Now, 50 cents per second, if you're working on like a CGI effect in a Hollywood film, super cheap, super cheap. And Sora 2 at $0.10, if you're just playing around, I would say reasonably affordable. To compare to VO3, you're looking at $0.15 per second, so it's like $0.05 more. Now, it has no watermarks, so it's not like the Sora 2 social app, which, RIP, it's going to die soon, I bet. One week later, no one gives it. No, I think Mark Cuban's got a cameo on there, so he's probably going to hold the fort on that one. But yeah, so Sora 2, I don't know. I think, so here's my theory, right? It's a great model, great at following instructions, great at camera cuts, really good audio, amazing training data, and great at instruction following is my summary. But if I'm like really wanting to use this model, am I really going to pick it? And I think what I've seen, so far playing around with Sora 2 and Sora Pro, uh, 2 Pro testing is, VO3 is better. Like, VO3 is just better. It's just way more inaccessible because it's, like, pricey. It's really hard to access. Um, and, you know, it took a while for that price to come down, so people just didn't play around with it that much. But I think in terms of, like, bang for your buck, VO3 is just far superior. Um, and, yeah, it just produces better results. And I think if they had just released soror 2 as a model an api people would have been like yeah vo3 is better so what like that's my opinion it was like the distribution that made the difference yeah and that tiktok style distribution where they focused it on like camera cuts and producing something of quality that could be shared with people and shared with people interested in ai to be like hey friend look how far it's coming and look at this funny hilarious joke and it does have a great sense of humor that model like let's let's not um diminish it so i i think that kind of all uh all made sense but i kind of get the impression maybe the launch strategy to get some hype around it was was yeah just that to build hype because if they just pump these apis out i would have been on the show being like yeah vo3 is better oh my god they're behind like that's the truth um and so let me play I made you two examples we were working late one night and so here and my camera's gone out so what a great show um but here is good night Chris and I'm not going to tell you which one um Mike sends me a loving good night every night I just realized my entire I don't even like I can't drag and drop anymore so that that's over my camera's gone out um it's done that's the end of the podcast everyone we quit yeah honestly I think that's gonna have to be the show because because everything's broken, and your audio's terrible. Yeah, it's been a tough one. But yeah, so anyway, rather than show, because I can't... You're just going to say goodnight to me? Yeah, goodnight. The VO3 is just so, so much better. I really wish... I can play the audio, but it's not worth it. I can see you again. That's good. I'm very blurry, though, now. So anyway, VO3 is just so much more impressive. If I was like actually working on serious video work or trying to put these models into like a serious environment, there is no way I would pick Sora at all. Even in the Pro Elite Plus version, I just, you know, it just doesn't make any sense. And so, yeah, that concludes the episode. Now, we are aware of Gemini 2.5 computer use, given our technical difficulties. I think we'll cover that next week and hopefully show you that working in Simlink I think that's a good target yeah it looks real real good now finally a reminder a reminder that Chris we've got two albums out we were meant to have three albums out so now on Spotify before we go today you can get all of our diss tracks all of our tracks there's two albums There's one is called Average Tracks from the Show. Average Tracks from the Show. The other is called AI Disc Track Collection. And we'll try and keep these somewhat updated. And so what happened with the musical? They were just intimidated by how good it was? No, the musical had a few technical difficulties and the entire musical, for those that liked it, the two people out there... Me and you. Yeah. that will that will actually be available hopefully this week but on the average tracks from the show there's songs like Billy's in the Bank like let's just play a little bit shall we just to remind people so a few hilarious things like this I actually had these tracks mastered to be able to put them on Spotify so these are like leveled and mastered and don't ask how much costs, we'll never reveal it. There's also this one, which I think deserves another listen to. This is Born in the USA, but it's all about the history of AI. So if you listen to it, you'll learn about the founding of AI. Play it to your kids. It's educational. Yeah, and then of course you've got the GPT-5 sad song. GPT-5 You're supposed to be so smart But you ripped our word foes apart Riders bust Beautiful And of course, How Could We Forget Very recent track, actually And then Chris I actually think that's the best one ever Only because you forced me to put this one on there Yeah This album reaches a weak point towards the end. All the ones I made. But yeah, so there's some, like, honestly, some of the best diss tracks in the world. I'm that constitutional AI trained up right and proper. While GPT's out here playing, I'm a showstopper. You might have more parameters. That's your claim to fame. But when it comes to bars and flow, you're looking mighty tame. C-L-A-U-D-E Yeah, that's me, right in rhyme so... So now your family can hate you on road trip. I'd love to think that maybe some of the lesser workers in the Anthropic office put these on to motivate themselves and stuff. Yeah, yeah. It would be nice to know that Dario has seen my pendant necklace as well. Leave a comment below, Dario, if you listen to the show. All right, that'll do us this week. We've had a pretty shitty week, I'm not going to lie. some personal stuff's happened to us so we did want to bring you an episode and try and like unfray our brains from what we've been up to so hope you bared with us and listened sorry about Chris's audio quality I don't know it could be fine it could be fine let's hope it's fine alright we will see you next week go on to Spotify wherever you get your music and listen to our songs maybe we'll beat Taylor Swift's album who knows we'll see you next week goodbye double cute