OpenAI's Dev Day: New App Interactions Explored

OpenAI's Dev Day is always something to mark your calendar for. I feel like they were a little bit more exciting in 2023, 2024, and now it's not like, oh my gosh, here's a new model. But although I don't think they really tend to do new models, but there are hints at this OpenAI's, a past OpenAI's Dev Day, that we could be looking at new ways of interacting with apps, with agents, things like that. Or is it just something that looks cool right now and is going to crash and burn? And I'll be honest, I was listening to our friend and watching how Ali Miller, our friend Ali K. Miller was there and kind of giving us the rundown on the same with Rowan. Chung giving us the rundown on all this. And it was interesting to see, Jaden. But what I would say is that the apps part is really interesting because it gave me a little bit of plugins. If you guys remember plugins, plugins vibes. And I keep thinking, is this a new way of working or is this a do I even want to work this way? So I don't know. Jaden, thoughts? Yeah. Yeah, so I mean, the first thing I'll say is that I actually don't think it's anything new from OpenAI. Like you mentioned, they have plugins that came out like a long time ago. So there's basically this big dev day and they're like, it's amazing. You can chat with apps inside of ChatGPT. I will say it is more useful than it was in the past. But they had plugins and the next evolution of plugins was the custom GPT in the GPT store. And now the next thing is that like just inside of ChatGPT, the same way you would call a plugin. Like in the past, you had to hit a dropdown, select the plugin, then chat with it, and it would do it. But it's the same thing today. Like you have to call Canva. You have to say at Canva, and there's a dropdown. You click on the Canva button, and you're like, in Canva, do blah, blah, blah. Is it useful? Yes, it's useful. So not to kill the hype, it's super useful. I've actually tested it out two different occasions. One was good, one was bad. I'll give you the rundown on both. But like it isn't new is my only thing. So, like, I don't know why they're getting a big PR thing for it other than, like, I guess the dev day. We got to keep it interesting. There's more interesting things on dev day. We'll get into that. They're, like, agent workflow builder for developers, I think, with Sweep. But basically for everyone listening, this is very useful. So I'll tell you what I think is most useful about this. Number one, Canva. If you do anything with designs, and if you don't, now you can. It's so easy. Canva was already easy. It was, like, the easier version of Photoshop, man. And like, I never, I haven't touched Photoshop since college, even though I was literally a wizard with it growing up because Canva was just so much easier. And like, you get like 80% of the functionality, maybe 70%, but it's like just a hundred times easier. So anyways, Canva is amazing. Basically what you can do with chat GPT now, and maybe I'll even share my screen and demo it for you. I'll tell you what it didn't do good first while I'm pulling this all up. What it didn't do first was Zillow. So if you go to and maybe my expectations were too high. Basically, I know everyone has heard me say this 100 times, but I will say it 101 times. And this is a great business idea for literally anyone listening. Although someone said maybe for legal reasons, you can't do it. I don't know. My wife really wants a Zillow app, which basically you ask for like a house in a specific location and or, you know, like your Zillow thing. But then you have it overlaid with the FBI crime statistics map because that is literally what she has open in two tabs. Whenever we're looking at like places to move or houses, it's like, how safe is this? It's like, wow, look how cheap this amazing house is in like Michigan. And it's like, oh, yeah, that's in like a drug like crime zone war area. So, you know, you never know with Michigan, right? No, I'm just kidding. But no shade at Michigan. But like this is literally what she looks like. How safe is the neighborhood? Because I think we have PTSD. Our first house we bought, we didn't know anything about the neighborhood. Not a great area. The house was super cheap. That was great. but uh we quickly moved to a better area anyways someone got to build that and i tried asking zillow i was like hey you know like a safe neighborhood and blah blah blah blah And it basically was just like sorry we can do that because it not the right like it not the search parameters of Zillow I was just talking to ChatGPT. But it's kind of tricky because it's like, if you have to get the search parameters or like basically only settings that that app supports put into there for even to like try doing, like try using the tool that's not useful. Basically, what it should have said was, sorry, we don't have any data on safety, but like we can do your other search parameters of like a one acre home with a lake view in this city. And it should have pulled that up. So I think maybe they're ironing out some of those those bugs basically with the apps. OK, so that's my caveat of what didn't work. I'm sharing my screen now, but for anyone listening, I'll explain. But I guess, Connor, any thoughts on like if this is still worth pursuing, even with some of its glitches? glitches. So this is, I mean, what I'm dying to see, like, and I'm watching what you're doing right now. So what, like, you know, again, Jaden has ChatGPT open. You've hit on, and I haven't used this yet, sort of like you're calling, it's starting in the same way, you're kind of like tool calling in a way, like in the same way you might say like, hey, let's, you know, use whatever, DeepThink or something like this for this answer. So Jaden has kind of like dropped, used a dropdown menu, had like, you know, Canva, some other things in there. He's now using Canva. So now he's typing in a prompt and we're going to sort of see if I just don't understand the interface. And Jaden, sort of as you're doing that, one of the reasons I'm skeptical, and again, maybe this is going to be very old school, but it's the same reason I'm skeptical about was skeptical about plugins, which is Canva is already unbelievably easy to use. AI is coming to all these tools. Like I'd rather have GPT in Canva and GPT and all these other things rather than need it here and think I'm getting a less, I don't know, effective product. But I guess one of the things that I'm thinking is if all of a sudden this is like where everybody lives and you just have like one site to sort of like rule them all. Well, I don't know. I guess we'll I guess we'll find out. So, Jaden, explain sort of like what you're doing, what your experience has been. OK, so the first thing I did is I went at Canva and I added the Canva app. You can like there's a little plus button on the chat where I think if you click on more, if you've used anything recently, then your most recently used plugins will show up. maybe if you click add sources, no sources is different. Anyways. So you kind of have to know, like there's like a handful of apps, there's like five or six apps that are currently on there. So you just have to hit at okay, yeah, if you hit the at symbol on in chat, GPT, it will pull up a list. And also what's interesting is it can also at and I think they've been doing this for a long time. I can't remember. I feel like I vaguely heard this, but like custom GPTs that you might have used in the past. Yeah, you can you you've been able to add those for a while. Yeah. Okay. So you could talk to your custom GPTs or I mean, you could just click on them and you're in your GPT thing. So I just made a business card by going at Canva. And then I said, create me a business card for my company, AI Box, make it super modern and techno cyberpunk. My name is Jaden. I'm the CEO. Hold on. Let me let me interrupt you, because like what I see there is images that it's created and I see that it's used Canva and they look pretty good. But can you edit those or no? Like what's the. So the cool thing is if you click on it, it will zoom in like normal. And in the top right corner there's an open with canva button if you click open with canva the design that it made is literally editable you can go and change it to be whatever you want so that to me is like actually useful like we're all used to making images with chat gpt like that's nice but like imagine you can actually go edit them and they're like they're way more like templates like there's text all over the place so they're they're using canva and they're editing it's like template although some things it looks like were AI generated images. Yeah, okay. So it like it that so interesting which is a downside but whatever Part of the design it generated for me was an ai generated image of like my logo basically but it misspelled my logo and i can edit that because it's not text but part of it's text so i don't know it's a template it's not perfect and maybe i could go regenerate this one particular image if i if i cared so much but it made an editable um like business card well this is what okay well let me jump in here because this is what's interesting about this so i kind of like love this episode actually because we're trying like trying this out in real time, because I think so often we just read about these things and we can make, you know, is this good or bad? But so, guys, what I'm seeing is Jaden is doing this, is that Jaden is clicking on a drop down thing that says Canva. It creates something that looks like just a normal chat GPT for this in particular. And, Jane, maybe while I'm talking about this, like I'd love to sort of see what it did when you dropped down Zillow as well. Sure. Oh, yeah. I realize I wasn't sharing my screen on Canva. So, yeah. Yeah. OK. So he's so Jaden sharing his screen. So essentially like that. This is one cool functionality. It's created an image. And so it gets kind of like what, you know, maybe you wouldn't have thought of. It sort of like helps you design something, open it in Canva, and then you can start manipulating. That's cool. Okay. But like the question is, will people actually do that? Maybe they will. Maybe they won't. I'm just not sure that this is how people use ChatGPT and think about ChatGPT. I think that maybe we're in the very early stages of this, but it just feels weird to sort of like have ChatGPT as the gate. And then you kind of like go into other apps rather than just kind of going to other apps. Maybe I'm kind of thinking about this wrong, but I'm wondering if this will catch on in the same way. But but Jaden, I can see now what that would look like for Canva. Give us a sense of Zillow like you want to do this in Zillow. It opens that up. And is it the same thing where it like opens up a house for sale? And then you say, OK, click on Zillow and it takes you to that house. OK, we're about to find out. I just said, find me a nice home in the Hamptons for less than 10 million dollars that is big and has more than an acre. One thing that I will say is you actually have to attach your account. So I've already connected my Zillow and my Canva. But when you first do a search with one of these things, it will have a pop up that shows up and it like has you like connect your account. Basically, I think especially with Canva, like because like I have Canva premium. So it needs to know like what plugins and tools it can use and stuff. OK, so for Zillow, it pulls up a map basically of the Hamptons. It actually did it this time. And it just set like my it set like my, you know, my pricing things. We're now zoomed in on the Hamptons. It looks like, just to sort of describe it, it looks like what you guys have seen before, which is like all the red dots with little prices. If you can imagine, Zill, it's an overhead map. And then you can kind of click on all these things. Yeah. And it's making sure that none of the homes are over $10 million. So it's just, you know, perfectly in my budget, just what I'm looking for. And these are, yeah, amazing homes. So I think at the end of the day, yeah. Well, let me just describe this because this is really cool. So now what's on Jaden's screen inside ChatGBT is on the left. It's sort of like a list of the homes with little things. And then over on the right, Jaden was sort of like scrolling through photos of the house. The layout actually on this is unbelievably good, Jaden. That's what I'm seeing. What's interesting is they have like a list of current filters, but it's not like on Zillow's website where you have like all these drop down filters. They're like bubbles. So it's like the specifications that I put in that prompt are into these filters. Now, I wonder, I'll just try one more thing before we abandon this altogether because I'm pretty sure it can't do it. But I wonder if I just said like only or I should do all New York, but only low crime areas. That's interesting. Yeah, because now this is the question that I have on the street, which is this is still chat. So what it's doing now, Jane, why don't you describe what it's doing now? Yeah, it said that's a great filter to add. Low crime area is a bit subjective and crime data is often available only at precinct county town levels but there are some thoughts on a refined search path Anyways then it gives me a big thing So basically it like if it not a filter that built into Zillow my point is it can do it So like my question then would become what if it just filters on Zillow, what would stop me from just going to Zillow and doing this? Maybe people just want to like blab a paragraph. But the problem is like, if you have anything in that paragraph that isn't a filter on like a filter inside of the app, and doesn't let you use it, It just seems like you probably get frustrated and just go use the app. Do you know what I mean? I do. Yeah. And by the way, this is what I love. And apologies sort of, I know this is an audio medium, but just to sort of give you a sense, as Jayden has done this in real time, it's been so interesting for me to watch this because it seems like some apps are better than others, right? So like, I don't see a great use for Zillow on this, but Canva, as somebody who uses Canva actually quite a lot, and usually it's my colleague Robert Haslam that's doing this all the time. Like he's this brilliant designer and he does all that kind of stuff. So he doesn't necessarily need this. But if I'm trying to say like, yeah, the vibe I'm trying to go for, that's exactly what I'd be kind of telling Robert. But Robert actually knows what he's talking about. But if I was just on my own, right, this is what I could imagine putting into ChatGPT, ChatGPT giving me a bunch of images and then being able to do that. Now, maybe, by the way, as I'm sure people are shouting into their car radios right now where they're listening to us on FM radio. Car radio, the number one place. Is this idea that like you can already do that in Canva, idiot. it like and that's probably true and so part of this jaden i think the big battle here is just do people know that you can do this or not and you know and will people do it and i think sort of like it really requires a shift in how you think and how you behave and all that kind of stuff in order to move from one system that you've been accustomed to to another and this feels like a huge leap forward people are just getting used to asking chat bt for uh you know instructions and help and all that kind of stuff to ask them now to like literally reuse apps i think this is more of a vision for where they're going rather than something that's particularly useful now but i know we're just about a time jaden but any any final thoughts on that the final thought for it is for all the limitation it has today you can see the the benefit that it will have right and so like while it says i can't get you the crime statistics i just i just um basically i just prompt engineered a way for it to do it which i it was telling me like it couldn't find anything i was like look at the fbi crime statistics like it's public and they're like yeah but like i don't know, it's all subjective. And I was like, I told it, you do the research and list the safest cities, then ask Zillow about those cities. So I can't give Zillow like this big fancy prompt inside of there. But I can tell ChatGPT to go do the research first and then go ask Zillow. And which is like what it should have figured out to do, but it didn't know to do that. It'll know how to do that eventually. But it did it. And it actually did pull up. It pulled up Zillow. It said, here are New York's safest cities per the 2025 FBI based analysis. Bedford, Carmel, Rye, Onoda, Hyde Park, East fiscal, whatever. It lists a bunch of cities. Those are apparently the, you know, ranked by SafeWise from FBI crime data as the safest. And now it's only searching those cities. So like, if you know how to ask it, it can do it. And eventually, even for people that don't know what they're doing or don't know how to ask it, it will be able to do it. So sometimes you got to work with it a bit, but it looks like it's actually able to do that. And this is not anything you could remotely get with Zillow. So my tune has changed as of five seconds ago. This is insanely useful. Insanely useful if you are using it. Here's the thing, guys, everybody out there will sign off with this. Just remember to experiment on this stuff. And when you're thinking about experimenting, do not forget to check out Jaden's AI Box. It's my favorite tool on the whole Internet. You get to sort of compare models. It's $19 a month. You get to build your own agents. Again, you're not going to figure this out unless you just sort of like dive in. So I really, really encourage you to check it out. And we will see you next time.