SED News: Jev、MetaのMuse、そしてAI減速論のパラドックス

Hello and welcome to SED News. This is the monthly installment of SED Daily with a slight twist, which is that we go through the main tech headlines, we then dive into a deeper topic in the middle, and then we take a spin around our favorite hacker news. Things of the month in Hacker News highlights. As usual, it's myself, Gregor Vand, and with me is Sean Falconer. Hey, Gregor. Hey, everyone out there. Yeah, so I'm going to be seeing you soon, Sean, I think. I'm coming in. I'm actually heading off on a flight after this to SF, going to our Superbase conference on Friday, and I think you're going to come along for some of that. Yes, I will be there, at least for some of the day on Friday, and I guess you'll take off, and then, like, I don't know, 30 hours later, you'll be in San Francisco or something. Yeah, it's one of the funny ones where you leave on a Tuesday afternoon and arrive on a Tuesday evening,

even though you just spent close to 30 hours traveling. So, yeah, it's going to be a good week ahead. Obviously, that's sort of a lot of my week looking forward is, yeah, going to SF. We've got, given my role at Surface, I also have a lot of people to meet, all the main companies we talk to, people like Rissell and OpenAI. And then, yeah, it's going to be fun on the Friday. We've got Steve Wozniak as one of the headlines speakers, So that's a bit of a pinch me moment, get to be in a room with Steve Wozniak. But yeah, what about yourself, Sean? What have you been up to? I mean, luckily, no work travel for me for a little bit. I do have a personal trip doing a pilgrimage to the Great White North to visit my family here for Canadian Thanksgiving in a couple weeks. Primarily just heads down on kind of Q4, 2027 planning. We also have our annual conference coming up in November. So we're all kind of busy working on that to pull all those things together for, you know, launches and big announcements that we need to make. So, you know, people listening that are interested, you know, stay tuned for all that stuff.

Yeah, nice. But, yeah, otherwise on my side, yeah, the pollution in Singapore has been a bit crazy actually lately. It's basically, there's a tech angle to this in a second. Yeah, so it's basically a lot of haze from forest fires come from Indonesia. It passes over Singapore. but unfortunately it's been sticking around for quite a while but it did prompt me to buy an air purifier and i'm sure some people have them but i'm a bit late to the party maybe on that one but yeah super happy i bought a dyson air purifier i think it's just amazing the tech is like i don't know it looks cool the app is really nice yeah i don't know it's one of these like new tech moments i didn't expect to be quite so attached to an air purifier but i i really do love it my wife's always just like go just move around whichever whichever room you want that thing in that's it's yours so yeah i don't know do you have one in sf or like probably not needed right or i don't think so no i mean i probably would have came across it but it does feel like something that my wife would invest in yeah i think some people get them especially when they have like early infant

kids they sort of get a small one to stick in the in the kids room but no certainly first time for me. But yeah, I love, because of the app, you can look at your air quality indoors from, I'm in the office and I'm checking. When it gets a bit high, I just stick it on. So things that you can get excited about when there's pollution. Yes. I mean, we do get fire, you know, there's fires usually in Northern California that impact the air quality. So maybe this is something that we, you can send me your tips for air purification. Cool. Yeah, I should do that. I say cool. I mean, it sounds like Yeah, awesome. Awesome, yeah, fire is awesome. But yeah, at least there is tech out there to help with this. So yeah, I guess like diving into the headlines, we've seen a couple of kind of like large product drops, if you want to call that, in the last week or so. First one is Jev. So this is from a company, TypeSafe AI. The company was actually founded in 2024. but I mean I don't think many people really knew what they were up to until about September 15th

when basically they released JEV which is this they pitch it as effectively a model where you don't the outputs are not chance they are type-safed responses with a score against them as like how accurate it thinks that structured response is and the whole point is this thing is not meant to be really used by a human. It's meant to be sort of layer one. They call it like system one layer for like structured decisions like tool calls on routing and classification. But the big thing here is it's made such a huge splash, if you want to, I don't know what other word to use in the tech world. And it's kind of being like touted as like, is this the next revolution on LLMs? It was founded by ex-ChatGPT engineers, amongst others. So it's got some pretty sort of big heft behind it in terms of who's behind it. But yeah, there isn't, on the counter, there isn't like a lot that we can really dig into though in terms of open weights or anything like that.

Is that right, Sean? Yeah, I mean, there has been recently, there is OpenJEV, which is now an hugging face, which is a full-size 27 billion open weight alternative with near parity. Came out two weeks essentially since the launch. But that was something that, I think that was the distilled model to kind of mimic the dev experience is not official i don't think it's officially from the owners or the creators of dev but it is really interesting you know what they've done it makes a ton of sense because what's happened with large language models is that people use them for a lot of times stuff that they're not necessarily this most efficient model to do like classification even like choosing a tool and things like that like these are more structured response and you have to do a lot of work to kind of force them into this behavior of accepting structure and generating structure. And it's generally, like a lot of times it's not that reliable. And it's inefficient in terms of token costs, inputs and outputs and things like that. So to have a model that's using the same kind of transformer approach, technology and general

purpose, but specific for solving these kind of more structured things where you expect a structured response around classification solves is like a much better way of essentially solving that problem and trying to jam that into something that's accepting large language models because those were really designed to kind of function as a human might function in human modes of conversation. But those are very inefficient for a lot of these types of tasks, whereas Jav is really built for machine-to-machine communication where we're kind of trying to jam an LLM into, but it's just not a great use of tokens. It's expensive and it's also costly in terms of just the time to compute and the inference cost. So if you focus on that, you make a much more efficient model, both on token costs and then also the time essentially that it takes to process it. So it makes a ton of sense. I think that's why people are excited about it. It fit this gap in the market essentially between what you might do with traditional purpose-built models for things like classification or routing with the nice thing of the generality essentially that you get from the large language models in the transformer

or technology. Yeah, I mean, I think it's, you know, it's just one of these huge, like, step changes, which is kind of nice to see, where we've spent, I don't know how many months, talking about the next frontier model or open-weight model, but they're all kind of trying to achieve roughly the same thing, if you want to broad brush that, whereas Jev has come along knowing that there is, there is, like, another way to kind of approach this. I like the fact that, yeah, Jev does, the name does come from Jevon's paradox, so you know that idea that the more efficient something becomes that the more we actually consume it and then the efficiency is kind of lost completely i.e like light bulbs you know that we we have these like super efficient light bulbs now but we end up just leaving them on or have i have like 10 light bulbs or we used to have one but yeah it's like when was the last time you turned off your phone or your computer and like that right you just leave these things on perpetually basically exactly so yeah i like the naming of that so yeah we might actually have an episode coming up with the founder so that'll be exciting if in a couple weeks hopefully that's

going to happen so look out for that on our more regular programming yeah i think i think so far some of the pushback that you see is some of the benchmarks that they've published are they're kind of all self-published there's no paper or calibration and we know sort of independent studies of these yet. I don't know. I mean, of course you would expect that. It's like they just launched it. But I think what's really interesting too is how people have gotten creative with it. Like you see people using it to solve like games and things like that, where it is like you have a player in a game, you think of like Mario or something like that. It's like, well, Mario is a fixed set of things that he can do at any given moment. Let's use a model essentially to figure out like, what is the next thing to do? Do I jump? Do a duck? Do I shoot a fireball or whatever it might be? So people are using those, and that's basically a routing problem. And we have systems for solving those problems for years, but now you have this really dynamic system that can do that. You don't have to go train a net new model, essentially, every time that you want to apply it like that. Yeah, funny, you mentioned Mario.

I saw you can watch Jev playing Pokemon in real time. Yeah, there's a website where it's live playing Pokemon. Yeah, I did find a GitHub repo of someone doing this for Doom as well. Of course. Just to be on theme. I like the, yeah, I've seen Pokemon at least as like, okay, that at least looks creative and different. Yeah. But yeah, nice. I mean, I think the interesting thing is like, if you look at a lot of like compound AI systems today for doing things like model routing or other types of decisions, a lot of times they're pairing a traditional ML classification model with a large language model, or sometimes they're doing what I said, where they use the inexpensive large language model to force it into being like a classification model. Basically, JAV is like the perfect solution to that kind of problem solving. Yeah. So, yeah, I mean, definitely, well, I say give it a try. It is difficult to get on the, it has a wait list, I think, at the moment, or? I know a bunch of my colleagues have been playing with it, so I don't know how hard it is to

actually get off the wait list and try it out. I think they cleared a ton of people, too, if you look at the data. I forget the exact numbers, but they cleared tens of thousands of people off the wait list within a very short period of time. Nice. Okay. Yeah, I saw some of my colleagues talking about it, and it looked like some hadn't managed to get in, whereas one that got early access, as long as they had a credit card on it, then they could invite people to their org or something like that. But yeah, very interested to see how this shakes out, and I would love to play around with it. I don't have access yet, even though some are my colleagues do. So maybe I will, on the episode, if we record it, I will hopefully ask for some access as well. So yeah, moving on. Another big product drop was Meta's Muse. So quite a different product in the sense that it's not aimed at developers per se. It's much more in keeping with, I guess, AI products of late, which is, you know, a personal assistant. And we've kind of covered this area in many SED News episodes.

But it's the fact that Muse actually seems to be solving the problems fast and well, I think is the best way to put it. So they've launched this. It's an app. You can download it on Mac and on devices. But they are actually going to be also releasing hardware to go along with this called a Muse charm, which is a sort of like keychain device with a little screen think of like a sort of Tamagotchi looking thing but like I guess you'll just be able to talk to that as well and I suppose it's you know I reckon they're kind of cruising on the whole labubu I don know if you all heard of this but yeah labubu in Asia and stuff with these little charms that people put on their bags and it like ridiculously popular and I suspect they playing off because they look very similar to be frank so I thinking my kids have them yeah okay so the inspiration has probably come from something along those lines but the idea that you can show people that you effectively use Muse and you can kind of give it a name like even

even if you're just using it in the app itself I think you can give your Muse because I mean Muse the noun You can give your music a name and so on. And yeah, I mean, I guess I've been reading and hearing people talking about different examples. Again, I can't access it right now because in Singapore, it's on a wait list. So I look forward to flying to the U.S. in 30 hours and hopefully trying to get it onto a device. But what I've at least picked up is that it is able to solve problems such as, you know, booking a restaurant 10 times as fast as Claude, or 10 times as fast as it actually completes it, whereas something like a cloud gets pretty far and then it sort of says, oh, sorry, I couldn't actually book your restaurant for you for some reason. But, yeah, what's your take on this one, Sean? Yeah, I mean, I definitely, as we've covered before, like I think this AI assistant market is going to be like a huge, huge market eventually. And I think like OpenClaw was kind of the first real breakthrough where people realized that, oh, maybe there's a more comfort level with AI

taking actions on my behalf than we thought we potentially realized. And it kind of shocked on this. So I totally understand the investment. I think where Meta is going to potentially run into challenges, and maybe they'll overcome this, is just that because Meta is so strongly associated with Facebook and the history there in terms of using people's data for advertising and stuff, and then you have this device that potentially you're going to use on your behalf to manage part of your life? Like, how much am I willing to kind of trust my personal information with that company? And does it line up to ads and so forth? Supposedly, all this runs within a secure VM that's supposed to create a boundary between your Muse system and these other systems. But just kind of looking at the history of some of the moves that Meta has made, like, I can understand some level of distrust there from the consumer market as well. Yeah, it's interesting. Last month, we talked about company Instinct. and this was kind of their play as well.

And just as we were talking, I just thought I'd just go back to their site because I just thought, well, hang on, we were talking about Instinct last month and Muse is literally in this space and what are they doing at all to sort of recognize this? Interestingly, if anyone listened to last month, we talked about Instinct's website, which looked like it was from 1992 and had Times New Roman. It was trying to be all kind of back to the future kind of thing. Right, yeah. They've actually updated the site. Now it's not Times New Roman. It is a sans-serif font, more modern. It has a little logo now, a little stick man logo thing. And interestingly, at the bottom, it just says text instinct to get started. And I'm curious. I haven't used Wayback Machine or anything, but I'm curious if that was a bit of a... Before it was a very like, oh, you can't really get access unless you're special. So I wonder if that's being prompted. But anyway, I'm sidebarring here, but I just thought it was kind of interesting, given that there's this like just a kind of explosion in the personal assistant space. I mean, yeah, I think it's super interesting, like the fact that they actually even talk about the architecture here,

like the Muse secure VM architecture. I think that's something that people are quite concerned about with any on device, especially in this case where this thing is only valuable if you give it access to all the files or I say all the files, but files and especially data that you have, like your calendar, which would often be through some cloud service. And people, I think, are, you know, I think the sort of paradox here is people probably have given a lot of data to meta already, and it's not so easy to pull out the ecosystem altogether, or you just lean into it and say, well, you probably already have a lot of data on me, whether it's from having been on Facebook for a while, and then Instagram, and I don't know exactly what they would pull from WhatsApp, but at least WhatsApp is, I imagine, able to come into this in some way as well. So, yeah, it's a very smart play from Meta, I guess, especially given, again, through the last few months, you and I, Sean, we've not been able to really pinpoint where Meta was going exactly.

It also feels aligned with where Meta has been successful as a company. Like, this is much more of a B2C AI play than going after agentic coding harnesses or something like that, or even going after building models. I think this aligns with where they've been successful with Facebook and Instagram and these types of apps. So the personal assistant could be essentially the killer consumer app for AI beyond where we started with things like ChatGPT. Yeah. Have you used it or would you use it, do you think? I don't think I would. The hardware part of it doesn't really interest me. I don't really like the idea of, like even like metaglass and stuff like that. I don't want to advertise my use of things that could be perceived as like monitoring other people and things like that. I just feel like there's a certain uncomfortableness that comes with that. I don't want to, it makes me uncomfortable. I don't want to make people uncomfortable with it. That part of it doesn't interest me.

I mean, the idea of a personal assistant, AI personal assistant, certainly would be great. I just don't know if this is the one that I would personally put my faith in. I don't know if I'm ready to like give it my banking information. I agree. I'm curious to try it. I mean, I think for some of these more... I guess like hotel bookings. I mean, clearly it needs some way of paying for those things. So I'm sure there's some credit card transaction part of it. Yeah, I think that's going to be something... I mean, I'm aware, like obviously Stripe have been on this for a while, but I think they're finally getting the recognition that they want on like machines paying for things. they've got this machine as MPP machine, machine payment protocol. But I think people have maybe been struggling to kind of really utilize it because you need a machine that you trust to go and do the payment in the first place. So yeah, perhaps something like Muse is kind of the first step to that where it's at least an interface that people are going to use more. And yeah, I'm curious how much I would use it like day to day, but I am as a novelty, I'm definitely

curious just to give it a try, like book me a restaurant in SF or something. But yeah, let's come back next month. If I've managed to get access, we'll see if anything vaguely useful came out or trying to use. But yeah, very interesting. Your customer lives in the real world and it's messy. Intermittent connections, mid-onboarding drop-offs, edge cases on devices you've never tested. Mobile apps reflect reality in a way no other surface does. Yet, from an engineering perspective, they're the hardest to understand. It's common for mobile engineers to see green backend dashboards, normal error rates, no crashes, but inevitably somewhere there's a frustrated user watching your app spin. After a few seconds, they'll lose patience, close the app, and turn their attention somewhere else. They may never come back. The worst part? Most observability tools never see any of it. BitRift, on the other hand, captures 100% of mobile data, unsampled and in real time, so it's immediately queryable by engineers and AI agents. It's mobile observability built for the real world.

Try BitDrift today, bitdrift.io slash signup. Sponsored by DeepGram, the leader in voice AI, so you can use voices like mine. This episode of Software Engineering Daily is brought to you by VariaCode. If you're an engineering or IT leader, you know how painful this is. An open rec sits unfilled for three months. Your tech lead is burning out, covering the gaps, and your product roadmap gets pushed back again. VariaCode solves that. For over 10 years, they've been embedding senior Latin American engineers directly into U.S. engineering teams in under 10 days at up to 40% less than local hiring. We're not talking about generic contractor pools. These are pre-vetted, high-quality engineers with real on-call and SLA experience. They work directly in your Slack, JIRA, and daily stand-ups in your exact U.S. time zone with zero friction. If you're ready to stop losing sprints to open recs, head over to variacode.com slash SED right now and get a custom shortlist of senior candidates in 48 hours. That's V-A-R-I-A-C-O-D-E dot com slash S-E-D.

Finally on the headlines, so we did talk about, it's not the third time running, we're going to be talking about N-Scale, which is not any scale and it's not scale AI. it's nscale and nscale is basically a inference data center company but crucially they sit in geographic areas that are different for example like they have a center in the very picturesque looking part of norway for example but they're about to go public and so by going public they fell i think on s1 and we get to sort of see a bit more into the business also interestingly this is founded by an Australian person. Used to be a crypto mining company. Yeah, exactly. So it was founded by a guy, Josh Payne, who's actually an Australian. It's interesting, the lineage of this. He was a miner, like an actual miner, in Australian mines. And then, yeah, as you mentioned, Sean, the company, Nscale, that he then eventually founded was a crypto mining company, effectively.

But then, yeah, has pivoted into inference for running LLMs and so on for large companies. In the S1 for their IPO, this is where it's sort of interesting where at least the FT's reported that ByteDance, i.e. the owner of TikTok, was 73% of 2025 revenue. But interestingly, they just don't name ByteDance at all on the S1, which I think is quite interesting. So it feels like they're sort of like an arbitrage company here where they pick up companies that couldn't go and use inference data centers, say, sitting in the U.S. So they are, instead of picking up any of that traffic, where there's kind of trade bans at the moment. Yeah, I think also the fact that a fair amount of their revenue is coming from a small number of customers. I think we'll probably see this more and more with the AI companies that end up publishing an S1. And part of it is just because I think if you do close an open AI or an

Anthropoc or something like that, the volume of their use is just so high that it can easily skews your overall usage numbers and the amount of revenue they're growing at. So I think we'll see that more and more, which of course is a potential risk for the business depending on the longevity of those companies and so forth. But yeah, I mean, it's an interesting journey, miner to crypto miner to Neocloud. Yeah, exactly. Yeah, Neocloud, of course, that's the term here. yeah just a very smart business guy i think it's interesting that the uk has tried to sort of claim it as one of its own like it's a uk company well it is but it is an australian founder and the problem now is that they're actually going to look to ipo in the us on the nyse which is a pretty well-trodden path now but companies that maybe start in the uk as tech don't end up listing in the UK because they think all of the liquidity and better valuations and so on all happened in the US, which is pretty much true. And I don't know what it will take to break that cycle,

but it's unfortunate that we've got another one of those here. Interestingly, he is almost the, if they were to list in the UK, he would have actually been effectively one of the top paid CEOs, like almost the same as AstraZeneca, AstraZeneca CEO, which maybe does also suggest Maybe the numbers are a bit crazy here at the moment on like valuation, pay. It just kind of just like a yeah very opportunistic punt on pivoting this company from crypto mining So yeah curious to see how this shakes out or whether this is yeah just trying to sort of draw in people that just like the term NeoCloud and think that that's where they should be investing. Yeah, I mean, that's a really, like, I guess, like, hot area in term that's kind of come up over the last year is the NeoCloud. I think, like, NScale, are they the first to kind of file an S1? At least the first one I can think of. yeah yeah that's true so yeah it could be like an interesting canary in the coal mine huh yeah

exactly yeah but especially you know being the first perhaps again there's like a sense of is there a quote crash coming should we get this kind of one and done before any any other problems so yeah we'll follow along we'll see how that one shakes out moving on to the main topic for this week. So we're kind of, I guess, subtitling this AI risk frontier lab slowdowns and conspiracy theories. So it's really that there's been a series of events, especially over the last few months that don't quite kind of add up maybe in some people's minds as to are the large frontier model creators doing what they say they're doing or more like when they are talking about things like, oh, we should be slowing down, we should be being more careful, but then at the same time shipping new models and saying that these are the most catastrophically smart but dangerous models we've ever produced,

like, is that not some kind of paradox that's kind of going on? So, yeah, so, I mean, like, where do we kind of start with this kind of story, Sean? Yeah, I mean, we've covered parts of this before, like, even going back to mythos, we kind of touched on the sort of byline from Anthropic was like, oh, we have this like super powerful model. It's too dangerous to give to everybody. So we're only going to give it out to like a few people. And we had talked months ago when this came up of like, well, there's definitely like a marketing angle to that. It's like the art of scarcity. It's like you talk about the thing that is so amazing and incredible, but no, you can't have it. And then eventually it is available. And then of course everybody wants it, But kind of tracing back what's happened over the last little while, there was the hugging face intrusion that happened back in July where, you know, roughly 700 open AI agents that were running some sort of internal cyber benchmark with the hacking guardrail switched off, compromised, hugging face production infrastructure. And we talked about that. And then there's been employees.

We've also seen this even going back. There's employees in July that had said that sort of whistleblowers of like, oh, these models are dangerous. It's a risk to human life even. And then there was a bunch of employees from the Frontier Labs back in July that were urging the government to deliver a pace automated AI development. And I think the big thing was on September 12th, Dario, the CEO and one of the founders of Anthropic, came out and published his essay, Pace the Frontier, which I recommend people check out. And then Sam Altman, Elon Musk, Demis from Google, all essentially publicly agreed in the same day, I think, on X, that they agreed with Dario and that we should pace it. And, of course, immediately there was, like, conspiracy theories around this. But then on top of that, like, 10 days later, Anthropics ships Opus 5.5, OpenAI ships GPT-6 essentially the same day.

So there's this, I don't know, like, somewhat, I think you referred to it as a paradox, but, like, they're saying slow down, but at the same time they're releasing models really, really fast. So, you know, which is, in a sense, kind of a strange signal to people. But I say to, like, Dario's credit is that he's been consistent in, I think, always kind of warning about the risk of AI. So it's not like he just, you know, in the last month suddenly started talking about this. He has been talking about it for a long time. But some of the things around the conspiracy theories and why people are concerned about how genuine this is, is that one is if you take all the top models in the space and all these companies kind of, collude to say like, hey, we should slow down the pace, we should add regulation here, then in some ways it's a way to protect their position in the market because it's much harder for a smaller company, a newer player, to potentially catch up because they already have a head start and they have a huge amount of financing and the people to navigate whatever

the regulations say in terms of what you need to do as a model provider. So that's one thing. And then one of the other conspiracies is that it's around the related to the IPO and margin protection because the open weight models coming out of China are really threatening how much people are willing to pay for tokens. And all the frontier labs see that the writing on the wall that there are companies are going to be devalued. And this is a way for them to explain why they're not doing certain things or slow down the launch of the IPO or even protect against later of like, why isn't the latest greatest model out? It's like, well, you know, like we have to adhere for the regulations or we're not sure of the safety so they can kind of control the narrative around performance is another one of the conspiracies. But I think the other big part of this, too, has been President Trump's reaction as well as Jensen Wong from NVIDIA. they've kind of come out on the opposite side where Trump said that, you know, it's all a hoax and Jensen spoke out against like that they shouldn't, you know, pace AI

and it's unrealistic and that these threats aren't real. And of course, NVIDIA also has a strong incentive to say those things because the faster AI moves, the more GPUs you need to buy and you're probably going to be buying those from NVIDIA and it helps their company. So there's all this going on. It's a lot to try to parse and dig into, but I don't know, Gregor, what are your thoughts on this? Yeah, well, I think one thing that stuck out is I think a lot of people, news outlets or otherwise, kind of have interpreted it as, oh, it was this call to pause. And I don't think that, you know, that wasn't really the word, the essay that you referenced did actually use the word pace. I know we're sort of splitting hairs potentially on words here, but they are quite different. And pacing is saying, hey, let's, as it sounds, let's be clear about how fast we should be developing models. Because if we kind of all just let loose, then who knows what could happen? That's quite different to think pause, which is, as it would sound, let's just stop producing new models until we've figured out tighter guardrails or whatever it was that they would have done if they'd said pause.

so pacing i think that's the problem though with pacing because pacing can then sounds to me like from a business angle is like could almost sound like colluding and colluding is kind of when you have like you know a cartel a cartel is when you have multiple companies who all basically behind the scenes agree with each other to do to sort of what i would call sandbagging i you know you you know you can do more you know you could jack a price up you know you could produce something more powerful but it's all in your collective interest not to do that because you're kind of trying to eke out the most amount of performance and especially financial performance. You've kind of modeled it over like three years and gone, well, hang on, if we jump to our best possible model like tomorrow, when I say best possible, what we know we could achieve ultimately, if we jump to that tomorrow, well, where's all the step changes we can keep charging people for over the next two years? And I think that's kind of, you know, some people saying, oh, it's a cartel with its safety label.

I got to say, it's hard to ignore that at times because we've got all these IPOs. I say all these IPOs. We've got likely OpenAI and Anthropic both wanting to IPO in the next, I mean, they keep saying they're kind of gearing up for it. So like, I don't know, in the next six months, one of them will do it. So they're both on the same path and they both have like the same incentives to kind of, I would say, to collude on some ground on this one. And again, trying to eke out as much financial performance as they can. That's definitely where it sits with me at the moment. What do you think in terms of the risk, though? Like some of the risks that people are calling out, either the hacking risk, and then there's, of course, people saying that some people within the labs or the whistleblowers are saying that there's like a 3%, I've heard there's 3% and up to 10% chance of human extinction by 2030. How do you react to those things? Yeah, that's a good question. Even my mother asked me the same question. Yeah, my wife has been asking. When I spoke to my parents a couple weekends ago,

they don't often ask me about work or AI stuff, but they'd seen these doom headlines, and they even said, anthropic, somebody from anthropic. So I slightly think it is hyped up. The problem is I can't put my finger on why. Why would someone say all this stuff and then leave the company? I think what I'm getting at is it is absolutely possible. Like these Doom scenarios, of course they are possible. I think it's just where are we seeing the real rubber hits the road on the Doom stuff? It's like, okay, we've seen the hugging face hack was pretty sort of eye-opening because it kind of went on for a while. But at the same time, I keep saying it, you can put guardrails on this stuff. Like the bit I don't understand, I'm sure there's some shouting at their earphones right now, that works in one of the frontier labs, but you can put guardrails on this stuff. And I'm curious why it's not more of a, we can still develop models,

but without them actually needing to go off and do this kind of hacking and so on. Yeah, I don't know. That's like the layman's take, given I don't work in the frontier labs themselves, I'm still, I feel like a layman when then thinking about these concepts. Yeah, I mean, if you look, though, the history of technology, like all technologies kind of have white and dark aspects. Like any new technology can be used for greatness or good things and can also be abused either accidentally or on purpose. Like, you know, if you look at things like electricity, in the 1890s, electricity caused tons of fires and the government was kind of too slow to react to it. So what it ended up doing was the insurers actually ended up funding their own kind of private testing to deal with that. Drug design is another example where drugs do amazing things. They also do potentially bad things. But there's checks and balances to medical drug design. It's slow process, slow process

by design. Even stuff like the printing press, it was the brought mass literacy to the world, but it was also used for mass printing propaganda. And, you know, social media is a good example of that where it's like there's been good things, I'm sure, that have come into social media, but it's also a propaganda machine that people would use. So there's all these things. And I think AI is similar. I think that the challenge, of course, is that, is it moving so fast that the view scenarios sort of take over before you are able to recognize, like, the truly transformative things that you can get out of AI. And truly transformative, I see it as, like, being able to use models that solve diseases that we haven't been able to solve or really crack into things that have been too complicated for humans to address previously. We're probably quite a ways away from that. But the danger, of course, is that we move so fast to try to get there that we end up hurting ourselves in the process. So I think that's like the balance that you're trying to carve out right now. And I think the truth in all the conspiracy theories versus what the

intention is behind Anthropik and the others that are involved is I'm sure it's somewhere in the middle you know it's probably not extreme like this is all like faithfully like they realize they just doing this for the better of everybody involved but then I also think that extreme conspiracy where four or five companies got together like we going to go and trick the world and manipulate the public market it's probably a little extreme as well. So the truth is somewhere in the middle, I guess. Yeah, I think that those are really good examples. I mean, yeah, like we've seen this through history. I think, yeah, I mean, you touched on it, like encryption, you know, being like classed as a munition before it was like seen as like a you know a technology that was actually a force for good quite frankly again in most cases of course encryption can be used for nefarious purposes but hiding information that would have otherwise perhaps incriminated someone that probably should have gone to jail for example but you know we like to think that actually

encrypting the majority of sensitive information in the world has only been a good thing even though the ability to crack it is getting almost easier and easier except for quantum the printing press i think is the interesting one because again i think that's it's closer to what we're talking about here because it plays on fear like people feared that oh people that can't read right now let's keep them down because a printing press will mean that more and more people have access to this crazy thing called words on a page and they will potentially be able to learn how to read and then read information that we don't want them to know about. Yeah. And that really feels kind of at the moment we maybe have people in the frontier labs. That's why I kind of like that I don't sit inside one because I can kind of postulate like, well, here I am not sitting inside one of these labs. What do I think? And, yeah, I do think perhaps that what they see is sort of, oh, the normal people outside the labs can't possibly have access to this. That would be a terrible thing. I'm not sure I agree that just companies founded, you know,

not that long ago in relative terms should necessarily be able to decide where this technology now goes. I mean, they are now one of hundreds and hundreds of companies that can produce a model. So, and that's just, that is a technology that is now available. So on that point, I mean, it seems that Google, OpenAI, Anathropik, they're wanting to set up something called the Standards Authority for Frontier AI, which is modeled explicitly on FINRA, which is the body that polices Wall Street brokers, but on behalf of the industry itself. It's not a government agency. Do you think that's like a step in the right direction? I mean, I guess at least in some sense, they're trying to put more than just words behind this. Because Dario's essay essentially lays out three different strategies for pacing AI. One is embedding some sort of third party within the frontier labs that can actually evaluate how safe the models are

before they launch the models. Then there's like a government level regulations involved. And then there's kind of like a, like ideally like a worldwide. It's similar to like how we have controls around the development of like nuclear weapons and things like that, right? So they're trying to do that. But of course, it's one thing to write it down. I guess it's another thing to actually make it happen. To try to get everybody to cooperate on it, I think, is very challenging. You already have the president of the United States having a completely opposite end-of-spectrum reaction to it. And then the other challenge is just even if the U.S. can decide that the labs here are going to slow down, doesn't mean that the labs outside of the U.S. do as well, and then that potentially becomes a threat to the technology and how the technology is used and are the people in the US okay with it. It's overall like a very complicated issue. I remember even hearing an interview with Dario last year where he talked about how obviously his views maybe changed,

but he said that he didn't feel like they could afford to slow down or pace things at that time because they had a head start on, say, China and other areas of the world, but it wasn't that big a head start. So if they were, you know, multiple years ahead, then they could perhaps slow down to make sure things were safer overall. But because there's this external threat, then they can't slow down the pace essentially. So I guess things have changed somewhat. Maybe they've left ahead far enough, or maybe the threat is big enough or real enough. But all of this has also been sort of self-reported when you look at the stuff that the labs have come out with in terms of where there's been a breach or someone has declared some sort of issue. If that happens in things like airlines, like there's some sort of incident, there's third parties today that go and investigate what actually happened. We just don't have any of those kind of checks and balances. So it feels like something is necessary. So I guess overall I would say it's kind of at least a start to something

that probably needs to be done. I don't know enough about how some of these other things work in other industries to say like is this like the right approach or not but it feels like it definitely feels like something needs to be done because you look at almost any other area where safety like public safety is a potential issue like whether that's making a vehicle or flying a plane designing a drug like there's there is multiple checks and balances before you're able to do any of these things like I can't just go create a car and sell it on my own and like people will come after me. There's things that prevent you from just doing that. Yeah. Yeah. That's, that's a good analogy. It's always like, where's the tipping point between like something that, you know, until, yeah, I'm still, well, in many areas you can just create software and sell it and no one's like, Oh, but what if that software does X or Y, unless it is something that clearly crosses a legal boundary, it could be still be doing something quite terrible as software or like cause damage but it's kind of encapsulated in like well if a company causes

damage that's like a liability and that's like a civil thing whereas i mean a different example i guess then is you know when crypto all took off and like basically financial regulation had to come in because they thought that could cause damage to the average person from like a financial standpoint like if you're being misled as to what you are doing and yeah so now but now in this case It's like, should there be regulation on top of just developing a model because of all the ways a model can be used? Yeah, I think that's a, yeah, the car, the airplane example is interesting where you can't just, you can't just, I mean, you can design it, you can build it, but you can't sell it to anyone. Yeah, I can't just start offering commercial flights. Exactly, yeah. So, well, I think that's a really good place to leave it, actually. hopefully that's been kind of helpful a bit more i don't say theoretical but yeah obviously just like looking through why has any of this even come about and like where are things going in the last

i guess few few weeks few months in terms of where the frontier lives especially seem to be kind of positioning themselves on this so yeah i do want to say like you know i'm firmly on the side that we've already opened the i don't know the pandora's box like there's no sort of you know putting it back in at this point that the innovations are going to come. It's like this tidal wave is coming. And, like, you can either try to swim the shore and escape the tidal wave, get the high ground, or you can, you know, try to grab a surfboard and surf that tidal wave. And I'm much more on the surfboard side of, like, let's figure this out. Like, there's, I think, a tremendous amount of potential upside, like the light side of the technology, that there's too much upside to ignore. but let's figure out a way to also make sure that the dark side doesn't end up taking over the upside that we can look forward to yeah absolutely well talking about upsides how can use highlights yes time to move on to yeah usually our sort of favorite part of the show

do you want to kick us off sean what did you come across as one of your favorites in the last week yeah so there was this one that i really like the title of it it's called pirating the pirates which was posted by piotr grabowski i think so essentially there's this issue where film preservation has this like sort of piracy problem hidden inside of it where studios end up releasing a film but then they manipulate it in some way so i mean the classic example is like with star wars where George Lucas keeps re-editing the existing footage. So it's really, really difficult to actually find the original Star Wars footage because it's been re-edited by George Lucas over and over again, or something like MGM put 18 minutes back into the good, the bad, and the ugly with dubbed dialogue and sound alike replacing a dead actor's voice. And a lot of people who enjoyed the originals are frustrated, essentially, by this. So there's this effort where these boutique kind of restoration sort of combination of amateurs and somewhat professionals take stuff like the DVDs, the Laserdisc, the VHS, and various international cuts, and they kind of like try to reverse engineer back to a correct original cut, which is really, really amazing.

It's just like this labor of love to try to restore the original version. But it takes all this work and this article is about someone's firsthand experience doing this, working on something for several years with the good, the bad, and the ugly. Yeah, that's really awesome. Just having a quick look at it. And yeah, lots of screenshots kind of showing four different versions effectively of that film, depending on the medium and when it was released and who was behind that release. and the colors just are all over the place, like four completely different color palettes, if you want to call it that. Super interesting. Yeah, I'm going to dig into that more. I've got lots of flying time, so I think that'll be one that's on my list. On my end, a little bit more technical, I guess, but it could be interesting. I mean, the name just got me. It's called FakeCloud. So fakecloud.dev. And this was posted by, and this might lead into some interesting commentary here. It was posted by an user called The Anonymous One, and what was interesting was so FitCloud is meant to be an AWS

sort of testing harness effectively and they, I say they because again it's quite anonymous who's behind this they say FitCloud gives you a local AWS environment that behaves like infrastructure not a mock so your app can use the regular AWS SDK, CLI and IAC tools and it then says unlike today's local stack community setup if you do not need an account, auth token, or paid plan just to keep core dev flows local. Yeah, super interesting. Obviously, a lot of people that develop against AWS and can be pretty costly to just be developing against it. Because AWS have no concept of a free testing environment, really. We just kind of test against their APIs. But interestingly, in the comments, people were a little bit concerned that maybe, first of all, who's behind this? Second of all, I love this. It just looks vibe-coded. I'm not really sure if that's really a reason to distrust something these days because so many things are, and frontend isn't often.

There's a strong suit of some developers, so that's fine. But they have also suggested ministack.org as an alternative. So, yeah, go check those out, fake cloud or ministack. I thought that was pretty cool. We are bumping at time, so, yeah, we'll need to maybe save our predictions for next time. But yeah, looking forward to catching you in person again, Sean, and I'm sure we'll have a lot to digest on the next SED news when all things are done with conferences and so on. Yeah, absolutely. And I'll see you later this week. And thanks, everyone, for listening. Yeah, thanks, everyone. See you. you