Blitzyとエンタープライズ向け自律型ソフトウェア開発

Code bases and enterprise organizations often span hundreds of thousands or even millions of lines of code. In many cases, changes must follow established patterns precisely, or risk introducing costly security or operational issues. A common challenge with AI coding tools is that they struggle at this scale. LLM context windows cover only a fraction of a large code base, and output quality degrades the longer an agent runs. Blitzy is an autonomous software development platform for the enterprise. It starts by understanding your entire code base, building a knowledge graph that acts as a shared source of truth. From a well-defined spec that engineers review and approve, it orchestrates specialized agents that work in parallel to build, review, and test code autonomously at scale. The result is a pull request that engineers can then review and merge. Sid Pardeshi is the CTO and co-founder of Blitzy, and he previously worked at NVIDIA.

In this episode, he joins Kevin Ball to discuss why large code bases break today's coding agents, how a knowledge graph extends what models can reason about, how a reviewable plan keeps specialized agents aligned, and where autonomous software development is headed. Kevin Ball, or Kate Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc. Sid, welcome to the show. Hey, Kibol. Thank you for having me. Just to quickly introduce myself, I'm Sid Pradesh, CTO, co-founder of Plitzy.

I've been at NVIDIA since 2016, you know, back when NVIDIA's adjusted market price was a few cents. And it was not the trillion dollar company that it is. It was nowhere near that. And when I told people I work at NVIDIA, they said NV Media. You didn't get a job at Google, huh? So NVIDIA operated so much like a startup and it was a fun journey. I lived through, I saw the AI wave happen and I lived through it and I saw how Jensen was a visionary. During that wave, I realized what an incredible opportunity this is for applying AI to the world of software development, particularly autonomous software development. And that's how Blitzy was born and that's why I'm here. Well, let's give us a little bit of an overview of Blitzy. What do you all do? And what's the, don't give me all the secret sauce, but like what's the unique take? Yeah, so, you know, Blitzy is an autonomous software development platform. So people are used to platforms like CloudCode, Codex, which are, you know, more tools or even tools like Devon or Factory.

The way we're different is that we specialize on very large enterprise jobs. Unlike the other platforms, we are enterprise only. and we have enterprises designing their entire SDLC around us. So what happens with the other tools is that they're really great at taking bite-sized chunks of work that humans can delegate to, right? They give them tasks. Many of them position themselves as junior engineers or interns, like Devin, for example, right? CloudCode, for example, is a terminal tool that you can run micro projects with. but the vast majority of the work at the enterprise about 80 percent of it is well-defined work especially with legacy enterprises which very large-scale enterprises it's very well-defined work that was reviewed by a designer that was reviewed by an architect was documented it's ready to pick up you know exactly what the acceptance criteria is you know exactly what the process is to get from development all the way into production right and it's just sitting there

They're waiting to be done. The humans that need to work on it, they're juggling across models. It's so unpredictable. Office 5.5 just released yesterday, and it solved some of the problems that Office 5 had. But Office 5, a lot of people struggled to work with it. It changed so many things about the model. GPD, on the other hand, has a different product. So what you have is engineers find themselves optimizing their prompts, building their skills, hopping between tools, trying out the different models that come out, finding new ways to work with them, finding fit between the features and the prompts and the tools that work with, etc. So the other tools, they've changed what engineers are doing, but they haven't changed the fact that engineers still have to do things and sit there and babysit the tools and the agents to get work done. Where we're fundamentally different is you take that spec you have, you give it to Blitzy, you press build, and you're done. You go and do other things. You go and use the other tools, you go and do other more important things. And Plitzy does all of the work for you correctly, exactly as a person who's been on your team for like 30 years has been doing.

It understands exactly the right way you implement things because it does this batch learning across your entire code base when you're on board, understanding and mapping out all of the patterns, relationships, you know, identities across the entire code base, and then applying those at runtime using the right model for the right time. So we use Office, we use GPT, you know, we use a large variety of models and model sizes. And we always deploy the right model for the right job. It's all about, for us, we're not optimizing for cost. We're optimizing for quality, right? What we want to do is when you get the PR back, it is correct. It is as close to correct as possible. And what our customers have seen is that majority of the times we are at least, we've saved 80% of the time that you needed on the project to get to production. and the remaining 20% is often because we didn't have some access, you're not comfortable giving us that level of access, we didn't support some integration, or there's just other processes or policies that your team follows before you get something to production.

So we map against production, we're enterprise only, and we focus on doing all of the work one shot. That's how we're different. Awesome. I want to dig into a few pieces of that, but first let me say I love the focus on quality. This is the thing I constantly challenge my team on in my day job of like, hey, a lot of folks are using these tools to pump out a lot of things faster but at a lower level of quality. How do we do the same amount of work at a higher level of quality? What do we do to up the level with it? Let's maybe start a little bit with what the enterprise only ends up meaning, what you get to do differently because of that, right? So if you look at an enterprise customer, what differentiates them from selling to a developer or a small company? Yeah, very good question. So I would say there's two fundamental things, and we'll talk through more as we identify them. One is that the challenges and the kinds of code bases an enterprise is looking at are very fundamentally different than someone who's working on a hobby project or wide coding.

So one is the size. Two is the expectations and the standards. right like if i don't follow a style guide on a hobby project that's fine as long as it works i'm not going to complain on an everyday basis and the enterprise if i make a mistake in the style pattern or if i make a reckless change that ignores a pattern that's well established and required or i make a compliance error or i introduce a security vulnerability that you know doesn't take into account all of the constraints you have or even a load you know a load balancing defect, right, or a performance concern, that item, if I introduce that by, you know, shortcutting and not following standards, that means material cost in dollars as the code progresses through steps, right? Someone has to spend more time reviewing it. Someone has to spend more time testing it. And then ultimately, if it gets to production and the customer is impacted, the business has lost dollars, right? So the stakes are really, really high.

the complexity is high, right? You're not looking at a 10,000 line code base or 20,000 line code base. You're looking at at least hundreds of thousands of lines for a small enterprise and tens of millions of lines for the largest enterprises just for a single product, right? They have multiple code bases at that scale that they are actively maintaining with, you know, hundreds of committers and thousands of committers, right? So the scale, the complexity, the standards, they're all really high. The sizes of the projects and the features you're building are also very high, but it's a very high tax. Like, for example, if I have to make a 20 line change in a code base, right, I may think that I could get away with, let's say, a simple agent that maybe it's thinking to medium and get the job done. But the requirements of that enterprise is that we've already added the libraries or the logic that does this.

And that logic or that pattern is buried deep into millions of lines of code. And we don't want you reinventing the wheel for that. So finding that just at the right moment and then doing it exactly the right way, that's the hard part. That's the big challenge that we're pushing ourselves to solve day and night and we've solved to the most extent. That problem is hard for humans, right, to map out millions of lines of code and understand where it is. So let's talk then about the start of the process, right, because you mentioned a little bit, hey, we go and we ingest a lot of what's going on so that you know, oh, what is that one library off in the weeds that should be used for this? So what does that look like at Blitzy? Yeah, so if you look at how far models have come, right, models are really intelligent these days. But whether they fall down, they fall down at the context level. They have 1 million tokens of context, which maps to about 100,000 lines of code. But here's the kicker, right?

Every token in the context window needs to map to every other token by a dot product in the matrix. And there's no amount of compute or trick that you can apply that does this effectively across the entire context window. So most of the model providers that you have, the Amantropic OpenAI, they have the effective context up to less than 300,000 tokens, right? And they're applying techniques like sparse attention, sliding window attention, or something else across the rest of the window. So this is also why, for example, now this is not the official reason, but OpenAI has a pricing differential at 272,000 tokens. When you send more than that, it's a long context request. You get charged more, right? But because of that fundamental gap, as soon as you send more than 30,000 tokens, more than 300,000 tokens in the request, which maps to 30,000 lines, intelligence degrades. So that's why as you run these single agent tools, like, for example, the CLI tools or the single agent persona tools, quality degrades as they are more long running.

Right. The encouragement from all the providers is to do short running jobs. Now, the way we solved this is that we invented a graph database system that maps your entire code base by identifying its relationships and understanding what every file in the code base is doing and understanding the structure. Like, OK, this code base has 20,000 files, but here are the modules that are structured. Right. Here's what this file is doing in this module. Here are the features that live in different parts of the code base. If I'm looking for authentication, these are the functions and these are the patterns that the enterprise is using across the entire code base. And we do that pre-run in about 24 to 48 hours when you start working with us. So it's not like you turn on the terminal and you start working, just like Cloud Code. There is a cost and there is a pre-run. You have to go through that pre-run. But the benefit is we've now mapped out the entire code base and represented it structurally.

So this graph database lets us so for example let say you want to find a pattern in a code base that 30 million lines right And there tons of products There so much noise So Clot Code Curse any other tool they going to try to use grep or some variant of grep and try and find it. But there's so much noise. So, the agent is going to burn tokens and hopefully find the right one. But what we've seen in many of these code bases is sometimes there's deprecated stuff in there that's not even marked deprecated, right? So, how do you know that you found the right item without actually profiling the code, without actually running the code, without having reverse engineered the entire code base. So that's exactly what Pitsy does first. But reverse engineering it and saving those markers in the graph database is contextualized, right? Like even memory, for example, right? Memory for the other tools are files strewn across the code base. You don't see it because it's in Cloud's projects folder, but you can go and see it if you like. But then finding the right memory for the right module or file at the right moment,

that's the hardest part, right? So by having the graph database represent that structurally, semantically, and naturally, you have a massive advantage. So the magic of Blitzy is really just making it easier, drastically easier for these models to operate within their effective context window effectively. Yeah. No, I'd love to dig in one level deeper without, you know, asking for any secret sauce on this. But I think, you know, I've been experimenting with an open source graph database tool for my coding agents, and it does make a huge impact. So thinking about that, you sort of alluded to it's not just the structure. Like you could use a deterministic tool, compile this to an AST, map the dependencies and have that surface that up. But there's more that you're talking about here because you were talking about, oh, we're putting some weights on this. We're saying, oh, this one's actually, it's not marked deprecated, but it's not actually used. So like what are the different signals you're looking for and how do you expose those to the agent? So we're looking for, we're actually trying to build and run the code. Right. And we're actually mapping out what is the piece of code that's used based on what the agent can observe.

We're looking at not one file, but the entire code base. And right. And that like at that scale, when you have like 20,000 files identifying which file is relevant and which is not, it's hard. Right. So there are, for example, graph database solutions. There are even open source versions of those implementations. But it's really, really hard to solve for the entire code base. And that is what we've invented and solved, right? Inventing and solving and building a system that is reliable at scale and works for any language and is actually able to pull, you know, that context reliably and make it helpful for the agent and not actually distract it. Oh, absolutely. That's like what I'm wondering about. And, you know, as I said, without going into secret sauce, like what, like, what does that query approach look like? Like, are you exposing a full graph language? Is it a sort of semantic tool? Like, what does the agent actually interact with? So the agent has tools that let it, you know, search the graph database.

So, for example, we also employ semantic summarization. So, you know, we save embeddings of summaries and let agents search that. The agent is writing cipher queries that is traversing the graph. It's identifying, you know, what is the best way to query this. So the agent has an understanding of the graph database's schema, right? And it knows the kind of information that it can get and the kind of information it can save in case it finds an inconsistency, right? If something changes, what happens? How do you update the graph? That kind of stuff. So giving the agent access to the schema as well as access to the kinds of queries it can make and tools it can call, that's what makes all the difference. at inference time, right? Yeah, awesome. All right, so that's anything else interesting from the kind of ingestion process. How are you pre-processing this code to give your agent full knowledge that we should talk about? So I think memory, as I said,

is one of the most important items. What happens with memory is, in today's world, memory has to be earned. So you have to have used the product to be able to earn that memory, right? and then save it. And so you have to get hurt to know how not to get hurt that way. But magic happens when you're not able to get hurt in the first place because you learned from someone else's memories, right? So a few ways we approach this. One, when we look at the commit history and we try to understand what journey has this code been through in getting here, right? And how can we imbibe that and follow that process so that when we write something, it lands on the same planet that the customer expects, right? The others is the customers often use agents.md, skills.md, cloud, et cetera. Learn from those, apply those, but again, identify the context of when they are actually relevant, right? So saving that information, understanding the tech stack,

summarizing that, compounding that over multiple levels, that starts paying dividends as you start going, right? You can make fewer mistakes, right, with the customer's input because, you know, to be fair, humans and or users are not the best at giving it the most clear instructions. So to really deliver value, you need to be able to understand and interpolate instructions that were never provided. Yeah, that's super interesting. I've done some of that of like, oh, let's analyze my previous sessions. What do I get out of that? Things like that. But you're looking across essentially their entire history of this project and say, oh, where were the things that had to be reverted? What were the missteps? What was the journey? So during that process, can we talk a little bit about like what does the, are there like certain sets of signals you're looking for? What is the compiled version of this look like? Is it coming down to a set of guidelines or is this another graph of searchable practices? Or like what does this look like? We're embedding this into the actual graph.

So we try to replicate the structure of the entire core base. and the problem that you have is, as you correctly described, you can create multiple graphs, right? You can create a graph of the commits, commit history. You can create a graph of the file structure. You can create a graph of the folders, et cetera. But the problem is how do you tie all that information together and then how do you provide that to the agents in a digestible format where they can operate efficiently, quickly? What we're doing is we use a structure that maps to the structure of the code base and then we actually process all of those commits, you know, as much as we identify are relevant. And we apply the learnings from those commits to the nodes of the graph. So you don't have multiple graphs that you're trying to, you know, deal with. You have all the information contextualized. And another unique aspect, something that we perfected really early on where the rest of the industry has yet to catch up, it's not even being discussed, is running agent swarms without a coordinator.

So the existing model is you can run, you know, swarms with there's one coordinator and tens of agents, right? But there's not, in fact, research, current research has proven that if you run multiple agents in parallel, it's really hard to ground them and to get them to not overlap and do things, you know, solve the shared goal because the other agents can have multiple opinions. We run all of this processing in parallel and use multiple coordinators at different levels to achieve the shared code consistently and reliably. So there's a different agent, for example, that's focused on crawling the graph. There's another agent that's focused on integrating that just in the right place and handing off to the next agent at the right point. So that, I think, is one of the hardest problems that we had to solve early. Yeah, so let's dig into that a little bit, because as you highlight, you can have a single centralized orchestrator, but that breaks down fairly quickly. How do you deal with collisions, ownership, handoffs?

What does that look like in your agent swarm? So we use the graph database as the source of truth, because that changes everything. So what happens is, you can use, for example, disk, but disk is volatile. right as soon as I change the folder as soon as I kill the pod or destroy the location it's gone you can use git but you could destroy the beauty of the commit history and you can have a very cluttered workspace that makes it incredibly hard to find information it's one thing to dump information it's another completely different to retrieve it intelligently right so the graph database helps you do both by you know structuring it on the graph by contextualizing it at runtime and then structuring it on the graph, what you can now do is use that structure to recruit agents. So for example, if your graph is based on folders, you can assign agents to folders and limit scope. Once you have a scope document, you can break down the scope and then distribute it based on a structure that's anchored on the graph.

So what that does is it couples us to the graph, right? Whatever we do has to be on the graph, which is a trade-off. But the benefit is that you can run as many agents as you like. There is no theoretical limit to how many agents you can run in parallel as long as you solve the compute behind it. So let me dig into that a little bit more. So we already talked about when you do your analysis of history and things like that, you're attaching that to the graph. I'm assuming that's some sort of either metadata or a distinct type of node attached to the code node in some form. So now you're saying, okay, we're going to also attack tasks or desired transformations or something like that also to the graph, and now those are claimable, and agents can just come in and say, oh, I can do this part. I can navigate this. Does that require you to deal, like, what is the synchronization primitive? Oh, I have a dependency here. I need this part of the graph to be complete before that can go. Like, does that mean I have to claim this whole piece, or does that mean, you know, I do this and then someone else can claim that?

Like, what does that end up looking like? that is another hard problem we had to solve. So we have dedicated steps for synchronization where if you've completed... So all of this is pre-planned. When you're running a job, first figure out what is the review process. So for example, with any other tool, you have to... Claude just finished working on the tool. Now let me do a code review. And then, okay, the code review is done. Let me use... Let me do a security review. A security is done. Let me run a test agent that tests everything. I have to ask you to do this every time, right? It's not going to do it on its own. And for us, we do that out of the box. You don't have to ask Blitzy to test what it did. And again, whether or not it needs testing is completely dynamic. That is also agent determined, right? So by deciding what is the path to production, right? By being an enterprise-only product, what you can now do is just focus on production, right? And then you decide what the checkpoints are, what the stages are for synchronization, when you should do it, what you should be checking at that point, and what are the expectations and acceptance criteria at each of those stages.

And then you can just drive the agents through it until, you know, you have something that is shippable, at which point you can hand off to the human. Yeah. So let's walk through that process a little bit more, right? So if I think about, you know, and because I'm not deeply familiar with Blitzy, I'm like mapping this to my mental model of an orchestrator. So, right, I take, let's use the example you use. I have an orchestrator handling my, you know, handing things off to Cloud Code. It says, hey, go and implement this. Okay, I got back. It's implemented. Hand it to a reviewer. What does that review phase look like for you? Is it also somehow tied into the graph? Like what is, do you have to have versions of your graph? Because now you're saying, oh, this is in this branch. It's still under review. Like what does that end up looking like? Yeah, so we anchor on the SCM for progress. and so that is the progress in terms of the work done and there's the agent work stream that is in flight, right? And there's a scope of the review of the, you know, based on the expectation of what should be complete at this point.

So the review as a process is anchored on the graph, the results of which are committed onto the SCM. And so throughout the process, you know, when you have a workflow that needs to be run by the agents, that is mapped and tracked onto the graph. And again, the results are captured into the SCM. So in case you pause the system at any point you can reconcile both and know exactly where you are in the overall journey And does that then continue on through you mentioned security review What does security review look like in this context? So security review, the thing is, you don't want to be writing any code that introduces security vulnerabilities, right? With the mythos scare that everyone had, especially because we're working with banks, you have to guarantee that the code that Plitzy writes is free from issues. So by default, you have, you know, another set of agents watch the works that are being done, catch security issues, and schedule security checkpoints to make sure that, let's say, those issues were temporary.

Maybe it was a scratch that the agent wrote, and it plans to discard that later. So let's make sure that happens, and that gap doesn't make it into production. or two, when we plan this feature, you're introducing a new API. I got to make sure that it follows the existing authentication patterns, that it follows load balancing and follows all of the core items that you follow, not just from security, but also from a housekeeping standpoint. So let's do that testing. Let's do adversarial testing. Let's try to break the system and product before someone else tries to do it. And let's flag that. Maybe there's a design gap that no one else saw. right so it's Blitzy's job to make sure that it exposes that if that item is out of scope it documents that if it considers that this should be addressed as part of the same job in scope it would go ahead and adjust that documenting is interesting so I think one thing to map back to like a human process as you're doing work you often uncover future work right you're like oh this is out of scope for this project

I don't want to explode my scope I don't want to explode my surface area but like I found this, we should really fix this. Does that end up getting resurfaced to a human to decide? Does Blitzy schedule itself future work to do those cleanups? What does that end up looking like? Yeah, we're on to something here. So we're working on the ability for Blitzy to file tickets and schedule itself and to be given a goal and recursively self-improve itself to achieve that goal. But what we do at the moment is at the end of Blitzy's work stream, we have two things. One, a project guide that documents all the work that proceeded and all the work that it could not or decided not to. Right. So that includes these findings. And that also affects the completion percentage if it was an in scope item. The second item is the PR review experience where it walks you through all the changes that it did, but just focuses on the key changes. So you don't have to review 500 files. You can review like 50 files and you understand the work that was done.

because if I have 100 files just with unit tests and you know they're all green and you know how they were written, you don't need to review my unit tests, right? You're focused on getting this to production. So using those two items, a user is able to feel more confident while approving the changes, reviewing it, and move faster to the review process. So that kind of raises another interesting question. I feel like one of the things that many of us are struggling with right now with AI-generated code is how do we keep our mental models of our systems up to date, right? It's great to have it working, but if I want to understand how I can evolve the system well, I need to understand the underlying structures. And I think, you know, in an enterprise setting, you have a lot of existing structures. Highlight Blitzy is very good to kind of respecting those. But sometimes things need to evolve. Sometimes things need to change. And especially if you have a long-lived code base, There may be pieces in there where, like, hey, this is hairy. We don't want to build in.

Even though everything is depending on that, new code should not look like that. We want to shift our model. We want to do things. So kind of what, how do you handle the, like, keeping the brains of the humans up to date with what Blitzy is doing for them? Yeah, there's multiple, you know, strategies. So, for example, especially we run into that use case a lot where, you know, I have this, but don't do that. That was a dark past. Don't follow that. So again, even when we look at commits, right, it's about recency. You may have one pattern earlier on, maybe years ago, and you maybe have moved on to a new pattern or a new technology framework more recently. So recency is weighed into the factorings of that agent that applies patterns. But other than that, we have knowledge base, which is one of the features where as you onboard and give Bitsy more code, it documents all of those branches and its understanding of those branches via XPEC, right? So it's a living document of what that code contains. So you can look at it, and as you update and resync code,

that document gets updated. So if you need to understand what any branch in your entire code base does, you can look at that knowledge base, and you can understand it. And then the more tactical items that, you know, our forward-deployed engineers work with customers on is creating documentation and keeping that documentation up to date using Klitsy, right? So what is the current state? What are the best practices? Before running a massive project, what makes the most sense is we follow this troll walk run approach where we suggest that you start with documentation. Let Glitzy confirm its understanding of your entire code base. Let us reverse engineer and let us show you what that looks like. Move on to adding testing, right? Because how do you validate the work that Glitzy did or anyone else did, right? So let's increase test coverage and let's add that. And then what those two things do is now it's very, it's much easier to describe acceptance criteria, right? It's much easier to explain to a human what that code does. And it's much easier, even for Blitzy, once the human has actually reviewed the documentation,

to validate its understanding and make sure it's on the right track. So by doing that, and then again, if there are any caveats, by capturing that in the agent action plan, sometimes there are conflicts to what the human thinks is in the code. and what they ask us to do, or even in their own prompt, right? So flagging that early and making sure that's addressed. Those are like some of the tactical things that you can do to make sure that the cost of this item is reduced. Yeah, that makes sense. Well, it's interesting, again, with the enterprise context, because long-lived companies often have parts of their code base that are poorly understood at best, maybe behave in odd and unexpected ways, but then there's downstream dependencies. I think my favorite definition of legacy code is untested code, right? And so this idea of, hey, first let's validate our understanding, both with documentation, but like you may not have anyone who can verify that documentation. So let's write tests that show what the system actually does. Yeah, I think our favorite moments and the customers that love us most

are the ones who have a lot of legacy code and a few people who can understand it. So it feels amazing when, you know, you have architects who have been there for like 35 years. You have COBOL developers who no longer write that and having them validate the code and really express satisfaction that Bitsy is able to understand and operate in a way that they would. I think that's the best part of building and working with Bitsy. Let's talk about another thing you mentioned here, which is how course corrections happen, right? No matter how good the system is, there will be mistakes, there will be stuff, and you have this human review process to check and verify. Let's say something goes wrong. What does the correction structure look like? How does that get parsed, represented, et cetera? Yeah, so I think you bring up a very important point. And part of my frustration with working with other tools has been that you blew up all of the context writing the project. It's like 700K tokens and it's done the work.

And now to do the next task, let's say you have to compact. So what happens is that it forgets many times the work that it is. Even though you have the plan, right? The plan is like a two-pager. The work is maybe tens of thousands of lines, if not hundreds of thousands of lines. How do you reconcile that, right? It's a huge problem. So again, because all of the work that Clipsy does follows the graph, right? The graph that it updated on the fly corresponding to the branch, we have a massive advantage. The way we operate is you saw the PR. We found some gaps. we have a workflow called Refine PR. And what that does is that it resumes the process, including the graph. It preserves the history, preserves the history from both the graph and the SCM and addresses your feedback. And the best part is that we bill on lines of code, right? So we don't charge you for mistakes that we make. Everything that you give us feedback on is a mistake. So what that means is if by addressing your items, lines of code in your resulting PR went down,

you will only be charged for the final lines of code. So let's say Blitzy went off and wrote 500,000 lines of code and when you were only expecting 300,000. Sounds like a cloud project. Yeah, recent cloud project, although I hope they fixed it with the recent models. You found that 200,000 lines were just comments and fluff that you don't agree with it. you didn't ask for, fine. You know, you can ask it to clean that up and then you only get billed for the correct answer. So that I think is very unique and special for Blitzy and I think is the right model for the enterprise because there's a lot of token anxiety these days. The tokenomics and the fluff of, you know, let it rip, burn all you can is all gone, right? Enterprises know that they need to be cost conscious with these things because if you let it run wild, it's going to, the wheels are going to skyrocket. and I believe we positioned our product and our pricing exactly with that idea in mind. Yeah, I like that a lot because it aligns all of your incentives towards, like,

let's write the right code, not all the possible code. A couple of things that that actually brings to mind. So one is sort of the convergence problem, and this is a problem that anthropic models have been more vulnerable to, but I've seen it on GPT models as well and other things, which is agents love to find one more thing to do. So you go, you build a thing, review it. Oh, you haven't thought about this thing. Okay, right. Okay, I'm going to go and build it. Okay, you haven't thought about this. And eventually it's like telling you, hey, once every Tuesday in July, this is going to break in this way. So how do you, within Blitzy, say this is the line, this is what's enough, we're done? I think you highlighted like a very, very important problem. It's not just, you know, anthropics. It's a problem with a wide variety of models. And people believe it can be solved, you know, prior art thinks that you can tweak it with, let's say, prompt tuning or with effort tuning. You know, hey, if you think it's doing too much, just drop the effort level.

But it doesn't really solve the problem, right? Because the way these models work, if there's one thought in there that is going to let it to think about finding new problems or have it document like one thing it found, you can't go back and change that unless you really edit the conversation, which is not something a regular user does. Right. So it's a really hard problem. But the way we solve it is. At any point of time, we use different models. the beauty of using different models is that you get fresh perspectives and you can align on the stronger model to be the tiebreaker. So we have internal evals and benchmarks. We know which model is relatively stronger for decision-making or which model is relatively stronger for front-end and then so on and so forth. So by applying that at runtime, so even if Claude thinks that two things worth flagging or this item is load-bearing in typical Claude speak, you have gpt come in and check and say nope that's not in scope that's not what the agent action plan says and gpt is more effective because it's doing less right cloud has the burden here

of implementing everything it has blown up its context it's operating probably in its in the ineffective side of its context window right it's loaded up with more than 300 kt opens gpt in this case was only looking at the AAP. And because it had to deal with much less constraints, it did a more effective job of identifying the scope. So this is a very vague example, not what we actually do, but we do something very similar across many models and we arrive at the correct answer to what the scope should be So before the user sees it that job has been seen by at least eight different models And all of those models you know are specialized at the task that they do and have been proven to be the best at that particular task right So imagine switching across eight different products locally to do one task right That's painful. It's not something you want to do. And you're going to have to maintain your environment and prompts and instructions for all of them, right? You don't have to do that because you speak to it.

So that is interesting and a nice segue into another question I had. So you're multi-model, which is great. I think that's definitely an advantage, being independent from the – right now, a lot of the harnesses are, like, tied to a particular model set and things like that. So being independent, letting you pick and choose is phenomenal. How do you decide which models to use for what? I think you mentioned internal evals. Like, what does that process look like, and how does it evolve? Yeah, so we work very closely with labs, model providers, you know, influence providers. and whenever a model is launched, we run it internally, benchmark it, we run it in the platform, we test how it's doing, we score it, and then we apply those learnings to decide which is the trial breaker. But at runtime, the process is entirely dynamic. We have these ladders and rungs, we have dynamic selection of the models, we have switches and routing in place based on capacity, based on intelligence, based on task specificity, and the agent chooses the LLM for the next agent based on the task.

It chooses the system prompt. It chooses the tools, right? So this is literally agent designed by agents, right? And what happens is as the agents and the models get better, it pays dividends because they're more precise. They're more effective at selecting the right model for the right job, right? So to build that engine, it took us years, right? And now we've perfected it. So it's really good at defining even the guardrails. Here's what's out of scope, and here's where you can check with me if you have questions. So rather than the agents are trained to go back to the human to ask questions, but adding this framework in there just helps us do autonomous work, where you don't have to go back to the human. Yeah, so that's interesting. I'm curious about that, having the agent choose the model, which I think is a really nice, you know, self-improving approach. But what do you use to describe the models to give the agent sort of dimensions on which to choose, right? Like if I just say, hey, your options are Opus 5.5 or GPT 5.6, like it doesn't know what those are.

Maybe it does, but like what are the kind of features that you're calling out about those models for the deciding agents? Yeah, so we ignore all the fluff that the labs claim about the model. You know, like one lab may think Model X or XYZ is amazing at something, right? Or the benchmarks may say one thing. So that's why you don't rely on web search for something like this. But instead, all of the results of the benchmarking and the e-bails that we had internally that identifies the strong points, you distill that into tags, right? And you distill that, you rank the tags. Like, for example, front end, you have a ranked ladder of which models to choose, right, based on complexity. So front-end, very complex, choose this model. Front-end, low complexity, choose this model. It's good enough, right? And then so on. So because you're optimizing for quality and speed, right, you need to choose the fastest model that gives you the best quality, right? So for that, we've built our system to be intelligent enough to make the right decision at runtime. That's really nice. So you're kind of mapping out what is that Pareto frontier along a number of dimensions.

And I think, yeah, front-end, back-end, language, complexity, those make sense. Okay, awesome. Is there anything that we haven't talked about about Blitzy today that we're missing that would be worth talking through? Yeah, let's talk about Sandbox, right, which is the new plan we have. So we wanted to make it more accessible to enterprises, and which is why Private Sandbox is what we've introduced. and what that means is if a company has more than 250 employees and you sign up with your work email, you automatically get access to Blitzy with a set quota and a significant amount of lines of code to both reverse engineer and to write. It's a decent quota. I don't have the numbers handy and they keep changing, so I won't cite them. But it's decent enough to do a small to medium-sized project, depending on what you scope it. And you could use that to build a completely new product, which is very easy to get started. You don't have to code your environment in or try to get it to understand your code base. There's no waiting. Just get started right away. You can easily evaluate how good Blitzy is.

Or bring in a piece of your code base. We don't train on any of it. We have very strict data retention policies and all of that. So it's an enterprise-grade platform. We're not doing anything sketchy. Your code is owned by you, etc. We don't store any of it. So you can run that project through Blitzy and experience the value. when you get the PR, when you go through the review experience. So it's a quick way to expedite the process where you don't need someone to handhold you. You can experience the value and then speak to our team if you'd like to move forward. I think it's a very generous offer. I don't think that's available with any other tool. So we really hope that a lot of other enterprises get to experience the value. And we're really confident about that, which is why we're doing it. That's awesome. So let's maybe now shift and look forward a little bit, right? So we've talked a lot about what Blitzy does. I think there's a lot of interesting things to pull out of that, whether you're in an enterprise where you can apply Blitzy now or considering or trying to think about how you're managing your own tools.

What's like the frontier? What are the things coming down the pipe? Well, I think the frontier is just two wild visions of the frontier, two broad buckets. I think there's some people who think that what's going to happen going away code review as a process is going away. Software engineering is dying. We're not going to be writing any code. We're not going to be reviewing anything. We're just going to ship. I think I find that personally outrageous, especially when you're talking about the enterprise. I don't think that's the speed at which the compliance frameworks evolve or, you know, that level of comfort is what the architects who actually use the models on a day-to-day basis have, right? I think they have a much stronger and clearer understanding of what the models are and are not. and to be fair, they're on the spectrum of Cloudify Autocomplete to AGI. They're not on either end. They're somewhere in the middle. At the end of the day, there are sequence-to-sequence models that are predicting the next token, right? But they're really, really good at real-world use cases, right?

So where does that leave us? If you go back to the beginning where I said most enterprise work is very highly defined and can be captured in a spec. And that's how enterprises have been building software for generations. We see the opportunity for the enterprise to hand off that work, submit the job, do something else, do more complex discovery work, do more complex research work that you cannot hand off. The requirements are not clearly defined. And the models are amazing at that. So the tools that you have, plot code, codex, etc., They're amazing at helping you do that work, right? And the well-defined work gets done by autonomous tools like you'd see, right? And these tools, the next frontier is like recursive self-improvement, right? So you can set an autonomous system, perfect something. Like I want to ensure that performance for my e-commerce website, my page always loads in less than a second. I have a very large team.

They're all shipping tons of commits every single day, but I cannot regress on this one metric, and it's very hard for me to enforce this with every single committer. So what I do is I set the system behind it, and this system monitors and makes updates every single day, floating up for human review dynamically when it needs it, but it achieves that goal, right? And we see that growing into not just distinct goals within the enterprise, but to entire products, right? If you can capture the product objectives and the metrics, right, and the standards that you want to maintain and achieve, then what you can have is autonomous systems like Blitzy run these products that are very well defined, that only need human supervision once in a while, and that can be easily managed, right? At the same time, shift the human attention to more complex items, to products that may need to be far more dynamic, where you cannot trust an LLM, where you need research

and you need to do a lot more auditing and compliance checks, etc., and focus the humans on that. So it's an incredible opportunity for every player in the system because the market is big enough to accommodate everyone. The software engineering is such a huge, huge market with such an elastic demand that it's a really incredible time to be alive and be an engineer in the space. Yeah, absolutely. Actually, one thing you mentioned there that I want to dig a little bit. So in this view of almost self-managing or autonomously managed projects, where do things like runtime environments and logs end up? We didn't end up talking about that for Blitzy, but are you feeding back operational data in? How does that end up working? Yes, so Blitzy is connected to the cloud, your cloud, essentially, AWS, GCP, whatever that is. Blitzy has its own environment. So typically when we work with customers, we have them capture their environment variables, secrets, configurations, instructions into Blitzy's environment. And then Blitzy replicates that environment across projects.

You can have micro environments. You can share environments across projects, etc. And for the vision that I talked about, right, that is the first step of getting to work with the system, right? Get it access to your development environment. It operates like an independent engineer. to have it recreate its environment, make sure it's good at that. And our environment is completely dynamic. There's no static item, right? The agents set up the environment every single time they operate. They do it dynamically, and they maintain that, right? So once you've solved that environment problem, once you've given it access to observability, logs, and once you've set goals, guardrails, boundaries, right, the agents are incredibly good at optimizing all of those simultaneously to making sure they do the right thing. And if they're unable to do that, to escalate to a human at the right step, to stop and get the human city, right? So we do that internally, and we've been successful at getting customers to trust us and to deliver value by doing that.

And so therefore, we believe like that is the place where the industry is going. Yeah, I really like that. And also thinking about things like security, vulnerability updates, all these different things. LLMs have increased the pace of vulnerabilities being found, but I think autonomous systems like this can also then increase the pace of reacting to and fixing those issues. Yeah, I think the biggest problem that people don't realize with LLMs is that LLMs have increased the pace of vulnerabilities being found, but they've also drastically increased the pace of vulnerabilities being introduced. And that's because you have to prompt an LLM to write secure court. You don't have to prompt a human right a human learns it and remembers that llms don't learn things until they're trained right and i'm sure the labs will continue to work on and get a lens to write better code more secure code you know but they'll always be because llms get much better at writing code it's not a lever that you can control there's not like write secure code optimized to 0.99 so this

is always right secure. You can't do that, right? Training is very dynamic. You can only control some other variables that are not this. So because of that, it's on the harness and it's on the system to enforce security, right? Which is why we take that very, very seriously and we have an entire team dedicated to this and we have entire workflows within Blitzy. We factor this into the design of how all lines of code written by Guitseer are built. Because once you solve that, once you help the customers get comfort about security, compliance, the guardrails, code quality, that's when you can move to the next level and actually bring in this transformation that you're thinking about. you