すべての企業に継続学習を：Trajectory 共同創業者 Ronak Malde 氏

A lot of our philosophy in product actually comes from Apple. In the early 2000s, you had a computer that had access to the internet, and you had a phone that you could make phone calls. And I think no one really thought to put the two together until Apple said, okay, what if you actually put the internet on a handheld device and combine the two? And I think by doing this, basically putting training, eval, and deployment in one place, making that just dead simple for every single person to do, And then you can close the loop and basically now upload that model back to production, gather new traces. It suddenly is this magical paradigm that I think every single company is going to adopt very soon. Thank you for joining us. It's a pleasure to have you. We would love to dig into a bunch of what you're building, how you see the world of research and where you think the future is going. Maybe to

start, you guys spend a lot of time thinking about continual learning. Can you define it for us? And you've also said this field is in a state of pseudo continual learning. What is real? And how will we know when we get to the real thing? That's a great question. So when I think about continual learning, I actually think the human brain is a really good model for where we should aspire to get to. Like, in some sense, we are true continual learners. We learn online, we are constantly updating iterating and i think the world we're in right now with agents and ai uh is kind of like you said pseudo continual learning when i was growing up i uh my family used to do a bunch of road trips right and uh i don't know how you guys did things but we would go to triple a uh back in the day and like they would give us a bunch of these maps so they would like highlight everything uh and then we'd go on our road trip uh and like if there's any point where we wanted to change our route, we'd have to either call AAA or go to a physical location. And I feel like that's the world we're in right now, where we have these amazing agents, they're very intelligent,

we have these mapping algorithms, but you can only consult offline a couple of places. And anytime you want to update your course, you have to just do an entire mass batch update. What the first goal and what we're building at Trajectory is first the ability to just democratize that access, right, to even just training these models and customizing them for what people want. I think of that as kind of like MapQuest, where suddenly it's still offline, but we at least got the ability for everyone to not need to go to AAA, but get the ability to map these algorithms. But the final stage of continual learning is something more like Google Maps, where every single time you're driving every route that you make, it is updating. It's learning aggregate what the fastest route is and where the traffic is. That's the world I want to get to. That's a really good analogy. So we had recently Rich Sutton and David Silver on the show, and both of them kind of made the comment that continual learning isn't really possible with today's architectures. I'm curious in your point of view, what's doable, what's not doable?

What's the state of play? Do we need major architectural leaps to kind of bring this to fruition? Where are we today? Yeah, so it's funny because when you think about a transformer in LLMs, I feel like a lot of people have seen maybe Karpathy's TinyGPT video or something where you basically have this stack of attention blocks and transformers that are built on top of each other. And a lot of LLM scaling has just been stacking these transformers on top of each other. But when you look at any paper, like DeepSeek v4.1 Flash that just came out recently, their architecture is actually kind of lobotomized or changed a lot of that base architecture, where you have things like fast and slow memory within the weights. You have skip connections that are going between layer 2 and layer 50. And so I think these architectural changes are actually already happening within the base models. And it will be a key component. I do agree for continual learning. Where I want to sit is basically leverage all of that technology and research that's happening at the architectural level,

but then actually connect the dots to the real world. Because at the end of the day, the real value of any AI progress, but certainly continual learning, is how are people actually using models in the real world? And I think companies have this idea of basically what good looks like in their mind. And the hardest part of this entire loop is then translating that to actual improvements for the model. I think that's where that missing layer is. Yeah. So how do you connect the dots to the real world and how much of that is product or productizable? And how much of that is just handholding and helping people through the process? Yeah, so up until now, it's been really hard to just go from this fact of data to intelligence. Up until now, that's only been in the labs, and you need a massive research team. So in the last couple of years, we started to have vertical companies going into this space as well and building a massive research team. So I was previously at Windsurf where we embarked on this journey and trained a model.

So we won. So did Cursor as well with Cursor Composer. but the problem was you needed hundreds of millions of dollars of r&d in compute you needed a really massive research team to invest in and then you also needed a lot of time so it took us six to nine months to actually get to that point where we could train a frontier model and that is with a kind of hand-holding sort of consulting model that's still sort of the world that we live in I think any good product exists when you have every single company in the world trying to do the same thing over and over and over again. And if you can package that into a really beautiful interface that just says, here are the knobs and levers that you care about to translate that, what good looks like. And then everything else is packaged away. I think that's, that's a beautiful interface to do that. What did you and your co-founders see when you were at Windsurf and when they were working on models as well to help you realize, hey, this is repeatable, that there is a project build. When did you have that aha moment and that realization? Yeah, so part of it was, you know, we were training these models on production, like basically saying, what are all these edits, the retries that are happening in the product and learning from that at Windsurf?

And then combined with that, so my co-founders, Michael was at DeepBind as well. He was working on Gemini Robotics. And they're starting to see the same thing where you could actually deploy these robots in the real world, build continual learning infrastructure to basically allow these robots to update every single day. and then Arjun was at Apple working on the Vision Pro and the same thing was he was basically saying how can we take artist feedback and use that for multimodal foundation models so in one way or another all of us were basically figuring out this grand alignment problem of how do you actually take this feedback from the real world and improve the models the problem is we were all at these big research labs and so I started to realize okay continual learning is going to be the next learning paradigm is there a way that I can actually do this within the labs And the thing I very quickly realized is at DeepMind, our entire focus was training one massive model and then distributing that one model to everyone in the world. And that's been a winning business model, obviously, across the board. But if you want to basically take one model, customize that for a single person, and then distribute that one model just for that customer, like, for example, what our platform does with Harvey,

that is an immense amount of infrastructure for a massive lab to build just for one customer. Yeah, talk about what you guys have done with Harvey, maybe as an example. So Harvey, basically, they were up until now riding on top of the frontier closed source models, right? And they've been able to build incredible interfaces and products on top of that that help tons of law firms in the world. But what they realized is they have an immense amount of knowledge of data of all of this nuance built up that they are actually able to see what the law firms actually care about and the built up experience of just 30 years of working in this domain. And with using frontier models, they're unable to actually take that data and use that to their advantage. And so what they realized is, hey, okay, they're able to construct, for example, a legal agent benchmark and what they released and using all of the expert information that they have. And then if they get the power to basically use that data to train a model, then they get to shape their own destiny. So with Trajectory, what they were able to do is train a model that is six times cheaper. And it goes from around 4% accuracy for an

open source model to around 18%. And there's still so much room to grow, obviously, what they're able to do. But now suddenly they get the power of a research lab. And on top of that, the nice part is these models end up being six times cheaper than the Frontier. And so you're able to expend so many more tokens to achieve these tasks. So you picked the name trajectory as it's related to this continue learning concept. How did you pick the name and tell us how it fits into what you guys are doing? Yeah. So it's funny because the world right now is very focused on tokens. And I think a lot of people view tokens as a unit of work. But if you think about the way that we interact with other humans, I'm not counting the amount of words that you're actually talking about. right and with co-workers we a lot of the time are thinking about a task or a trajectory and that's the unit of work that I really want to get to it in RL it basically is synonymous with a rollout or what you would actually do in a training stack and with continual learning you have an RL training stack that's like a rollout engine and a trainer a rollout engine is just like in a

simulation basically like doing a bunch of rollouts and doing these trajectories in the real world for continual learning, your rollout engine becomes just the inference and the tokens in the real world. And now suddenly your trajectories are the tasks of economic value that are happening. And so I want this basically paradigm to propagate where the unit of work that we think about is a trajectory. Beyond that, I think for also people that don't understand RL, it is also super nice that we are basically letting companies just own that path and really grow on their own. So I think has this nice double entendre. I like this idea of a trajectory as like a unit of economically valuable work. You've also a couple of times talked about what does good look like, which kind of sounds like an eval or a benchmark. What are the core primitives for continual learning, or what are the key ingredients that kind of go into this? Yeah, so our goal at trajectory is to actually limit these ingredients so that it is literally just, you know, what good looks like. However, right now, there are a lot of components that companies need,

and then we as our product as well try to bridge that gap and make that as easy as possible. So right now, as a company, you need a couple of things. One, you need to be able to trace all of your trajectories that are happening in production. And there's lots of AI native companies that have gone through this transformation where they're logging not just the main agent call, but also all the sub-agents, what's happening in the product as well, any feedback that the user is giving. So the ideal world is you get this kind of perfect timeline of all of those things so that you can track them. So that's the first step. And just on that, what are people doing that in today? Like where do those choices live? Yeah. In my view, I think the entire ecosystem of continual learning needs to be a lot of players that basically build up all this infrastructure. So some companies are doing this in-house, but there's also a lot of great products out there like Blankchain, like Braintrust, Raindrop, where you can basically plug into your infrastructure or your production agents and then basically trace all of these things. So what we're finding is a lot of AI native companies will use a lot of these products

in order to get that first step out of the way. Okay, so we got traces and then what? Yeah, so that's traces. The second part is then making these replayable. You need some sort of infrastructure to say, hey, okay, given a trace that I've seen in production, how can I actually then replay that for an eval to see what good looks like or right now for training as well? And so that's where companies also need to invest some sort of time. What we make really easy in our product as well is to bridge that gap. And so if you bring your traces and you say, okay, here's just the concept of an environment that I might want. In more technical terms, it's a Docker file or some sort of system requirements. Then you can basically replay that entire trace. So that's the second part. And then the third part is then the actual modification and the algorithm. And I think that's where it's really difficult to do this in-house as a company is basically the entire training stack, the entire loss function, the reward function, all of these sort of components that need to be built up to translate these tasks into actual improvements for the model.

And so that's the main part that we want to abstract away off the bat. To your point on abstracting away some of that, what has been abstracted thus far? What's next? What's kind of further out? how does this product come together over time? Right now, it's mainly you bring traces and reward and some notion of environment. The two parts here that require a bunch of manual work is the environment, actually constructing this thing that matches the real production. And a lot of the time, it's actually impossible to create a perfect environment. And then the second part is reward. A lot of the times, a legal workflow or investment banking workflow doesn't have some perfect verifiable reward that you can train off of, right? And so these are the two areas that we're really pushing on the research side. So for the environment, there's an algorithm called GRPO. That's the main kind of post-training algorithm that exists right now. What you need to do there is you basically, like imagine you have a player that's in a video game.

and it's basically going around and hopping around a bunch of platforms and then at the end of the day you get some reward. What's needed here in GRPO is basically comparing like the same task, the same level of the video game and you need to play that out eight times or 16 times and we call that a group. Now the thing is with humans a lot of the time we don't need to replay the exact same environment over and over again. We're just in the world experiencing and doing one rollout and so the goal is could we modify GRPO as an algorithm to get to a world where it is what we call group size one instead of needing a group of 16. And so we've made a lot of strides here where now if you can take 10,000 trajectories and basically they're all from different tasks, we can actually learn from just that one behavior or that one rollout. And so that's one algorithm that we actually released a blog post developed is going into this group size one paradigm. And so that actually eliminates the need for an environment and all of this infrastructure that needs to be built out. The second part is for reward. And so this is another factor where, let's say I have, again,

10,000 traces. The default is that I need to go and label every single one of those traces to say, here's verifiable reward, or here's what good looks like. And this is tractable. It's just a lot of work. And so now our goal is, how can we actually reduce that amount of time needed to take this notion of what good looks like and do that in less amount of work. So what we've realized is, let's say I have 10,000 traces. If I can smartly select 100 of them that are basically representative of all of these broader 10,000, and then have the customer or the user basically densely say, okay, here's what good looks like. Here's a rubric or verifiable criteria for just those 100 traces. And then what we can do is basically train a metagrader that learns from those behaviors of the 100 to the broader 10,000. And with that, you get a really clean interface that allows you to just propagate this reward across the board So I think obviously it not the perfect solution in that direction The ideal goal actually is that you have some sort of observability interface where you like here the five things my agent good at here the five things it bad at

and I just circle those five things and hit train. That's the world I want to get to, and that's where we're pushing that research direction in. Very cool. So you have a product that you just released to do some of this. Tell us about it. What are you releasing the product, and who is it for? Yeah, up until now, it's taken a massive research team a ton of time and a ton of compute in order to train these models. What we wanted to do is just make the easiest and most intuitive interface to train models and customize them for what you care about. And so Trajectory, the platform that we released, is basically a very simple system where you can take your agent traces and what good looks like. It's a one-click train interface that just takes a couple agent prompts, and then you can eval the model and then deploy it all in one place. And I think the really magical part about this, a lot of our philosophy in product actually comes from Apple. And when in like the early 2000s, you had a computer that had access to the internet, and you had a phone that you could make phone calls. And I think no one really thought

to put the two together until Apple said, okay, what if you actually put the internet on a handheld device and combine the two. And I think by doing this, basically putting training, eval, and deployment in one place, making that just dead simple for every single person to do. And then you can close the loop and basically now upload that model back to production, gather new traces. It suddenly is this magical paradigm that I think every single company is going to adopt very soon. Very cool. Who is the product targeted at? Like who's the ideal customer for you guys today? Yeah. So what we want to do is make this very simple for companies that have already their agent harness in order. They've spent a lot of time there. But that next step of basically modifying the most valuable part, which is the model, making that super, super easy. So right now, what the product is meant for is mainly AI native companies like Harvey and Rogo and some larger companies as well, like NVIDIA, are already using the platform. But the next step after this is I want to also move beyond just the model.

So another important piece is the harness as well, because I think the model gets you efficiency. It gets you a model that has seen a task 100 times or 1,000 times over and making that faster, cheaper, and more reliable. What a harness gets you also is new capabilities. It's just something that was not possible before. And I think a lot of larger companies that are adopting AI, they also haven't done their work to optimize the harness for sub-agents or for basically squeezing out that last mile even out of the frontier tokens. And so the eventual world is maybe you use a frontier model like Fable for 10% of tasks, and then for 90% of the time, you can smartly route to a model that you've trained that is far more efficient and also gets you better margins. Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples. Yeah. So one really interesting one here is Rogo. And I think Rogo is taking a very interesting paradigm to agents that I think will become the future.

So right now, the way people use agents are very much like synchronous, right? You prompt the agent, it goes and does its thing, and then it comes back. What Rogo is very interested in is not just this kind of verbose model that will go and basically spit out a bunch of garbage. What they want is just a very concise and clear finance agent that basically does the work that you care about. It can autonomously do things on the side, but then it comes back and is very efficient. And so what the goal of Trajectory is is to basically say, okay, Rogo, what good looks like for them is they also want the idea of intelligence density. They want the idea that a model can do the same task, but do that far more efficiently than the frontier models. They have the data to do that. And so with Trajectory, what they're able to do is train a model that is several times more efficient in terms of its tokens and also happens to be cheaper and faster, but then is also able to gain and climb on the things that they care about.

Very cool. Okay, so we talked a little bit about Group Size 1 with GRPO. What's next? Where is continual learning headed? Yeah, so a lot of what we've been talking about right now is mainly just helping these companies translate over what good looks like for them into improvements for the model. I think to get to this true world where humans, we are extremely efficient about learning from the real world. And we're able to have, I think, inbuilt curiosity as well about the world. And I would say that's kind of one of the missing pieces to get us to agents actually learning kind of autonomously on their own via continual learning. The models right now, they do a task, but they don't actually accumulate any experience or learn anything about the real world. Whereas if you think about like, let's say I'm throwing a football, right? I feel like as a human, maybe if I've never seen a football before, I actually have no idea what's going to happen. and but I do that a hundred times over and now I suddenly build this heuristic right yeah and

the important part is I think with humans like as kids we are constantly exploring the world we see a soccer ball we see a football and we kind of just try out things to build this mental model and that makes us incredibly intelligent whereas with agents that doesn't exist right now and so we've done a lot of thinking and how do you actually quantify curiosity and I think one interesting way to think about this is, so there's this idea in RL called a Markov decision process. And it's basically, you can characterize the world as a bunch of state action pairs. So a state is like, I'm holding the ball in my hand. And then an action is I throw it. And then the following state is where does the ball end up? What LLMs right now are really good at is basically going straight from state zero to state one. They're next token predictors, right? But what's missing is that intermediate part of what is that transition function? When you actually go from S0 to S1, what is that transition function that happens? And LLMs don't really have a way of

encoding that right now. So I think the interesting thing here is with humans, we are very aware of when we don't know what a transition function is the world. And we say, I'm basically going to try to figure out what this transition function is. And I think this is where curiosity comes in. And so if there's a way to basically quantify, here's all of the world that I don't know about right now. And my goal as an agent is to figure out if I do this task, what happens? I build this mental heuristic. Then I think we are starting to solve this curiosity problem and get to a world where you can autonomously kind of learn and continually learn from the world. How do you make sure that an agent's curiosity is aligned with a user's objectives? So I think that's actually one of the reasons why I'm very passionate about working on continual learning. So there's two kind of directions that it seems like research is going right now. There's RSI, recursive self-improvement, and there's continual learning. And both of these are very tightly tied together, right, or very similar concepts.

They're essentially just how does an agent actually learn from experience. The thing is RSI is all about taking a model, putting it in a black box, and basically making it the best AI researcher itself so that it can improve itself. But this is kind of a scary world because it exactly begs the question of are these models and agents actually going to be aligned and have the best interests of humans in mind. Where I think continual learning takes a very different approach is it says I want to learn from experience but learn from the experience of actually interacting with humans and then getting better at the tasks that they set out and that they want to do And I think that's where the key difference is, is you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue. And I think the way to do that is to keep on this alignment path. You guys have done a bunch of unique things across product and research at Strictory to bring these two halves of the whole together. Tell us more about what makes building at Tradector unique and who are some of the people

you bring together to make it happen. Yeah, so the cool part that we are sitting in is the intersection of frontier research and product, where we have to modify the core RL algorithms in order to get to a place where you can actually learn from the world, but also package that into a really beautiful and clean interface for people to train these models. And so some of the research team that we've built. It contributed to, for example, the co-founders to Gemini 3 and Gemini Robotics, other folks to GPT-4.0 at OpenAI, and building used infrastructure at MSL. But then on the product side as well, having people that have built scalable infrastructure at Stripe for payment SDKs. Also, we have the youngest design engineer at Netflix. These sort of people, and I think the intersection between them, is where the magic starts to happen. So the way we've actually structured the team. Normally you have like a research stack that's the spaghetti code base in one corner of the company. And then you have the product that's kind of well built out. What we've done is actually productionized the code base for training as well. So this means

microservices, Kubernetes scaling, auto scaling of every single part of the trainer, the rollout engine. And then on the design side as well, our design engineers are kicking off training runs all of the time and understanding deeply what the research experience is. I think this is a really interesting and cool place to be in. A lot of this is also only possible because of Frontier Coding Agents. Actually, most of the time now, we are spending our time whiteboarding, and we'll spend a lot of time just arguing about system design. And then the actual implementation is just a software factory of agents that will go and actually implement it. So building a product here is not easy. What is the role of a product builder in a world of continual learning? I think as a product builder, you should be thinking about how you actually take these traces and experience in the real world and start to shape and improve the product autonomously. So we're starting with the model, right, for trajectory. And I think there's some interesting work to do there. Eventually, that will be the model and the harness and basically self-improving both of those. And I think once you have the harness and products are kind of

becoming more AI native. And so the harness is kind of becoming most of the product for a lot of these companies, right? Well, and if you have the right to kind of self-improve the harness and then the product, then you have the right to kind of improve and self-improve the entire company. And I think there then products and product builders start to become almost like a gardener. Like you have your plant and it's going to grow autonomously on its own. And your goal as a product builder is to basically just set these trellises in the ground of here's where I want to shape my product. And I think this is going to be a very exciting world where you can just put a product out to a couple hundred users. It's going to grow on its own. And now we're going to have basically you just imbue your taste into where the company grows. So I think that's going to be a very exciting world. And companies should kind of start building towards that. For somebody listening who's thinking, boy, trajectory sounds like a cool place to work. What's the pitch? Like, why might somebody want to work at trajectory? So I think there's a couple of things. The first thing is continual learning as a paradigm is the next learning paradigm.

Any massive company that you can think of of the previous age. So let's take Google or Amazon or Facebook. They were all doing continual learning, basically. If you think about, like, recommender systems or ads, the idea is taking all of these preferences from the real world and then creating online systems that can learn and get smarter the more they're used. And I think for agents, that hasn't existed yet. And that's exactly the place that trajectory I want to be in. We're starting with just V1 of the product, which is a very easy way to train models. But in aggregate, I want it to be a massive recommender system. Every single agent should be attuned to what companies want and what every single user eventually wants. And I think that's going to be a fundamentally exciting world to get to. There's a lot of work that needs to happen. And I also think Trajectory is not the only company in the ecosystem that needs to work really well to make this happen as well. You need two blocks down from us is modal at our office that works on sandboxes.

sandboxes. And that powers a lot of the infrastructure that we have. And two blocks down from that is Thinking Machines also in SF. And I think there's like all these sort of companies that need to be built to basically power all the infrastructure to make continual learning work. But I think the product interface is where a lot of the exciting work can happen. Yeah. One of the things you said when we were first getting to know you many, many months ago was this idea that the world right now has plentiful intelligence, thanks to the foundation model labs and plentiful experience, thanks to all the people who do hard work in lots of different domains every day. But there's this big gap between the two. And in some ways, continual learning is kind of the thing that bridges that gap. And it's kind of the thing that honors all of this earned experience that people have and makes it legible to the intelligence coming out of the foundation models. And I thought that was a very kind of elegant and beautiful way to think about what it is you guys are doing. And then the question that that leads to is, what is your most optimistic

view of the future in terms of where continual learning can take us? I love that question. A good example of this is Adam Smith's Wealth of Nations, where we started out in the 1700s in this world where everyone was kind of doing the same thing, right? You would do this end-to-end workflow, you could call it, of an artisan. And what he theorized basically is a world of abundance needs specialization, where you want someone to basically not be context switching all of the time and work on the thing that they've built up experience about. And it seems like in AI right now, we're in this pre-industrial revolution society, or even a hunter-gatherer sort of society, where everyone is just doing the same work over and over again. We're consulting one oracle of technology. I think what needs to happen is basically get to a world where it's no longer just one manufacturer of a car, one manufacturer of every single unit, but we get to a world of specialization where everyone can basically customize every single

part of the harness, the model, and that's where we get to a world of abundance. And there's a reason why only 1% of enterprise has actually been adopted by AI. And there's so many workflows out there that are super manual and could be automated and actually increase intelligence based on all of these agents that could be in production. And so my goal is to basically take that 1% and turn that into 99% so that we are actually learning from all of this built-up experience that lives in the world right now. Love it. Ronak, thanks for being here. Yeah, thanks so much for having me. Thank you.