顧客データが見えない中でリーガルエージェントを作る — Legora CTO ジェイコブ・ローリッツェン

People were asking us about mythos and everything, even though I think realistically a lot of legal work does not require mythos-level intelligence. Paralegals, gone. Junior lawyers, gone. I don't need them. That's actually not what we're seeing. We had one problem where it wouldn't cite properly, and so next model release solved, it went from $0.70 to $0.99. We did nothing, and great. If you're doing a $400 million M&A transaction, you're going to want the war. We're just going to be 20 engineers for the rest of the company's history, and we're going to beat everyone who's much bigger. and we're 100 engineers now and it feels like we're way too small. But it's something that I'm starting with is like how much of our product can just be the agent versus all the other stuff. Okay, we're here in the studio with special guest, Jacob Lauritsen, CTO of Lagora. Did I get it right? Lauritsen? That's fine. Yeah, that's okay. I'm not surprised to show that how it's Danish. Yeah, exactly. Does it have a special history? It just, no. Sen is like son, son of. Son of Laurits. Just like you have son and, yeah. So son of Laurits, that's my name. Nice. We've crossed paths here and there. You spoke it a few times, and obviously Legora is one of the fastest-growing legal tech in history.

Yep. And you said something at the start in a pre-show that I thought was really interesting. You said that your origin is very European, even though obviously you're expanding in New York and SF. What feels different about your culture? I think we are just at the core probably very humble about what we're doing because we've always been the underdog from the very beginning. No one believed a small startup in a basement in Stockholm was going to outcompete this huge behemoth that was way ahead of us in the U.S. So I think we've always been underdogs, and so we've always been very humble. We've always sort of had to fight for our existence. I think something we've had to learn as we've also become more and more, as we've entered the U.S. and been in the U.S. a lot more, is to be a little bit more bold and tell the world how great we are, because it doesn't come naturally to, I think, some of us. I kind of want to start by going over your guys' product because I feel like with a lot of these legal products, they're kind of under wraps. You're not supposed to show it. But yeah, if you could just talk about what does the product look like and maybe how did you guys get there?

What have been the product iterations that got you to where you are today? I think that would be really interesting. Yeah, absolutely. The way everything started was ChatDVT with Swedish Loss. Okay. There was a search tool in it. That was literally the entire thing. So that's where we started. It was started with legal research, which is like a huge market basically in the world where lawyers spend a lot of time figuring out what are previous cases that look like this, what's the legal precedent, like, is this, you know, what have previous people done in these situations? That's extremely important. And so that has always been part of our product. Basically, we started from like an agent, you could ask questions and it would query loss. That was a very simple beginning. And then from that, it's expanded. And so it's expanded in like, what can the agent do? It will now draft a lot of stuff really well. It will edit Word files. It will move your files away, organize everything, run really complex analyses. And we've added just a bunch more features to it. So one of our features is called Tabular View, which is now a name that's being used in industry. And we're very happy to see Cloud for legal use in our terminology.

We've got to talk about that. We've got to talk about that. which is essentially like a way to just bulk extract and bulk review many, many, many agreements, which you do in due diligence and when you raise money or whatever. So that's part of it. It's like we help lawyers just do their work much more efficiently, much, much better. And then outside of just doing the work, we're also now helping orchestrate the work a lot more than we were just even a year ago. So we have different sort of – we have a thing called lists, which is sort of like linear for us. and we have Intake, which is a way to collaborate with clients and we have Portal, which is also sort of a way to collaborate, you know, lawyer to lawyer, client to lawyer, lawyer to agent, agent to agent maybe. We're spending a lot of time there because there's just, frankly, the way that you work with a lawyer 10 years ago or maybe even today is really painful and there's loads of room to do that in a much better way, to be honest. And so those are, I think, the two things. So it's doing the work, and it's also, like, orchestrating the work. When did you guys start doing or building the agent?

So, I mean, we basically had to switch from the assistant to the agent. Yeah, we call it right. I guess that the agent, I don't know, I want to say a year ago, probably. Yeah. So this is a rough timeline for people who are understanding YC 2023. Yes. And then you were a YC founder as well. YC 24. Yeah, I was in YC 23. Yeah. Agora was 24. Okay. And then, so there is, this is like a little bit pre-agents really taking it. Exactly. And like reasoning models obviously change everything. I mean, you know, in one sense today, you know, we had a preview of the products. It does look, well, preview, we had a private thing. Well, we will put up a video on YouTube. It's mostly like chat GPT for lawyers, right? Like very fair to say that. But then like the sidebar has a lot of nice custom tools like the tabular view, like the intake, which I really liked, and a bunch of other things. Exactly. That's exactly what it is. It's co-work or chat GPT. They're really, really good for legal work. And the workflow for either in-house corporate lawyers,

people looking for upcoming regulations, all these things. And we can talk about more of the products there. Yeah. What models are you guys using underneath? Can you talk about that? I mean, we're using a bunch of different ones to be on task, right? And then that's not available to the lawyer to see. You guys pick yourself? For the agent, yeah. Well, we've been back and forth on this. Yeah. Just like everyone else, I think. In the agent, there's a model selector for the primary model, but there are certain tools that we use different models in, and there's like tabular view, and like we try. I think we found it most effective to subdivide the problem and route on like as we subdivide the task, but we do right now have a picker in the agent. I'm not sure if that's going to stay there always, to be honest. So, yeah. Yeah, I mean, so I bring a little bit of context to this because I'm an advisor for Cognition. Yeah. And when I joined, you know, part of my thesis for what an agent lab is, an agent lab is, you know, ideally a domain-specific GPC wrapper, right?

Like we run the, we are the domain experts. We will pay attention to you. We'll serve your needs way, way more than the Frontier Labs. And we'll give you the best of the Frontier Labs. Yeah, we'll have to put away our job. So our job is to decide for you because your job is not to... Exactly. We'll always keep you on the Pareto frontier and make sure that it's great all the time and you don't have to dive into all the details. Because one of the things that I've exactly to this point, if you go to a lawyer and you ask them, oh, you're going to draft this contract based on this precedent, do you think that's a sonnet high or an opus mid task? They won't know. Just give me the smartest model. Just make sure it works. My hourly rate is $1,000 an hour. Exactly. Just make it work. so yes I originally came in with that I was like oh yeah this is what Devin does differently and now we have five things in the model picker including Fable and Opus and Astra and people want to experiment with the fancy new model do you think is that the same I think so I think it's probably it's definitely way less illegal than it is in coding engineering yeah but there are definitely people

that just want table one right away and you know people were asking us about mythos and everything even though I think realistically a lot of legal work does not require mythos level intelligence yeah yeah so exactly right like i think basically you're like okay we'll give you the option you ask for this exactly but you're gonna run down your credits like 10x faster than yeah i would have loved we fought introducing it for a very long time and then in the end we just kind of caved in and we're like okay if you want this level of control yes you know then we'll give it to you yeah so you think the models are already like as good as lawyers need like there's no need for them to get better I think it's a power distribution. So, like, there's probably like 60% of the work we can do really well right now. And, you know, from now on, it's like a cost and latency thing. But then there's the rest of the thing where, especially for, like, very complex analysis, we still need more intelligence. And then I think there's also just the work expands kind of. Like, that's the work that we have right now, but also the work with what a lot of thus probably expands with intelligence.

Yeah, I imagine. Early on, when you guys first started, I guess, transitioning to the agents, or building the agents, what models were you guys using? I guess it would have been like Opus 4.6 or something. That would have been, yeah, Sonnet 4.6, Opus 4.6 is probably right. What were the agents not good at at that point, that they're much better at now, even though it's only nine months later? Sub-agent delegation was one of them that we've gotten working really well now for fan-out type retrieval queries. works extremely well for us now. That was difficult before. We basically track, so we have a team that's just doing evals, and so we track all of the model's performance in different practice areas, which is like subgroups of legal, and different tasks for us all of those. So we basically have like a big heat map, and we're like, and then we can see them move, and it's like, usually they improve most stuff, but there's actually a lot of model releases where they improve some and regress in others. Okay. And that actually happens a lot more than you'd think. Like, what are some of those tasks? Like, are you familiar with how you guys are running a lot of the emails?

Like, what are those different subsegments, I guess, that you look at? So, like, there's different just, like, framing stuff. So, we have different, basically, like, worlds, like, RL environments, essentially, which is, like, here's a, because lawyers work in projects always, more or less. So, it's, like, here's 10,000 file data room. You're doing a due diligence. In the end, a grid report looks like this. Like, it contains all these red flags, and, like, it doesn't contain whatever weird stuff, and, like, we want it to be concise, et cetera, et cetera. That's kind of how we've built it. So there's different tasks, which is like part of it is drafting a report. Like does the draft look good? Does it contain all the right stuff? Is the wording sound the analysis sound? There's also the part of finding the red flags, like the analysis before that. So you can kind of like self-divide it a little bit. And that's how we run the evals. Interesting. What does your evals team look like? Do you hire lawyers to like do that? So, yes, it's a mix. We have engineers and then we have lawyers and it's one team. So there's like three engineers and like three lawyers right now. And then we have a pool of what we call legal engineers,

which is people that have worked in big law for like three or four years. Typically, maybe they have a CS degree and like they're very tech forward. And then they come to us because they want to like help change the industry. Are they IT in big law? No, they're lawyers in big law. They're lawyers in big law. With CS degrees. Yes. Oh my God. It's a Venn diagram. Yeah, exactly. That's like our hardest person to hire for. but they're also such a force multiplier because we can pull on them for the eval stuff and for the product development which is like you have a pool of 100 experts that use the product every day and go and talk to clients every day and also like they live this and have lived this and we just pull on them to do evals and just like help us shape the product and it's such an advantage. And curious, like when did you guys start that team? Evals grew out of the assistant agent team at some point. I think there was a point up until recently where it was like honestly just not really worth it. Because all models just got better. So it was like, do we want to spend a lot of our time doing really complex evals? Or do we just want to build? Because there was so much to be gained from building a better harness

and just sort of leaning into everything's going to improve. And everything kind of just improved. Yeah, like every two, three months you get a new model. Yeah, exactly. Just pass it through, right? Exactly. I mean, we had one problem where it wouldn't slide properly and so it would be difficult to follow. And it was like, next model release solves. It went from like 70% to like 99%. So we did nothing and like, great. Yeah. Maybe the real work is keeping all the data private, making it taggable, sortable, adding new features like the tablet view. Is that the... You mean over and above? Well, I mean, yes. But obviously now you're also doing evals. I assume your use of open models is also increasing. We can talk about that. So, yeah, I don't know what I'm saying here apart from, like, you know, what is the job? The job is maybe not, in the early days, it was not so much to do evals because it was mostly exposed model capability to lawyers. The huge part of it is just figuring out what does the legal profession look like in five years, ten years with all of this stuff.

Coding is maybe a year or two ahead, so we can kind of look to that a little bit. But what are the primitives? How do people work? How do you collaborate if it's not a 200-thread email with V5 word files back and forth? What does that look like? And that's not even necessarily purely AI. That's just a much better product experience and make sure you do it with audio logs and data retention and all of those really important things in regulated industry. And then there's the AI part, which is make the work faster and better and proactive agents that react to stuff when it comes in so you don't have to kick off everything. So I think a lot of the work is just figuring out what we think is going to look like in five years. If model capabilities keep increasing, If price keeps coming down, what could we do? Can you log in in this command center where it's just like, you need to react to these five things that the agents have been doing lately and review and check and improve the system? Maybe that's where we're going. Or is where we're going. Yeah, I was going to ask, could you go into that in a little more depth? Yeah, what is the law profession going to look like in a few years?

I should preface this by saying I don't know. I mean, who knows really? but I think we can look to coding a lot. I think all lawyers are going to be way more productive than they were before. They're going to be able to do a lot more stuff at a higher quality for a lower cost. I think the profession itself, the work itself, is going to improve a lot. It's going to be a lot more. I think lawyers are going to spend more of their time doing the stuff that they became lawyers to do, figuring out what's the right risk profile, what's manageable risk, what's like all of their judgment, strategic judgment, and like working with clients, they're going to do that a lot more and much less of the like trolling through big documents like find the smoking gun or like do all this menial, super annoying work that a lot of lawyers have been doing. What do you think is going to happen in the partnership model? Like do you think that saves or like what happens? I gave this, I actually gave this a lot of thought just like last week. Yeah. The partnership model, I don't want to get into too much of the details, but like the partnership model kind of exists because the only type of capital that these companies have is human

capital. So it's not like us or a software engineering firm or a manufacturer or whatever, you have physical assets or some sort of intangible assets that you accrue over time and they pay dividends to the shareholders. But if you work in an industry like professional services, and it's the same for consultancies and whatever else, all of your value is in your people and you can't really own human capital. You know, you can't, you know, slavery is abolished. You can't own people, but you can rent them. You can pay the money. And so partnerships essentially is a structure where you make the human capital the owners of the company so that they stay. Yeah. Will that change? Like, I don't, it doesn't have to, but I think what we really want to do with lawyers, we actually want lawyers to be able, law firms to be able to build some sort of intangible capital, some durable capital. And like, you can build a system that learns of how you work. A lot of firms right now think about like, well, there's the ex-law firm way of doing things and we can codify that, I think, and we can build a system

and whether that's like a self-improving knowledge graph or whatever it is, and like precedent files and documents where there's like a way of working that produces a really good output and that can be their IP. So it's like a way to codify the human expertise and judgment into a system. That I think is, now I'm just rambling completely, but that's something that we want to help empower. And does that change the partnership model then? Because the incentives are different. In the partnership model, you just got to bring more clients, and you don't want to take away time from bringing in clients to quantify your information, basically, because you get rewarded from eat what you kill or make this lockstep, whatever. That needs to change a little. We need to figure out an incentive structure where partners are incentivized to quantify things to build an asset for the company. but right now they aren't because as soon as they leave the partnership, they don't get any dividends from the assets that they built. So there's like a, yeah. I think the whole business model is, we'll see.

Yeah, yeah. It's all in question right now. Yeah. My actual other take, you know, so on the partnership level, that like the top end of the industry Actually my take is more on the bottom end right What happens there Paralegals gone Yeah Right Junior lawyers gone I don need them Yeah That's actually not what we're seeing. Okay. I think it's... I don't know. Would you say that we're seeing that in engineering? Yeah. Yeah. Junior engineers are kind of cooked. You kind of have to come out of... Drop out of university fully baked, or else you're not going to make it. I feel like I'm being a bit dramatic for comedic purpose. But, yeah, I know a lot of lawyers, they started as paralegal, they go to law school, all these things, right? And now they're not needed. They're seriously just not needed. Yeah, it's interesting what happens to the partnership model, like the pyramid. I think generally a lot of lawyers, you need the next generation. And at least for engineering, the way I think about it is you have a good engineer today.

they're worth a lot more even than they were two years ago because they can produce so much more stuff like if they're good and i think it's the same thing in legal where they can actually do a lot of really good work if they're good so i think it's more that like really good lawyers or junior lawyers are worth a lot more yeah and it's always been the case right yeah even more so like it's yeah it's polarizing even more so yeah i'm actually curious about that so like the whole like law business is basically like um charging per hour so you know if each project now can be done a lot faster, is it going to be like way cheaper? This was actually, so, fun story, I was reading Max's YC application and like helping him getting into YC, get Leia as it was called back then into YC. And this was my one thing, I was like, I'm curious, Leia is a Danish thing for something? No, it's Swedish. No, also it means nothing. There's this really funny story and I think it was a Spanish newspaper that wrote about us in the early days, Leia, and then they explained that the abbreviation was legally efficient Yoda assistant.

Why Yoda? Yeah, I don't know. I was like, why? See, hallucinations are not just from LLAs. No, I think journalists hallucinate. But no, it doesn't mean anything. So he wrote the application. And my feedback to him was, you know, people have tried this a million times. Legal tech has never been a thing because, like, there's no incentive for lawyers to, why would they be more efficient? They earn less money. Like, it doesn't make any sense. but I think the reason that lawyers do want to adopt this is that they understand the gravity of the situation. They know that the business model and everything is under pressure and it's like they need to change because the competitive pressure is so high. The billing model itself I think is also going to change. We're already seeing that particularly in the US that a bunch of stuff is actually being priced on a project fee and not on an hours fee. Particularly the lower end stuff on the spectrum, like whatever it is, small reviews and whatever. Some of it, honestly, is being done for free, and some of it is just being priced on a flat rate. So where is the money being made then? Is it because... In the strategic work. Yeah, I feel like a lot of what you guys talk about is M&A specifically.

Like, that's where all the money is made. That's just a very easy example. Oh, okay. Litigation is also... The big work still is there, and it's very valuable. And so they'll spend... Employers will spend more of their time there. Client relationships are also super important. Like a CRM? No, just like spending more time with the client. A lot of law firms will, or historically have basically done stuff for free just to have that relationship with the client. And so now they can do that, but they don't have to lose a lot of money. I thought I just had to pay the lawyer money just to be my friend. Isn't that the retainer thing? Yeah. You're telling me I can do it for free? Yeah, you just have to find the right lawyer. Okay, cool. Or the latest space has to grow much bigger. I don't know. Yeah, yeah, we're trying. you know so you're saying you know obviously the strategic work and all these high value ads is still there but our legal fees are not going down but you also said paralegals are not affected or like they're just shifting and this is something that we also did worry about with cognition

like people worry about the phrasing of like we will replace your junior engineer because nobody likes that kind of statement but in actual fact yes like some part of it does happen, other junior engineers embrace it, and they become more productive. Exactly like you just said. So I guess, is there a new term for this role, or this job that is having traction with your customers? You mean the super AI pilot junior legal? Nothing I've found yet. Maybe we should coin one. Isn't it a legal engineer? That's our, and also now an industry thing. Yeah. CSJDs. Exactly. Yeah, yeah, yeah. Yeah, maybe those are the people that are best positioned to do this kind of stuff. I think that the derogatory term of meet proxy, where you just sit in front of the machine and type in something that someone else tells you to type, and then you repeat what the machine tells you to say. But I do think a very good meet proxy is just like, well, you know agents very well, and you know Legora very well, and you know how to, like, this is my toolkit,

and every industry has a toolkit, and I can use my tools to do the thing that you need me to do. But I guess then you're not really a proxy, right? You're actually just doing meaningful work. Yeah. It's just like I no longer, like, you know, I'll talk about tabular review because that's a thing that happens a lot. I have hundreds, thousands, millions, thousands. We do up to a million cells, I think. In a single, that's impressive. We should talk about the table and the windowing. Yeah, I'm working on a big case. I have a million documents. I type it in. Usually I'll throw a team of paralegals locked down in the room. They'll sit there until like 4 a.m. that doesn't need to happen, right? Because now you can extract it and you can go and reprompt everything. So like, yeah, I think now I don't need a paralegal for that but like a good like super AI-pilled paralegal would do something else. You know? Yes, yeah, exactly. They can do more stuff or spend more time on the things that they, you know, didn't have time for before. Or actually sleep, who knows? I have a question. So you guys bought a company, Kira. Yes.

Is that how you say it? You guys use your own product to do the M&A? Yes, we do that So how did that work? Can you talk a little bit about that? Yeah, that works great. We're doing that a lot. Our legal team is, I mean, Intake has been built with our legal team. Because right now we have a Slack workflow where everyone flows their stuff in. They're like, this is annoying. Yeah, yeah. Right? And it's efficient. So it's like, great. We should lose the gore for that. Yeah, yeah. Can you talk a little bit about that? Like, you know, what that process looked like? Yeah, absolutely. So we do a lot of the doodillions ourselves because we can do it with Tabular Review and other tools. We do still have counsel coming in to assess certain risks. Or there's payroll. So, Zucour was simple because that was all in Sweden. But we've done one in Canada and one in Australia. So, there's payroll in Australia. Okay, do we want someone to look at this a bit more strategically? So, there we will still use LORIS for the strategic part where they can actually be sparring partners and help us a lot on how to figure out how to structure this the right way. But a lot of the more menial stuff we could actually just do ourselves. So a lot of the early things we would find,

and we have had cases where we found something that's been a red flag, and then we've pulled the brakes. Can you talk a little bit about any of this? No, I don't think so. All right. Early on, you said your early customers were Swedish, like Swedish law firms. Have you seen a difference between working with Swedish law versus American law firms, that sort of thing? And you guys are mostly working with big, big law firms, right? Well, we're also smaller law firms also. We definitely started really, really big law firms, but 50 lawyers also. Okay. And get a lot of value. Yeah, so have you seen differences between working with the Swedish ones versus American ones, maybe even, I don't know, Asian ones? Yeah, yeah, yeah. I think the European ones have been a bit slower to move. The American ones are very forward, move really quickly. I think it's also very high on the agenda of the American companies, more so than the European ones, I think. It's like a strategic, super important thing we need to just do a lot here. The American companies generally, both corporates and law firms,

also have a much higher appetite to custom-built stuff. That's a thing that they all want to do. Like, hey, we want to build this huge thing with you guys and structure it in whatever way, whereas Europeans are much more like we'll just be off the shelf. What's an example of something they would want to build themselves? integrations into their role. Some of these customers built their own CRM and their own document management system and their own all sorts of stuff, and they want to build that, and then they want to build triggers that you do a review in Salesforce and it goes to this one person for whatever reason, and they do something, and then it goes into Lagorre, and then it goes back in. And that's the simple end, and there are some that are extremely ambitious, where it's a whole new product that they want to build with us, and we can't really do that. But that's definitely a thing where American companies really want to build their own more than European ones do. And the European ones are like, hey, they give you product suggestions? Yeah, for sure. I mean, we have great collaboration with all of them. So they'll say, hey, this sucks. This is great. We want more of this. And we'll just build a product that does that.

Okay. So they don't see an existential threat here in their business, whereas the American ones do? I don't know if that's the reason that they want to custom build versus not custom build. I think they both know that this is extremely important and key to their firm's success and their corporate success. So it's both extremely important. I think it's just a different way to achieve it. Okay. Another question I think maybe you want to bring this up is product metrics. How do you know that we're actually doing something good here? How do you think about that? I'm assuming that's also related to evals. So we have the regular product metrics that we care a lot about. Are people spending a lot of time in our product? Do they seem to get a lot of value out of it? Do they renew it? Do they come back a lot? We have really good numbers for an enterprise software of our DAO over mile and our hours spent per month in the product are extremely high. And then we do online evals also, where it's like, try to figure out, generally, did we solve the problem here? Just with a super cheap LMS address,

try to just have an online metric, and then we can use that also when we do roll-ups of new things. We can just compare feature flag-wise. okay, did it improve or did it catastrophically decline? Because one thing actually that might be interesting that's difficult for our business is we can't look at any traces basically. We can't look at any data because it's all super confidential. And that's extremely frustrating to work with because you basically don't know what people are doing. So it's like here's an agent and it can do so many amazing things and I have no idea what people are actually doing. I would have assumed you guys signed some sort of agreement with the law firm and they're like, oh, yeah, sure. I wish. Yeah, I wish. No, but the thing is, like, a lot of the data that they have is not even theirs, right? It's their clients, and it's super confidential, so we can't look at it. And so, basically, we have an eval scheme that's, like, trying to approximate what we think everyone is doing and, like, build that into eval sets. So that's just, like, a super annoying kink of the business. Can you do paraphrase? So, for example, TryGVT has, like, a splash feedback thing that directly fully sets its full trajectory. But obviously, that is not possible for you guys. but you can use an LL to paraphrase

like from this real thing make a synthetic thing that is like close enough but change all the names, change all the details we can't I mean there's always a range right of like if you paraphrase all the way down to like three values and you pick one of them then obviously that's fine if you just remove the names then that's maybe not fine we are generally pick the most safe way Yeah, because we restrict ourselves as most possible because there's no risk at all. But there are things where it's like, okay, we know that the failure modes are probably these 10 or 20, so we can classify it into one of those, and that's the helpful signal. We can classify it into the types of tasks and the practice areas also, and then we have an idea. It's something that we're always trying to figure out how to do in a smart way. Also, on the privacy front, I guess, I'm assuming even two years ago, it must have been insanely difficult to get the law firms to be okay with this. How do they think about their data being sent over to Anthropic? Yeah, yeah, yeah. It must be a huge thing.

All of our infrastructure is on Azure, primarily for this reason. Because when we started, law firms were like, AWS? No. No way. Law firms don't trust AWS? Yeah, yeah. Why? It's different now. No, but they were like, Amazon, Google. We can't trust them with any of our data. So basically, we had to host everything on Microsoft because they already have 365. So they're okay with Microsoft? Exactly, because they've already done whatever security review, et cetera, of all of their documents and all of their Word and whatever. So Microsoft is an approved vendor in their mind, and so we had to host everything on Azure, and that's changed over the past few years. So now people understand, okay, we need to get on cloud, we need to do all this kind of stuff. There was also a lot more people talking about on-prem back then. That's also disappeared completely. People understand that you should just do it in the cloud, and it's safe, and it's probably safer than what you're doing. So that's changed. And by the way, it's only been three years, so it's actually like... Yeah, it's a huge shift. It's great. I mean, good for you guys. But yeah, I mean, I don't expect the legal industry has somehow ignored this for the last 10 years.

Yeah. I think we needed the AI taking the butt for everyone to understand that we need to change our ways a little bit. Okay, okay. So also then on open source models now are getting really good. So yeah, how are you thinking about that? Yeah, we're just thinking about, to your point, just bring our clients along on the Pareto frontier, and the map of all the models and what they're good at and their latency and their cost and their governance and their regional availability, how do we just optimize across all of that? So some open-weights models are great for certain tasks, and then we'll use them there. Some will post-train, and some will just use the frontier models. Yeah, you call it model arbitrage. you have different favorites, you did some research on this where you were firmly entropic at the start of the year, Grok is improving a lot you want to give an example of a model that wasn't ranked highly overall but you want to shout it out as surprisingly good? Yeah, I think not overall good, Gemini

one of the early Geminis extremely good for certain types of transcription and speaker attribution, so we use that for transcription. Obviously you have a Yeah, are you going to screen screen? Like a granola. We don't have, we actually probably will build like a meeting thing, but this is just like talk to speech, speech to text, text to speech, just like in any other app that has this. And we do actually also transcribe files. If you upload hearings or whatever, we'll then like transcribe them for the mental text and so we can do all our analysis on it. No 11 labs. No 11 labs, for GPA reasons. Same thing as like the data processing agreement, I should say. So like our data processing agreement has very few vendors on it. Yeah. because we just need to keep it super tight on where any data goes. And so I think that is one of the benefits of like a Gemini and OpenAI where you come to us, we have most of the things that you need and that is actually very valuable. Even if it's like, it doesn't have to be like, stay with the art, as long as it's like mostly good enough for all tasks,

then who cares? Yeah, absolutely. I should actually shout out also Microsoft's transcription model. My Transcribe? Yeah. Okay. It's great. Okay. It's better than the old Gemini. Yeah, yeah, yeah. They did some good results. I don't even know. I think usually all of them are top two, top three, top four in terms of the arena scores, and that's good enough. Yeah, exactly. It's like you have all the governance, and it's super scalable, and it's like, then that's probably good enough. Yeah, I was going to say, at this point, are pretty much the speech-to-text models good enough to where you're like, one word here or there, if it's off, that's good. No? No, no. For sure. I've done this a lot. Particularly in legal. There's like some, we had this thing where I think it's, I forget if it's like in Australia where there's like a certain name for like a person in court and it would consistently describe it as condom. Oh. So there, yeah, there's certain things that you don't want there. Yeah, but so usually you run an LLM layer on top of it and then you extract some keywords and then you run the transcription model again with the keywords.

Like this is kind of my workflow so far. But yeah, if you've just done this on any large scale, you will pick up tons and tons of transcription errors on any domain that is not like average day-to-day conversation. Yeah. Absolutely. If there's any sort of jargon in there. Yeah. It's going to be tough. Yeah. Like I think Whisper and 11 Labs still transcribe cloud code as cloud code. Yeah. Yeah, I've actually had that. Very, very common. Yeah. So that's how you know there's a lot more to improve. Yeah, absolutely. I do want to cover a little bit on the agent product side So we covered a few of the top level stuff right We already named job Tabular Review We named dropped Intake And you got some other stuff that we haven even touched on yet. But I just want to cover the big two, let's call it. Is that fair to say, the big two? Agent and Tabular Review? And our project. Yeah, and our Word add-in is also something, because a lot of people work

inside of Word, and so our Word add-in is a huge and Outlook add-in also. Yeah, so I just want to cover the big ones, that everyone must know. So, tabular review is basically like fan out of like a lot across each document. You can ask the same question across all documents. It's collected on the table and then work back and forth. A million is impressive. What's any other like engineering considerations or performance things? Like what's hard about this? Before we do that, can you just like walk through like what is the user experience for a lawyer? And then like you want to see a top level flow? Yeah. Yeah. So like what are they doing like with tabular review? And then we can go into that. Yeah, sure. So they, it's typically, I mean, a good example is just due diligence, right? Where it's like, we need to review all employment contracts, all supply contracts, all of whatever. And so employment contracts, you put 10,000 employment contracts in there. And then you... Like each cell is one employment contract? Each row is one employment contract. And then for cells, you ask questions, basically. So like the first question, the first, sorry, per column. So the first column might be, what's the termination period?

And then it's going to like fill out for everyone. it's 30 days or whatever it is. And the next question is like, is there an IPE assignment clause or whatever? Next one is like, how much do we have to pay them if we fire them? Or whatever it is. Like how much are they getting paid? And then it's going to fill out the entire thing. You can also just ask the agent to do this. Like if you just drop 10,000 files into the agent and say like, do a due diligence on this, it will spin up a tabular review and run all of them and wait for it to be done. And then analyze the results and it can then highlight like red flags. Okay, the CEO is getting paid an outrageous amount or there's like this weird bonus structure. okay, then obviously we need to look at that. And then the agent from that can draft a memo to send to the client, or it can just do the review and then have Laura, the user, just be like, okay, let's look at this. Here are the red flags. There's also collaborative so that you can comment on it for other people to look at, etc. And why do you choose that spreadsheet format? Yeah, it's just a lot better to – it's super easy to work in. It's really easy to get an overview super quickly of what's going on and to review it.

So all of the stuff that we produce also has citations back to source documents. And so you can basically review it super easily. If you want to just make sure that everything's right, you can also, like, highlight cells. These are reviewed or whatever. These are the ones that are flagged, and you can comment and collaborate. That's much easier than any other form factor that we've tried. Yeah. I would say that the spreadsheet form factor has a lot of pull in some areas, but somehow hasn't really spread. I find it very weird. Like Clay is another one on the GTM side that, you know, very known for the spreadsheets. Yeah. You guys, I don't really know if anyone else does a large spreadsheet AI thing. No, no. There was like V7 that like tried to launch. There's like sheets. There's even in finance domain, I don't really have like a top name that like does spreadsheets. Yeah. I agree. I think it's like a great, if you have fan out and you do the exact same thing and you're extracting stuff, it's such a great form factor to like understand what's going on. Do you find, like, you must export to Excel or, like, people are happy to live in your spreadsheet?

People are happy to live there. Do you use AG Grid or your own? Do you use AG Grid? AG Grid. That seems to be, like, the leading one. I mean, they've done this for, like, I don't know, 15 years. Yeah, yeah, yeah. They know how to. Yeah. Yeah. So, okay. Yeah. I mean, I'm happy on that spreadsheet angle. I mean, I think the only other interesting thing for me engineering-wise is the fan-outs. Yeah. Engineers maybe are well-equipped to understand the trade-offs of, like, well, okay, before you spend it sticking out on a million documents, maybe test it on like two. Yeah, yeah. Does that work before you go nuts? Yeah. Do they do that? Yeah, they will. So our types of view actually will run the first 10 rows. Yeah, this check, right? Yeah, does this make sense? Did it show us what we thought it was going to show us? Yeah. That's really good. Oh, but also for people, when people build this, it's just really good for them to iterate quickly on the first 10 rows until like they have a, they're getting the results that they want and then they can go ham. Yeah. I am happy to move on to the next thing. Intake, which is your Slack bot. Yeah. Everyone has a Slack bot. What's different about yours? So ours is just legal specific. And what happens with ours is it's not actually,

it might also, we'll probably also build one that's just an interface to the agent, but it's actually specifically to basically kick off an agent that then adds a task to our list. So we have this list thing, which is like linear, for example, that's the closest equivalent. And so it will add it to a specific intake list. And then lawyers and agents can go into the core and just be like, okay, these are the things that, you know, people are sending to us as our law firm or as our legal team in a big corporate. We can kick off agents for them. We can collaborate on them. And then we can just, like, mark them as done and then get sent back in Slack. So it's more of a – it's not just, like, kicking off an agent. It's more of a, like, I don't know, storefront thing where it's like, here's the ticket. I want this thing reviewed. Okay, now it's – here it is. It's back. No one needs to change anything. And the legal team just sits in Lagora and, like, manages the agents and everything there. I think that the data structure for this kind of thing is really interesting because it's basically the inbox for all these kinds of queries that can potentially translate into a thing, into something else. And I don't know. Is it hard to integrate all these with all the other features,

or do you just not worry about it? No, it's not. I think we have some primitives that work really well. So Lists is a great primitive because it's essentially extremely flexible. It's like a Notion database. Yeah, a Notion database. It's like a Notion database, and so you can basically configure it for whatever you want. And we, of course, invest a lot into making this super scalable and work really well. And you can have triggers on them, automations on them, and it's like a great glue to glue everything together because then it can kick off an agent run based on a list if something happens. And then that comes back and populates a field or whatever as new documents or whatever it is. And so it's like just like you might coordinate a bunch of coding agents with linear or some sort of task management tool, we can do it similarly with lists. And so agents can create lists and lists can create agents and agents can read lists. and so it's like the agent swarm comes alive. Yeah. Do people get excited or overwhelmed by that? You know, like, I feel like this is basically end-user programming. You're, you know, you can let people set up crazy automations. Some people get super obsessed by it and they become, like, influenced. Like, you know, it'd be cool to have, like, more influencers.

Yeah, right, they have influencers. Is that worth it? Too overcomplicated? We need the flexibility, even if we don't actually expose it to the end-user. Let's say this was just all internal. then it's a really fast way for us to utilize people like our legal engineers to basically build a lot of products extremely fast. So, like, you can build custom, like, fund management dashboards. You can basically build that on list with, like, your own generative UI, and then you basically have a new product that some people are going to love, and we can have a marketplace of just, like, a bunch of specialized products that we can scale super quick because all of our legal engineers can just build new ones. If they're out with a client, oh, you want your own? Easy. Like, we'll just fix one for you. So it's, like, a lovable inside of the war a little bit. And if we don't want to expose it, that's also how we build our own custom features. So we made another acquisition in the UK called Wechsler, and they're doing fact intelligence, which is for litigation mostly, which is like you upload a million documents for a lawsuit or whatever. And then we extract facts from those with named entities and dates and whatever.

And then when you have all the facts, you can basically hold them up against each other and see which ones are corroborated and which ones are, what's it called, disagreement. and that's extremely powerful because then you can also transcribe a court hearing and you'd be like, oh, he said this thing but actually the email over here says that he was not there and if you see it live, it's like so fucking cool and that we can also build on this primitive and so we can build a lot of really cool things on this one primitive, basically that's how we think about it because we have to be, as we scale the company we just have to solve so many things so I think a lot about how do we build a core primitive that we can scale to many different end products and create very customizable experiences. Okay, so what core parameters do you have? Let's just restate for people. Lists is a core one. Yeah, super configurable. Yeah, projects, basically containers of files, file systems. The agent, of course, you can take that off and run that in very different configurations. Do you have a name for the agent, by the way? No, we don't. No, you mean like you called it Leia or like we gave it a name?

No, yeah, Leia. Yeah, Leia would be cool. People used to talk about Leia as a named entity. But yeah, we stopped that. We might as well bring it back. It might be confusing. So list, agent, tabular review, which we're considering is that actually an instantiation. You can take this as far down as you want, right? But those are probably the big ones, yeah. Yeah, so I actually wanted to talk a little bit about FTEs because this sounds kind of like that. And I think Max has talked about that a little bit. that, like, okay, when you onboard a new, like, big customer, you want to get them value, like, ASAP so that, you know, they stay on Agora. Yeah. Yeah, how has that evolved? Like, when did you guys start that forward-to-play team? So we hired our first forward-to-play engineer. That was our first time in New York, also YC founder, nine months ago maybe. I'm not actually sure. Are they considered Google engineers or no? No. Okay. They're engineers. But we do, it is something that I think about is, like, how do we exactly enable that at scale? and I think a really core thing is like how can we empower our legal engineers,

like how can we empower our power users and our legal engineers to customize the product in a powerful way and then of course we will still have forward-up engineers like help out, you know, do all the custom integrations and the scoping and figuring out what's possible and feed all of that back into core product development. So that's kind of how I think about it. It's like almost a first-line, second-line defense where it's like legal engineers should be able to customize a lot of stuff and build a great experience and then where they can't or we need to make core product changes or custom integrations or whatever, we have forward-deployed engineers who also then cycle in a little bit behind, like, sometimes they'll just work on the product and sometimes they'll work with clients. Can you walk through, like, an example? Like, okay, so you just got this new client. Like, what's step number one? Like, are you guys like, okay, hey, we have a forward-deployed engineer for you guys. They're going to go build, you know, some kind of product for you. Like, how does that work? So it's very different. Find a client, right? But if it's a big client and they're super engaged and they, like, want, they have all these streams of things that they would want to do or they already have, typically they have already tried to build something and it's failed, and they're like, this is a problem that we'd like to solve,

then we'll basically staff them with legal engineers and a forward-deployed engineer potentially, depending on what their thing is, and then we'll just be like, hey, these guys will come, they'll make you successful, they will sit with all your lawyers and teach them how to use it and figure out all the workflows that you have right now and build them into Legora, and some of that will just be a legal engineer, and some of that will require a forward-deployed engineer. What is that separation, I guess, if you could talk about that a little bit? Legal engineers configure the product as a power user would. They don't write code. Yeah, yeah. And a forward deployed engineer writes code into production, basically. I guess that's the... What kind of background do the forward deployed engineers normally have? You said YC founders. Yeah, founders, regular engineers. You're old. Yeah, D was your... Yeah, exactly. D was our first. Two YC companies and worked on search at Meta. Yeah, she's awesome. Yeah, yeah. Okay, that's a high bar. She's great. Yeah, yeah, yeah. Yeah, first boost on the ground in the U.S. Yeah. So what are some examples of, I don't know, workflows or things that these people are building for the clients? Do you guys have, like, oh, these are, like, you know, there's a couple use cases that, like, all the law firms are, like, you know, this is the first thing that RFDs are going to do when they go in?

It's very different. some core workflows everyone have and we can just build in Lagor and that could be anything from like this is how we handle these types of NDAs or whatever and you activate it and it's more like skill building really for at least and most of the AFDs that's more like there's something that we can't do yet like we need some more flexibility in some of the primitives or whatever it might be or we just have this really complex system that we need to integrate into and then they'll go out there and they'll build the integration against our APIs and maybe expand or APIs or whatever to make that work. And how does the, like, legal structure work there? Because you said you guys yourselves can't even work on deals. Yes. Yeah. So, like, how do they work on client projects? You mean the law firms, how do they work on client projects? How do your FDs work on client projects? So they don't touch the data still. So they'll just build the system. Okay. But they will also be onboarded like an employee sometimes. So, like, they might get a laptop and everything. Like, do they often? Yeah, yeah. Oh, wow. Interesting. That doesn't happen very often because, like, usually they just build an integration and, like, kind of works.

But, yeah, they can if they need to. And you only onboard FTEs and do a customer, I'm assuming, if they're a certain size? Yeah. Okay. It needs to make sense. Because RFTEs can also work on core product, right? So it's, like, a constant prioritization thing. Is there a rule, like, I guess, does every, like, you know, so Palantir famously had, like, you know, four days you spend on site, one day you come back and contribute back to products, anything like that? No, we don't. I mean, to be fully honest with you guys, it's still early days in a lot of this. Are you waiting to see it added maybe over the next year? I do think that it's going to be more, maybe not exactly like Palantir, but I think we will have a larger in constant circulation group of FDs that will babysit with a client for like two months or whatever to build out this one thing, and then they go to the next one, et cetera, et cetera. Yeah, but I do imagine us rolling in with legal engineers and forward-to-plot engineers to work on specific projects. Yeah. The API business is actually interesting, like you said, because that's another rule that you can have

where you will be CTO maintainer of the Logara platform. Yeah. Right? We will offer everything that we have in the Logara web app as an API, and then you guys can use whatever. Is that a promise you make? That's not something we're promising now. Yeah. We do have APIs that you can build against, but not the entire surface of the product. That's the curious thing. Is that the strategy? and obviously some people will love it because I don't have to learn a new tool. I can just use the existing tool I have and just install it for example with the Word document stuff. Other than that, I do think then you have to maintain it. Exactly. So that's been my thing. That's why we've tried not to do it. We haven't had APIs. I mean, we started our APIs like six months ago or something. We've folded for a very long time mostly because I just don't want to... We move so quickly in the product that I don't want to be like slowed down by having to maintain old API versions for like a data model that we've changed. I mean, now we're bigger and things are different. So now it's a cost that we can take on.

But like when we're 20 engineers, it's just like we move so quickly that we can't do this. I have a point with this, right? Like it's just generally good for all companies to have a platform team and a product team because the product team, you have one, but then you can easily spin up a second one as long as the platform is not super tight to product. Yeah. Yeah. You know, like a very simple one is web app, mobile app. Yeah. or, like, WebMobileApp, MTP, and then, like, you know, whatever, like, other integrations. Then your FTEs just are one more client to the overall platform, basically. And I think, like, even the agent, you can think about it this way. There's, like, a harness team that runs the agent, and you can even build a product around it with, like, scales and automations and whatever else. Yeah, this is, like, very famously the Amazon Web Services mandate. Like, Dezos was one of the things in the past. No more backdoor access. you only offer other teams your APIs the same APIs you offer customers. If you don't have it, you build it. And then the customers also get it. I think that's also changing a little bit. A lot of APIs now,

one thing that we're toying with is can the agent be the API? I know it sounds weird, but for some of this stuff what they actually want to do is they want to trigger certain things to happen. For some part of this you could just be, I'm not saying that we're doing this but there's a thing that I'm toying with in my head is could that just be like, okay, you trigger the agent and ask it to do this thing. Like, obviously, it's way less efficient than if you just had, like, an API that you always hit, and it's like, you know, but it's something that I'm toying with, it's like how much of our product can just be the agent versus all the other stuff. Well, you know, unbounded execution and cost and latency, is that the only whole bet? Yeah, and I think deterministic, like, maybe, you know, 100 out of, or in two out of 100 times, it's going to do something whack that we didn't want it to do. We're going to go into, like, harness and model fine tuning in a bit, But I think there's a product question about restraint, about what you don't do, because you could do all that. And obviously, there's a prioritization issue. Are you more on the point of let say no to most things and then only do a short number of stuff Or no actually we want to do more things and we just need to hire like 100 people I was very strongly

the first thing for a very long time let's just say no to everything like keep the industry I think I've said this before but like one of our presentations after I've been at Ligora for half a year I pulled up this slide of you know the 300 Spartans and for the movie and I was like we're just going to be 20 engineers for the rest of the company's history and we're going to beat everyone this much bigger. We're 100 engineers now and it feels like we're way too small. I wanted to do that, but I think in the real world, we're in fierce competition. People want so much stuff. It's like I could be a Puritan and just be like, no, we have to stay 20 people and we can only build whatever, but then we just lose and people would go elsewhere. So we have to build more. And so I think my job is to figure out how do I build a system where we can build a lot, but actually only – the customer feels like we've built so much stuff, but we've actually only built a few things. that's that's how i think about my job right now yeah i mean primitives is right you know yeah exactly i think that's gonna be that's really powerful do you testing your products you're seeing if like hey there's like any pull here and then you just kind of like you're like no it's just focused on our main thing we haven't made i mean we've made certain strategic bets

yeah but we've not tried anything that we don't have a lot of confidence in already mostly because we just have like a mountain of demand yeah it's like we want all of these things and so it doesn't It hasn't really made sense for us to try to be like, hey, maybe this thing could work because there's so many people screaming for the other things. And so we mostly build those. And then sometimes we'll disagree with clients and we'll be like, actually, I think this thing over here will make a lot of sense strategically and then we'll try to build that anyway. Like strategically for you? Yeah, exactly. So what are those things? So one of those things was the agent actually. So no one was actually asking for it. I want the assistant that I'm using to have more agency and do things without me asking it. No one ever, but we're just like, this is definitely why the industry is going. This is going to empower you to do so much cool stuff. Collaboration was another one where we think the industry is going to a place where you can collaborate inside of where you're working, which is Lagora and not in emails and text messages. So yeah, there are a few of those. And then once you say, okay, yeah, we want you guys to use agents, I'm assuming like internally at the law firm, you know, it's going to take a lot for them to actually learn how to do that.

And so that's where you kind of like say, hey, FDs, go in there and teach them what they do. Yeah, legal engineers mostly. Oh, legal engineers. Yeah. But they're like, some people call them forward deployed legal engineers if you want, but that's essentially what they are. Then they go in, yeah, and help them structure their workflows around the capabilities of the agent. You guys said previously, I think Max said this, but that early on specifically, you guys weren't fine-tuning your own models. Is that still the case? Yeah, so we haven't fine-tuned any models up until now. I think now in the past few months, open-weights models have become extremely capable, and so we will do it for certain tasks where we think there's a huge delta between what a frontier model can do at the cost and latency versus what we think a post-trained model can do. Legal research is a great example. It's a fan-out retrieval problem, and you can make a really effective model to those queries super quickly and super cheaply at low latency. So specifically, you're post-training them to be better at... Very specific, bounded tasks.

Yeah, exactly. Okay. We're not trying to train our own fable to be the main agent. I don't think that's like a... We won't win that versus Anthropic, most likely. There's still a lot of, let's say, search and retrieval research or RAG research to do that is maybe less glamorous. It's not actually even model training sometimes. It's just architecture and pipelines. You did a case study with TurboProfer, but there's also, do you do late interaction? Yes, absolutely. You do that. We actually don't do that, but you're completely right. So there's one guy on the team, who really cares about search, and he basically comes to me all the time. He's like, we should do these things, and there's these new embedding models, and we can do this late interaction. He actually talked to TurboProfer teams to get them to build multi-vector so that we can do these types of things. But no, there's loads of low-hanging fruit generally, even before models. And even in the harness, honestly, if you just want to create a better output for our clients, there's so much stuff that we can do that has immediate impact before training models.

Like what makes a good harness for the law industry, I guess? You guys are making, I'm assuming, your own custom harness. Yeah, of course. So yeah. The way that we edit files is important. The way that we do an analysis is important. The way you do legal research is all baked into it. So it's like you might have a problem that comes up or sees something, then it knows how to go and do the legal research to figure out what the answer is, what the precedent is. That's all very important. And then, of course, there's all the governance stuff around it as well. Yeah, it seems like the search is probably, like, would you say the most important tool that you guys are, like, customizing? Because if you can't pull in the right information, then you're just going to screw it. Yeah, I think it's search and analysis. Yeah, it's search, it's analysis, and it's drafting, probably. because like drafting huge contracts is actually also not super trivial. Like what about it is not trivial? Because you have to, let's say you're drafting a huge contract from scratch, then you have to find precedence of like, how did we do this previously? Do we need this type of clause or not? A specific law firm, how did they do it previously?

Yeah, so I am the law firm and I'm about to draft a huge, whatever, NGA is very simple but like there are contracts that are way more complex and then you need to find the right precedent first because like lawyers just don't like sit and like start typing stuff. They figure out what clauses have we used in similar deals, and similar can mean a lot of different things. I'm sure you know in search. It could be, like, same industry, same type of company, same whatever. And so you need to identify the right person, and you might have, like, multiple documents that you then try to figure out which part should I take from what document, and that's actually non-trivial. So it's essentially also a search problem. Which model is best at drafting? So I'll say, I told us about this yesterday, definitely Fable and Office are extremely chatty. I'm sure you guys have known this. They're very verbose. That's not good. That's generally not good. I noticed Astra is pretty good at writing. Astra is pretty good at writing. Is it promptable? Are you finding like 100 different prompts to be sure? Yeah, we can do certain things, but I think the chattiness, verbosity is actually a little bit difficult for us to do.

So this is, I think, you know, let's see, but probably the next model from Anthropoc is going to be much better at this. Astra is actually very concise. Yeah, yeah. And I assume they all have a Slack channel with you and you're telling them this, right? Yeah, exactly. This is not news to them. No, no, no, they know this. Then the other thing that I really care about is hallucination. Do they make stuff up? So we have a system of, a citation system where we basically check what they're writing. Has a link, so the user has a link. They can click and it highlights with a bounding box, like in the PDF or whatever, where this information was taken from. And we can sort of make sure that what it says always is citable somewhere, even multiple layers down. So, like, if the agent writes something based on a tabular review, you know, it can cite the cell in the tabular review or the row in the tabular review. So, you can actually, like, click multiple layers down and find, you know, sort of validate it all the way up. Yeah. Are you finding that lawyers are, like, verifying that these citations are correct or not? Initially, it's kind of like the argument. But now it's the exact same thing. Are you actually doing it? People ship, like, hallucinated links to souping court. They don't care.

That happens. Yeah. Exactly. That happens. Yeah. Yeah. I think it's like coding. You know, you have engineers that just don't give a shit and just, like, send agents, and you have people that, like, sit and read every comma. There's just, like, so much that, like, how can one human do that, right? Yeah, but I think they probably, just, like, we know in code review what to look for. Yeah. What is the system or whatever. I think they also know, okay, this is the thing. These are the red, like, this is the thing that's important. Let me just, like, double check these things, and, like, these sound reasonable, so it's probably fine. Okay. Interesting. Just, like, if you receive something from a junior lawyer, it's the same thing you do, right? Just, like, I should double check this, and, like, this probably is fine. Any other elements of, let's call it legal reasoning, or what's the legal frontier that models are not good enough yet that you want from the next generation of models? Nothing I can think of specifically right now. I think long document reasoning is still difficult. Long, like, million token contracts? Yeah, exactly. You have a huge contract. And, like, you're making edits across it. It's a difficult thing to do for a model today. And, like, remember that you changed all these things up here and, like, the definitions changed and the ordering changed

and whatever else. So the simplest would be one shot. You feed everything in and then one shot output. Yes. You don't do that. You break it up into chunks and then you orchestrate it. Yeah, exactly. Exactly. And this is like one of those things that are funny, that are fun to be part of in this industry, right? So our word add-in was like, we've done map reduce and we've changed and there was one big one and there was like one big one in multiple sizes. Like we keep changing the scaffolding that we build around as models change. Yes. I would say I do a lot of news reporting, obviously, because of what I do. And I've recently shifted from MapReduce to throw everything in context. It's better. Yeah, exactly. But it's also different. I mean, we have this problem with search where it's like you're searching for certain things in a huge contract, and you can MapReduce it. But then if you're summarizing any sort of thing, then it might lose some important nuance in between the chunks. In the sense that you just summarize the individual parts up. And so this has been a problem where it's like we can do two types of search. We can also do the other one where it's like you need to have reasoning in the different steps there in between.

So I'm trying to think of a really good example here. Yeah, there's some documents where they talk about something at length over like let's say two pages or whatever. And then each individual is not necessarily relevant to the search term or what they're trying to figure out. But together it's extremely relevant. And so, yeah, these are some of the problems where it's not hard. And it has to be hard, like you said earlier, to actually make the models better at that because you can't even look at the contracts, right? It's the real contracts. So do you guys ever ask them for like, hey, can you make example contracts that are similar to the real one? Yeah, yeah. So we have feedback and we can receive feedback from them. And if they're willing to the age, yeah, exactly. Or they're just like, hey, this is like super important to us and it's not working well. And then we're like, great, can you like anonymize the documents and like the problem or whatever? And if we can recreate it, then we can fix it. Yeah. I mean, so one version of this is you buy a law firm. Yeah. Which some people are considering. Yes. But there's some stuff there on whether their data is actually their data or if it's the client's data. So if the clients are all gone, then maybe it's fine.

I actually don't know. Interesting. Yeah. But it's a way to get a lot of data. Yeah. People like Google's buying Spirit Airlines. Exactly. Right? What? Yeah. Yeah. They bought the, yeah, for like $10 million. Oh, no. Yeah. Yeah, yeah, yeah. Like, you know, if you have a failed company, you still have your Slack, you can sell your Slack. I think there's, like, a YC company that, like, buys your failed company. Yes, there is. Yeah, yeah, yeah. Yeah. Because, like, it's just like, oh, I just want, like, real people doing work. Exactly. And then, yeah, so, like, pull stuff out. Exactly. The problem with buying law firm data is, well, you get all the inputs, the real inputs, which is great. The outputs are a little bit more difficult because you just get what their end result was, and it's difficult to figure out if you have to basically reverse engineer how they got there. So in our product, when we get real traces from real users, it's extremely high value. But if we just got this output and we don't know if it's good or bad or how it was created, it's a little bit difficult. We're going to wrap up soon. The last thing I'll point people to is also, there's a lot of offer less on embeddings. If you're a rag-heavy workload, you're just looking at MTB

and you're taking top model. You're leaving a lot on the table. You should look at Voyage or some of the custom embeddings you've come to this. So basically, don't take default embeddings. Even OpenAI embeddings, OpenAI embeddings, it doesn't matter. Customize your embeddings. You will support that. Yes, I think that's right. I think that's right. The only thing is you need to make sure that you're multi-language. So for us, we serve so many languages. I mean, Europe is the king of this. Yeah, exactly. Mr. Are you helpful there? No. Until now, they just raised a ton of money. right? So maybe, yeah, let's be, they also raised a ton of money a year ago. I don't know. I think we need way more model companies out of Europe. Yes. I just wanted to ask you, like, what predictions do you have maybe, like, over the next year for, like, the law industry? We asked a little bit of that earlier. Yeah. I'll modify it to your second question, which is, in two years, will lawyers do their work inside Lagora? Will they use other tools like Quad or Chagency

and call Lagora? Like, you know, the API. Yeah, yeah, yeah, yeah. I think both. It really depends. I think if you're a big enterprise, you might have Claude Cowork, and then you'll, I think it's similar to, like, how I use linear through Claude, but I still go to linear. I think enterprise will do that. I think law first will just use the bar, because that's simpler and better. Yeah. It kind of reminds me of, like, open evidence for, like, medicine. Like, you're basically that for law. Yeah. Yeah, exactly. Encoding is going to a point where people, the labs will cut you off for competing with them, and this has happened twice. Do you think that will happen to law? No, I don't think so. Because they are releasing legal services. Yeah, they are. I mean, we obviously talk to them a lot, but it's also like, hey, are you going to... Your salespeople will say no, but they don't make the call. Exactly, that's not their... And they don't even know often what's actually going on. I don't think that's the case. They have so much stuff to capture in knowledge work, or in sort of enterprises, knowledge work generally, that legal has to be part of that, just like data analytics has to,

and sales and marketing and finance. So, like, they have to do things that are good there, but they're never going to go as deep as we are. And I think if Claude takes a small client, small, you know, corporate clients and they do some of their legal via Claude, like, that's great. That's fine. But if you're doing a, you know, $400 million M&A transaction, you're going to want Lagora. I had kind of an unrelated question, but I'm curious, are you guys going to law schools and kind of, like, getting them to, like, teach Lagora, like, in classes? Like, how does that work? Yeah, it's definitely a program that we are, I think we've started. I actually don't know exactly where we are, but we do want people in law school to learn to use Libora. Are you saying that law schools are changing their curriculums a lot or not really? I don't know. I wouldn't know, actually. Interesting. I would actually do it a different way. You should do scholarships for law school for CS majors, for engineers. After you've done CS, we'll put you in law school. CS plus some software experience, we'll put you in law school. they'll cover it.

Or you do like the MA, like Masters plus JD programs? Yeah, you don't care about the Masters. Meaning like, because there's a lot of people that didn't do CS undergrad, so then they do like Masters just to kind of like get set up. Yeah, yeah. I don't know, I don't see that as often. People do JD, MBA, that's the thing. Yeah. But JD, CS, I haven't seen it. Happens. Yeah. Yeah, okay, look, overall, you know, super impressive progress. I mean, Congrats on all your success. Are you still going to keep up the pace? Absolutely. Yeah, you know, I think you're hiring in SF, New York. You're hiring in engineering. We're hiring in New York, Stockholm, and London. Yeah. And all other roles we're hiring everywhere. How many people do you have in New York right now? A hundred. Okay. And you guys are planning on how many in New York? In New York? Yeah. Just like generally? Let's just say, yeah, generally engineering. A lot. As many as you can. Yeah. it's aggressive yeah

I mean that's the home of corporate law right as far as like American so like yeah I'm looking forward to you buying a law firm and you know becoming yeah it's super exciting to I think like you know every industry especially like the top like doctors lawyers coders you know they all have to have like you know two or three agent labs that like just dominate the space I agree so you guys have like a great market yeah I think so and I think is only going to get much bigger as, like, law changes and becomes more available to more people. Yeah. Also, I think, like, we're already, like, super over, but, like, the problems that you work on are very, like, the frontier of knowledge work. Yes. Yeah. Okay, like, unless you're literally, like, doing, like, science, and science, other than that, like, most work is, like, basically, converges to legal. Yeah. Like, yeah, yeah, It sells legal in the end. Yeah. Hiring, finance, everything. I just have to consult my lawyer. Yeah, I know.

I know. Which is why I think it's a really exciting thing to work on, because modern society runs in some way on legal. Yeah. Yeah. Cool. All right. That's it. Thank you. Thank you. Thanks. Thanks.