Airbnb が AI エージェント向けに「会社の頭脳」を構築――CTO Ahmad Al-Dahle が語る Project Everest

Your company knows a lot more than what's written down. Experienced employees know why a system works a certain way, which decisions apply, and how to ask when the documentation falls short. Giving AI agents access to that knowledge is a much harder problem than just connecting them to a code base. Ahmed Aldale helped build the original iPhone at Apple and led the team behind Llama at Meta. Now, as Airbnb's CTO, he's tackling that problem by building what he calls a company brain. The system, Everest, captures institutional knowledge from employees' workflows and makes it available to agents. But as that shared knowledge grows, how do you keep agents from learning the wrong lessons? I think if you aren't careful, markdown files could be exactly the kind of problem that I was referring to, where you have, like, it's difficult to find and there's a lot of, like, context poisoning that's happening. One person uses it, we extract a thing, we put it down a markdown file, and then the model's very good at finding the thing that's old or wrong.

Because the biggest mistake you can make is you reveal too much context too quickly to the model versus you try to give it enough context to move one step forward to then ask for more context. I'm Sam Sherrington, and this is the TwiML AI Podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in. When you arrived at Airbnb, talk a little bit about the AI landscape that you encountered and how you've seen that evolve since you joined. So the good news is there was a lot of momentum already. There was a pretty decent foundation, I would say. And I think it's both the timing that I joined, because really, magenta coding had just begun to become something that people could really leverage.

So the moment and timing of the capability was right. But also, I think the Airbnb culture, and Airbnb's had a long lineage of working on AI, and enderops and all sorts of things. So it had a strong sort of infrastructure investment. So what I've been able to help the company do is just really accelerate the adoption of AI across like maybe three frontiers or three boundaries. The first one is like AI for productivity. How do we how does the average engineer use AI to move faster? That turns out to be like you have to sort of engineer your platform to make it easy to facilitate AI agents in your infrastructure. And then the second one is, you know, how do we deploy AI agents to the marketplace? These are things that maybe the user doesn't see that really add a lot of accretive value to our platform. It either builds more trust, more reliability, quality, etc. These are things that you do in an offline fashion. And then how do we deploy AI agents to improve both the guest experience or the host experience?

So these are kind of like the three frontiers we pushed on. and each one's made a tremendous amount of progress in a really short amount of time. When you think about that first pillar in AI for engineering productivity, you mentioned you kind of came on just as agentic coding was taking place. When you think about how to help engineers be productive today with AI coding tools, there are lots of different ways that you could come about that. I see lots of different philosophies about AI coding in the marketplace today. I'm not sure we've fully nailed this is the right way to do it or the one way to do it. How would you describe the Airbnb way to do it? Or even is there an Airbnb way to do it or an Ahmed way to do it or is it team by team? Well, I mean, every company has a culture or set of principles that we're a bit curious to. So some companies will say, like, move fast. That's kind of the mantra.

So what you'll find is the way that AI is being deployed for activity is really about velocity. I think Airbnb really focuses on craft, which I think is actually really, really good. It's aligned to how I think about things. And so because we're very focused on craft, it's not just a velocity or speed question. Velocity actually has a heading, which is good. It's really also about quality. So we kind of have to balance the equities of both. And so when we look at productivity, we don't just look at, you know, PR throughputs or token rates or things like that. We also look at defect ratios, reliability metrics. So we try to guardrail the velocity with craft and judgment to make sure that we are sort of continuing to stay on track. That way, when you get exponential use of AI in your system, you're also pretty much reinforcing a culture that you kind of need to maintain the quality of your system as well.

So I think that's one of the very high-level things that stays true everywhere. But I think there's some common truths in how to deploy AI agents for your own productivity that are consistent at every company. So how do you basically MCP your entire company so that an agent can access what it needs based on any generic task? How do you construct context in your code base or documentation within your code base so that agents are able to specialize on the work that they need to do? So if I am a product engineer and I need to like work on a PR that lands in the payment stack, which is like complicated and regulated and like kind of gnarly, is there enough documentation to inform the agent so that your PR is actually kind of reasonable for the payments routine to review? These are all things that you kind of need to build out in order to get like velocity at the end, like to actually get more productive and also maintain some level of quality. And then the last one is like institutional knowledge, which isn't embedded in your organization,

really figuring out how to extract institutional knowledge and code it so that agents can learn from it. This is like underexplored. It's something that we've been working on. So we have something that we call Project Everest, which is effectively kind of like the company brain. So as people use AI and as they use Everest, it's able to sort of collect a lot of institutional knowledge from everybody using it. So we have a shared context pool of how people do things that maybe isn't in the documentation. What's the organizing principle there? Is it primarily for engineering and engineers? Is it workflow-based? Is it very freeform? No, it's for engineers. It started, actually, let me be clear. It started for product engineers, but it is now dispersing across the company. So we even have product managers, which we call actually product marketers at Airbnb, that are actually also using it. eventually design will use it and we'll kind of like bring this as like the company context graph that is like more about everybody's workflows and learning from everyone's workflows.

Interesting. That makes me think about, and many moons ago, I worked in kind of the knowledge management field and like knowledge management and collaboration are these perpetual challenges within large enterprises. People create a lot of documents, but there's a lot of implicit knowledge that never gets written down. And certainly AI and agents change all that because the agents can do some of that documenting for us and they can pull it from places where the work happens as opposed to being told to do it per se. But at the same time, At the same time, the agents can create a lot of documentation, and too much documentation or documentation about the wrong things kind of reduces the value of the city. It poisons the well. Yeah. So talk a little bit about how you've kind of managed that challenge and how Everest approaches kind of curation or prioritization, however you think about that.

I mean, ultimately, the best way to keep your system on track is to have a really strong evals. So one of the things that we do with Everest is we have like consistent tasks that we look at doing. Are we trying to benchmark those on, you know, if you use Everest, are you able to complete the task in a shorter number and is it more accurate? So is Everest adding value? What's an example of one of those tasks? Take what a product engineer would do in our pricing domain. If they need to land a PR in our pricing domain with a prompt, and they're like, you know, apply a new pricing rule change to the pricing domain, is it able to complete that task given that pricing engineers have been working with Everest, giving it context, and doing their work with it? is it able to land a PR that is faster and more accurate, as an example.

So it's complicated because there's a lot of policies and a lot of very specific ways of working with pricing. It's probably one of our most complicated domains. So is Everest able to let people work in the pricing domain, encoding a lot of the institutional know-how of how to work with something that's as complicated as pricing, as an example. So here in that particular domain, you've got a lot of existing code that illustrates how the pricing aspects of your system work. But the code isn't everything. There's also regulation and policy and other things. And the idea is to make that available to agents as well. That's right. I mean, we want to get there. I think it's early. But I think this is, to be clear, it's not that Everest has completely solved everything for us. but I think it is the beginnings of something that could eventually become, you know, empowering to every engineer who isn't an expert in everything to basically move faster. And do you think of that as a database of some sort or a collection of markdown files of some sort or something else?

It's a database of sorts, yeah. There's a question of what's the right way to structure it. I think markdown files are really good because frontier models have been trained to navigate file systems. So actually you can get pretty far. There's a question of like, what's the best structure for something like memory? I mean, this is kind of what it is. It's a little bit of a memory structure. And, you know, there's a question. So we're experimenting with the right data structure. If it's a database, it is a better approach. Or if actually a markdown file with like a harness kind of deciding what the directory structure needs to be is the right approach. So I don't think we've like, I don't, I'm not convicted that we have the right architecture for it yet. Do you think of it as primarily collecting the context and letting the agent figure out what to do with the context? Or do you believe that kind of extracting some degree of structure and making that structure available to the agents is something that is a promising direction? And I guess I'm wondering, you know, you're kind of creating vector embeddings on it and using embeddings to allow the agents to find context.

Are you doing like graph kinds of things to allow the agents to navigate this? Or is it, you know, here's a bunch of files or a way to like list a bunch of files in an area in a database. And, you know, you think the agents as they get more and more powerful will be able to figure out what they need to do with it. The short answer on this, I think, is I think we're going to end up looking at all of these to figure out which one works best. I find it hard to believe a vector search is going to scale to where we need it to work, where the whole company is using it. I think if you aren't careful, markdown files could be exactly the kind of problem that I was referring to, where it's difficult to find and there's a lot of context poisoning that's happening. One person uses it, we extract a thing, we put it down a markdown file, and then the model very good at finding the thing that old or wrong So I think we going to end up finding some hybrid where actually there some intelligence in between that is figuring out how to search

How to do progressive disclosure of context. Because the biggest mistake you can make is you reveal too much context too quickly to the model. Versus you try to give it enough context to move one step forward to then ask for more context. Kind of like a progressive disclosure approach. And I think that is still kind of open. In terms of the kind of tech stack or platforms that you use for agentic coding, like have you talked about what that is? Like are you an X shop or a Y shop or does everybody use their own thing? We started as one shop at some point, but we've quickly learned, like given the pace of the industry, it's a little crazy to be an X shop. So we've, at least my strong inclination was like, we need to be a multi-model shop. So actually, if you look across the company, you know, for internal productivity, we have something called AirChat,

which basically wraps other harnesses with our MCPs, but we're able to sort of select which one we want to use. And it's kind of nice because Codex is also open source. It's nice of open source stuff. So generally, we're pretty open and flexible with engineer preference. People can kind of pick which adjetic tools they want to use. And then for product, we actually also have our own set of harnesses that we deploy. And we also use a lot of open source stuff that has our own modifications to it to make it work for our systems. What are you doing around evals? That's an area that a lot of companies really struggle with to ensure that they're able to kind of get that data flywheel going. One of the systems that I mentioned earlier, Zipline, was feature engineering. and feature engineering plays a similar kind of data flywheel role in the MLOps platform as eval tooling does today?

How is the way Airbnb thinks about eval tooling and kind of closing the loop from interactions to kind of intelligence, how has that evolved? For agentic tasks, we've worked with a set of engineers. We have a council of senior engineers, and we have a set of tasks or prompts that people want to complete. The difficulty is the challenge with the evals. You could argue there's a very binary outcome, like did it build the thing or didn't it? But there's also a lot of subtleness to the correctness of the solution and the quality of the solution. and so we have built agentic evals that look at Airbnb engineering tasks. I can't think of stuff off the top of my head but there are a set of tasks that we've worked across various teams to engineer and then we run those evals against

all of the models, both open source and closed source, to have a sense of the Pareto between the cost of the model and the quality and the performance. And the hardest part is really trying to push the Pareto down. So you're trying to not saturate because everything clusters around 92%, 93% or something like that, which a virtual judge is giving you the score. So we calibrated a virtual judge to basically say, given this task, given the execution, what's the quality? So we get a percentage. And so we have this chart that we use that basically shows cost versus accuracy. And then we look at all of the open source models and we decide. It sounds like without a lot of care, what you're seeing is that the model will generally say, oh, that's about 90% good. Yeah, yeah. That's the hard part. Exactly. And getting enough separation between them is like sometimes a chore,

which actually either says one of two things. either your evals aren't good or the models are actually probably good enough for a lot of stuff that we're doing. And then it becomes like a cost and efficiency tradeoff and a speed tradeoff, really. But I think most of the time I'm very suspicious that evals, if we're not able to get some separation between frontier and non-frontier. Are these hard enough? Are we really calibrating the judge to look for enough signal in the implementation or the completion? Let's dig into the model cost versus capability a bit deeper. There have been some famous examples in Uber, for example, told all their engineers to go to town on agentic coding, and then some period of time later was like, whoa, you guys really went to town on agentic coding. This is expensive. How do you think about cost versus value versus, for obvious reasons, the topic of routing has gotten very popular nowadays, which is essentially how do we get the right tasks or the right model?

What have you done to try to optimize there? I fundamentally think that engineers are problem solvers and you just have to inspire the objective, not constrain the execution. So things that I adhere to is we didn't really focus on token leaderboards. The only thing that tokens tell me is if someone's not using any tokens, that is probably something to work on. and if someone's using a lot of tokens, we should try to understand what they're doing. And either, by the way, we found like two cases. One case where people needed help to better understand how to build efficiency into their workflows and others where actually they're doing really great migrations and it's actually really valuable and like, go hand. And so I like didn't apply a draconian token limit to everybody. I didn't really say token max. We didn't do any of those, fall into any of those pitholes.

What we did was try to apply nuance to understanding the shape of adoption. My theory on the right way to approach it is people who aren't using it need help and need support. People who are using a lot of it, we need to understand. And you look for other signals. People who are using one model pinned. Imagine you're using, name your favorite, frontier, maximum thinking. Yeah. And it's like it's like these large, you know, 10,000 tool calls and every sub agent is like your frontier model. And like at some point, you're like, hey, that's slow. You know, that's slow. And also, like, are you sure you need that much? And so, you know, we kind of look for those outliers and help bring them into the shape that we're actually going for. in terms of like impact I mean this is like what the whole industry is trying to figure out I don't know that we've like completely nailed it

which is like okay so you're using all these tokens are they accretive are they actually adding you know growth or velocity what we try to do is not anchor on usage we try to anchor on like the downstream metrics so you know we have like a pretty rigorous way of looking at like okay for software engineers do we see PR throughput increasing? Because that shows that the output is increasing, which is a good sign that they're producing more. But it's not a sign that they're producing more useful stuff. It's just they're producing more. And then I think we look at actual roadmaps. The best indicator that you're using AI well is your roadmap is shifting to the right and compressing. And so we track that, and I think we actually shared stats around it. There's so many confounding factors in everything you're saying, though. It is, 100%. Like PR granularity, roadmap distance. Yeah. Oh, yeah. Totally. Totally. But I think what we feel, like, what matters is that we feel that we're able to do more in shorter amounts of time.

And I think that has been true. But to your point, there is no perfect metric on whether your AI use is resulting in business impact. but there is like trend lines that you can look at that like again the confounding factors almost don't matter if you're able to shift more faster ai has contributed i also think by the way we underestimate how much there's been a shift in how we work it's not just about oh yeah i'm using lots of tokens but like now you know when when you think about product development in the old days used to be like have like an mle and the mle would work with a product engineer who would then work with a designer, who would then work with a product manager, and then legal, and then a policy. And, you know, each one's producing, like, a different artifact that needs to be cascaded across each other. But now you can just jump straight to the prototype. Everyone's just, like, right in code, including the designers, including the PMs, including policy, including legal. Like, everybody can just jump in

to better understand what is the implementation versus trying to pre-negotiate what might be at the end and then like slowly kind of going through that. Every time I hear you say the roadmap pull in, like I feel like the roadmap should be expanding because you can be bolder and think bigger about what you want to build. And if it's pulling in. That is true of companies that don't have more things to do than people. Yeah, I see your roadmap is so deep. Like the most I can do is pull it in and try to get it done. There's a lot to do. There's a lot to do. And so we feel you're right. I mean, there's two ways to look at it. Are we doing more or are we like pulling things in? I think we have a large ambition. I mean, Airbnb wants to reinvent itself. Our roadmap is like pretty insane. We're trying to move from, you know, a core marketplace of stays to like, you know, everything that you can do in the physical world is now on our platform to kind of reinventing the entire tech stack and the like platform to support all of that, but also to become more AI native.

And also there are things that we will introduce hopefully soon that kind of almost shape what the company can become. So I don't know how much more ambitious we could possibly be than that. But so, yeah, for us, it's really like we got so much to do. Can we do it faster? Can we do it faster? And that's like our main energy. But, yeah, I mean, some companies should be asking themselves questions of like, are we ambitious enough in the age of AI? So we've talked quite a bit about AI for your engineers and then helping them be more productive. Certainly, there's a lot more detail we can dig into there. But I'd also like to hear about some of the other ways you mentioned that you're applying agents in particular to the marketplace and the guest host experience. Like what comes to mind in those domains in terms of concrete things that you've been able to achieve with agents? Most people see like the guest side of Airbnb. It's kind of like my best description of it is like an iceberg where at the top you see like the booking engine and the UI that most people use, which is the guest experience.

But then underneath that surface is an incredibly complex, like two-sided marketplace where like the main thing that we trade in is trust and payments and quality. And there's just like this very complicated operational problem. customer support, insurance. There's a lot that goes behind supply and demand. There's a whole host side. There are millions of hosts that we have to support. And they're a core part of our ecosystem. And so we build a whole bunch of stuff for them. So there a whole very very complicated stack that lives below what most people see And so when we think about AI deployment a lot of what we been able to do even in an offline fashion is help support the core tenets of what Airbnb is, which is a trusted marketplace. And so examples, payments.

Can we use AI agents to help identify payment disputes? or like, you know, fraud? Or can we use AI agents to support better guest host experiences? Can we detect quality issues? Bad entree experiences. And these are like, can be done at scale in an offline fashion to help the platform improve its service, but it's kind of like behind the scenes can run large scale, you know, in a large scale fashion without actually getting in front of the user and trying to render anything to them. And then for customer support, I mean, it is a massive operational challenge, right? We have to bring people, we have to train them, we have to develop some level of consistency. And so most people can easily say, well, customer support can be an AI problem. But actually, we don't see it that way. We see it as how can it be a quality improvement?

And so we do want to make sure that AI is actually, if you have a very simple task, I don't want to call, wait for 45 minutes just to ask, can I move my reservation? I just want the answer. So AI has a really powerful place to solve for that. And actually, I would argue there's a whole bunch of reasons for why that's probably one of the hardest problems. It is actually a frontier in a really interesting way to basically build for a two-sided marketplace customer support problem because we don't own the product. If you are unhappy with something, it tends to be a mediation versus you dealing with a company. You're actually dealing with another person through a company. And two-sided mediation is literally having a person in between and two very unhappy people and you're trying to get to a very happy outcome, which ends up being a very complicated problem to deal with. And so that turns out to be

one of the hardest problems that we're working on that is AI related. What makes it so hard besides the broader context that you outlined of a lack of control? Do you think of it as fundamentally an experience or user interface type of problem? Is it like a game theoretical kind of algorithmic thing? Is it something all together? What makes it so gnarly? It's all the above. It can be very adversarial. So you have two parties with almost like a bounty that they're optimizing for. It's an outcome they're optimizing for. You have a difficult evidentiary track record, so you can ask each entity to produce its evidence or artifacts, but you can't really verify the accuracy of them. I can easily take a picture of a table, work with AI to modify the table, and say, well, I came to this place and look at the table. it's missing a leg. But is it missing a leg? Is it not missing a leg? I can't go

inspect it physically. So it turns out to be like a really hard adversarial problem. Did you saw that leg off? Exactly. And so, you know, it's a really, if you think about it from like a machine learning perspective, you have an AI agent and you have these two parties that you're trying to resolve. But you also have your own policies, right? So if you're, you know, if the company if you bought a stay or experience and it had a policy of non-refundable and you're trying to find a reason to refund it, you also have to stick to the rules that everybody plays on the marketplace. It has to be a fair, equitable relationship that you have with the company. So we can't take sides. We have to be very deliberate to applying our policy. And you have two adversarial entities that you're trying to move forward. And so it is actually a very complicated, How do you solve it? It's a frontier problem. The models today can't really do a great job of mediating between me and you.

And then if I'm Airbnb, and then Airbnb and the host, and let's say you're the guest, it is quite a complex, multi-party mediation problem. People talk about multiplayer. We're all putting agents inside of group chats, and it barely works. Now imagine if you and I had a financial dispute, and we stick an AI agent between our chat, like is the AI agent going to really resolve it or is it going to like not? And it's a physical interaction. You cannot observe it. Right, right. And so you're not saying that you're not using agents there. You're saying you're using agents or at least thinking about using agents and it's hard. Like how are you approaching it? We're in like research and development to deal with the hardest use cases that I'm referring to. But we still apply a lot of human judgment to those use cases where we can train humans to have the human touch, to have the interaction, to really make sure we get...

Again, Airbnb is all about making sure that we give the best quality experience that we possibly can. So we don't feel like it's ready for the hardest use cases. There is a large tale of use cases where it is incredibly useful. So we've already deployed, you know, AI agents into production for customer support. And, you know, we resolve about 50% of all chats with AI. So our agentic solution is already resolving a lot. 50% of support chats in particular? Yes, yes. Okay. Yeah. Is the user, guest, or host interacting directly with an agent? or are they interacting with someone that appears to be a person, but that person is really an agent or supported by an agent? We have a philosophy of being very transparent. We're not putting an AI in front of a person and saying it's a person. We're putting an AI in front of a person and saying it's an AI. So we're pretty transparent about that. But the 50% is purely AI, excuse me, resolve tickets.

So, like, we're not, this isn't, like, a human-assisted AI, and I'm talking to a human, and the AI is assisting the human to close a ticket. This is 50% AI to human resolution rate. And it's a hill-climbing problem, right? So, like, there's a set, like, informational queries turn out to be really easy. That's not the full 40%, like, those are pretty easy. And actually, it's way more convenient. Like, if I'm calling Airbnb and I'm asking, like, an informational query, just give me the answer, and I'll have my merry way. Don't put me on hold. I don't want to wait for a person. I just want the answer. And by the way, we also give cards, action cards, for people to resolve directly. So you can take action. It's not just information queries, but it's also agentic, which covers a lot of use cases. But there's still some portion of use cases that are extremely hard that as we hill climb to the frontier, that is where we really need to push the boundaries and we really need to be quite innovative to solve. So it's interesting, my personal experience as a guest and host with Airbnb, like I don't feel like I've run into AI usage that has shifted significantly over the past like two, three years.

Like if I need to go to the support site, it'll, I feel like the flow is the same. Like which day is it about, you know, is it a guest issue, is it a host issue? And it kind of walks me through that, but it's been doing that for a long time. and I've never had an agent answer or get involved in the chat. I have noticed in the chat, like in-chat card moderation, oh, it seems like you're trying to do like a change request. Here's the card, click the button, which is cool. I'm wondering, like, you know, one says that's a really good thing, because most of the time when I think about a company forcing me to use AI for support, it's not a very good experience. I see that as a success metric. The goal isn't to shove AI in front of people. It is to meet people where they are. But there is a chat customer support agent that you can initialize. So if you say contact customer support via chat, I think what you're referring to is like our help center, which, again, people who are using our help center are actually looking for a particular experience.

They're not looking to just call Airbnb or text Airbnb with a problem. And so those workflows still exist. and there's still a lot we should be doing with the help center, so that's going to happen. But the volume of our tickets actually happen via chat. And if you do, by the way, if you get into a customer support chat and you say, like, give me a human, the AI will, like, escalate and we'll route it to a human. So we don't force people. I don't know if you've ever used those, like, terrible call centers where, like, it just, like, traps you. sorry you have to press like zero 18 times before i get you to a human or maybe if you yell at me enough i'll get you there like we're pretty careful about getting you to a human if that is what you prefer i was just talking to someone about how difficult it is now because like i've been trained to say agent with like increasing urgency when i want to talk to a human yeah but now like what's the agent yeah exactly exactly oh you want an agent well let me introduce you to

But it's new age. Human, human, human. Human. Yeah. Yeah. And, you know, if you look at the call volume, like if you actually look at the number of people, there's people who have zero interest. And it's not an insignificant. We're talking about tens of percent of people who will like immediately look. That might change, you know, as we get better systems, we get more natural systems, as we get more capable systems, people might shift. But there is like a non-insignificant number of people that just have no interest in talking to an AI and just one human. Even if their ticket, even if their item is just informational, even whatever, they're just, give me to a person. And those are going to be the hardest ones to resolve for because it is really, it's like a, it's psychological. It's not capability based. Since we veered into personal experiences with Airbnb, I think one of the domains where I'd be most excited to see interesting AI things is in terms of agentic search. Yeah. I know you've worked a lot historically on trying to capture user preferences, but it seems like I often have to do the same mental filter for what I really care about.

and it'd be great to be able to... And there also, there is a lot of information that you have about properties that you don't expose via amenities or search filters or something like that. It'd be great to be able to... Yeah, it's deeply embedded in the imagery. It's also embedded in the reviews. It's embedded in a lot of contexts. I mean, part of why you trust to book these homes is because we have all these additional signals that make you comfortable to book them. And those signals are just as useful to a person to absorb or an AI to help match to a person. And, you know, like, search is such an important part of how people experience our platform. And we're really very deliberate. So we've experimented a lot with what to do here. And despite your personal preference, there is like a broader, you know, broader non-consistent user-based behavior that we've been trying to better understand.

People don't want a blinking box where it's like chat. The text back and forth is like, it's friction. I know where I'm going to want to stay. I know how many people are going. Give me my options. And I don't want to type like a 45-sentence query. You know, we've run tests for people. No, I don't want that either. I agree. Yeah. Yeah. So we've, so something we've, like, we're recently, I think it just announced it, or like, we're testing. But we're testing a lot of variants. But one of the things that we're testing is, like, just a natural language box where you can just type where you're going, who you're going with, and, like, why you're going, what you're looking for, and we should be able to return the results. But it's still very multimodal. You still have a lot of options. You still in control I not getting an AI to go pick the one property I sorry that like the one or the two or three I still giving you the ability to sort of transpose and like look at different cuts One of the things you see in how people operate these platforms is they don't, you don't, it's not

like a straight line to a booking actually. It takes like weeks for people to book and they like get inspired. They come, they search, they click around a lot and then they like don't come back for like three more days because I guess they go talk to their partner and they're like, hey, I don't know. Yeah, because these trips are like social and they're like multiplayer. So, you know, you end up like going on our platform, finding some stuff. Maybe you heart some stuff and then you go back. You got to figure out some other things like logistics, timing, travel. Like there's so much to figure out. And then you come back and then you click on some more and then you end up like, you know, maybe removing some hearts, adding some new ones. And it takes weeks before someone's like, all right, I'm going to spend $10,000 on a trip with the company. So you don't want AI to sort of remove that agency and exploration process. You want to enhance it. So part of what we're trying to do with our agentic search is accommodate for that journey that people go through when they actually decide what they want.

in some ways, for some trips at least, because I think one of the things that you're identifying is that, you know, even for me, not all trips are the same. And so for one type of trip, I'm just going to put in the city and find the first thing that needs the criteria and balance. And then for others, it's like this wish list process and like, yeah, oh, hey, check these out, babe. You know, and, you know, weeks later, we settle on something. For some kind of trips, you know, I might not even want to interact with the Airbnb experience, I'd want Airbnb to be like an MCP server to a world of places I could stay and like have my agent sift through all of the things. And historically, this has been, whether Airbnb exposes an API of properties, for example, historically this has been not a technical issue, but like a political kind of business issue.

Like, do we want to lose control of the customer? Do we want to, you know, let people scrape our information, all these kinds of things. But it does strike me that, like, that argument is starting to disintegrate because of computer use. And I could still have my agent do it. It's just, like, harder and takes a lot more time. So why not just give me the MCP access? Like, how do you think about that? I come back to, like, consumer companies and their DNA and what they need to focus on. Like, fundamentally, you need to focus on, like, what is the best user experience to experience our platform. And I think you kind of need to evaluate it on a case-for-case basis and, like, what makes sense and what doesn't. And so I think that is the lens upon which we will make all of our decisions. I do want to – I do think, though, that, like, travel is kind of a unique animal in a lot of ways relative to a lot of other things that you could be doing in the context of, like, convenience isn't key, trust is. and like a booking isn't just a booking it's like you have first of all you have the inspiration

question which obviously you know ai with its world knowledge and its web hooks and like the fact that it can search the internet and aggregate all this it's like useful but it is not like it's not the only thing they use for inspiration lots of people get inspired by going to like instagram and watching videos and there's like all kinds of ways people get inspired about where to go including talking to friends or events or there's all sorts of things or agents oh yeah ai is definitely huge and like i think you know 40 or something like that of like humans in in the u.s use ai for uh for inspiration but when you come to book like there's the booking and then there's like going on trip and then there's like everything that happens in between and the trust that you have when you do book. So we think that that's going to be something, like the best user experience is going to be the one that you trust the most. And because it's such a high judgment, high value purchase, you really want to trust what you're booking and who you're booking with. And so that's kind of, I think, one of our main strengths as a company.

It's a platform that you can trust. If you, you know, we have like insurance, we have like payments and a payment stack you can trust. We have customer support. We will deal with, like, issues versus, like, a random, you know, MCP server somewhere that you're, like, booking a $10,000 trip with. Yeah. Yeah, I don't think that I'd want to, like, I don't think I'd want, like, some agentic commerce thing where the agent just books to stay. I think I'd want the agent to help with screening, you know, just the same way your natural language text box would. but presumably the agent can know and store my preferences and kind of get through that more quickly. Yeah, yeah, yeah. And I think that's what I meant by case-by-case is we have a bar for what we want people to feel experientially when they see what we're providing. And again, a home, for example, is very personal to you, which is why the preferences thing really matters. But also finding the right SKU that works for you is a deeply multimodal thing.

I wouldn't want our product to be a bunch of techs that a user picks. I would want them to be able to experience it in the right way. So when we feel like there's a way to deliver the right experience to people, we'll work with them. But I think that's kind of very important to us. We talked a bit about some of the tooling and tech stack on the engineering productivity side of things. When you are deploying agents for marketplace, guest hosts, what are some of the platforms or tech stacks that you're using and building? Are you big consumers of open source? Are you using commercial agent platforms? Have you built everything from the ground up? It's a mix. Let's see. We are users of open and closed source models. we have definitely leveraged open source frameworks

and we've also built where we feel we need to, where we feel we can innovate and where it's actually valuable for us to build. So take customer support, for example, some of the agent frameworks there, we started building before there were a lot of open source frameworks available. So it was more out of like, we were ahead of where the industry is and we built what we think we needed to build because there wasn't anything there. And arguably, we always evaluate whether the open source community has built something that's further ahead than what we have. If it did, we have no qualms rebasing on the open source or migrating to the open source solution and then contributing, actually, to open source. So my stance on open source is well-known. I am a big proponent to basically build in the open where we can. and then on the model side you know there's a strong you know we're an experiential company there's a strong Pareto that we've been really focused on which is the speed

of our interactions we don't want people, we have like very very tight latency requirements and given our scale like for certain use cases it makes no sense to use large models for very specialized tasks and for that you know we have our own post training infrastructure that we our own RL environments, et cetera, so that we can take more efficient, smaller, faster models that could really kind of beat the frontier at the narrow benchmark that we care about, right? So if you have use cases where you have a narrow thing you're trying to do, you're dealing with customer support and adherence to a policy, I don't have to worry about regression on mathematics or regression on some other domain that I don't care about. then you can actually post-train a model to be really good at it. I would actually say that was why my hot take was like the models are good enough for a lot of consumer experiences. Because right now we're kind of focused on the jagged edge of like, I have now solved a millennium enterprise problem versus

can I do reliable computer use on a travel website or whatever? Like that feels less, you know, you need a 10 trillion parameter model to achieve. and actually that applies at various scales. And so the more you're able to, if you have narrow use cases, which we have examples of, like our search is a good example, where we take the smallest model possible because our TPS is so high that we really post-train and specialize and distill that model to be really good at what it used to be good at doing. I mean, we're already starting to see a lot of people train different types of models that are just focused on very narrow things. like the Java example is a good one where people basically trained a model for decision making which is awesome but it's a very simple model that focuses on something really really high leverage for a lot of places and so we've learned that and that's why we have our own investments in post training open models and do you have a favorite

small model that's kind of a go to? there's a lot of good ones, there's like Pneumotron from NVIDIA there's a bunch I mean, there's a bunch that we look at. We're pretty open. We evaluate everything. I'm imagining same as the case for the larger open-wage models. You just kind of have a staple that you, you know, eval everything against, and you choose the one that makes the most sense at the time. Yeah, that's right. That's right. Any thoughts on kind of where you see this all going at Airbnb? Like, what are maybe some of the big challenges ahead for you, and how do you think AI will help you solve them? Of the three things we're doing, I think we did a good job sort of reshaping the company to be AI native. I mean, like, Brian Down, everybody's using AI. Brian famously, I think, just shared at a conference that he has a second brain that helps him run the company where it's been trained on. He has built a context graph, which is local to his machine, that is almost like it's been trained on.

It has, like, every document that's ever come to him. It has every vision document, manifesto, email, etc. a piece of communication he's ever written. And he uses that to help reason through his day, which he's using. So if the CEO and the founder is doing that, you can imagine the company is very, very excited about it. So I think we're on the right clip. I don't know if it's a challenge, but there's a lot of complexity that comes from a lot of AI use. like Everest. Is Everest going to be useful or is it going to become problematic? Are we going to corrupt it as we scale it? We don't know. Do we even have the right architecture for it? Are we going to have to rewrite it? But this is all work. When you're trying to do it from zero to one it's much harder than going from where we are to just keep pushing to stay at the bleeding edge. And then I think for the marketplace I also feel really comfortable. I think we're still working on it but I think there's still a lot of degrees of freedom on how we shape the user experience that people have with Airbnb using

AI while staying true to what we are as a company. I think that is actually, like I said, it's a frontier of interaction. And one of the things that's exciting to me is travel is very complicated. Whoever's watching this can go try to vibe code something. It looks great. Try to use it for a few days and you realize like, oh my God, I can't use it. I want it, you know, itineraries. I want to be able to share with other people. I want to be, it turns into this like bloated mess. So actually the interaction model and the design of how that comes to life is actually really, really hard. And then, you know, for, for customer support and for this like multi-agent problem, multi, multi-person problem, how do you deploy agents that can actually resolve things in a, in a constructive way. How do we get to 100%? That sounds like a meaty challenge to dig into there. Yeah. Awesome. Well, Ahmed, thanks so much for jumping on and sharing a bit about what you're up to there. Yeah, thanks for having me, Sam.

This was fun. Thank you. Thank you.