私たちに代わって安全に動くエージェント：WebMCPとエージェント時代のウェブ

I've been thinking a lot about, do we need to change the web for agents or do we need to just make agents better so they can use the web for us? No, we should be changing the web for agents. But you could imagine if it worked out of the box for React and Vue and all you had to do is do a package JSON flag change, that that might make it a little easier. I do think that site builders should eventually explore WebMCP. It's like it works, but it's not what you want to have working. Can you tell me the story of WebMCP? Like, how did that even become a thing? It was a little wild. I'm not going to lie. What is the talk about at AgentCon North America October 22nd and 23rd? Yeah, I'm really excited to talk about how the web is changing due to AI.

It's changing on a number of vectors. One is, first of all, just that agents are present. And so we've got this fourth actor on the web. How do we secure it? how do we expose good APIs so they can navigate appropriately and well so that people can guide their businesses and then also just making sure that we're putting the user first in the whole transaction yeah so many different rabbit holes we can go down on each one of these I mean the secure part is probably what I think most folks think about when they think about agents acting on our behalf but there's also like just the web part and agents using the web and so I've been thinking a lot about do we need to change the web for agents or do we need to just make agents better so they can use the web for us no we should be changing the web for agents and by saying that I don't mean that we need to rewrite everything um webmcp offers really wonderful tooling and it's a

pretty elegant, simple API. What we're working on now are ways, and what I'm working on now are ways that we can actually use what people have already written to expose WebMCP tooling. What WebMCP tooling allows you to do is to specify what exact actions you want the agent to be doing, and you can also add annotations. So as you're adding the annotations, you can say that it's read-only, that it's user-generated content. that can allow the agent to know, oh, okay, I'm not going to take action on anything in here. Like if you have a bunch of comments on your site that are user generated, you can't necessarily trust that people aren't injecting prompt injection attacks in there. So the agent actually has a signal, hey, don't, you know, go forward with anything that's put embedded in there into the web ephemera. So that can help the web stay secure and safe. There's also a consequential hint, which allows you to say, hey, agent, you're about to do something that's potentially

consequential for the user. Please go back to the user and confirm. Otherwise, agents are just scraping the DOM, scraping the accessibility tree, taking screenshots. It's token inefficient. It's slower. The site developer is just kind of like, you know, at the whim of however non-deterministically these LLMs decide to take action on the site. And so this gives us a really deterministic surface area in order to guide the agents exactly what we want them to be doing in a safer way. You're speaking my language. I love that. There's so many pieces to that that I'm a huge fan of. How do you look through and think about like computer use or browser use agents versus the API or MCP type agents? Yeah, I mean, there's so many different ways that a browser can be used these days, right? You have in-app browsers where people are, you know, if you're on a certain like

super app, they might keep you within that instead of allowing you to go to whatever your browser of choices. You might have computer use apps or computer use ways of working like 24-7 headless models where you kind of set it and forget it and let the agent do whatever you also have co-brows where you are a human in the loop and you're actually in the experience we're seeing more and more that people might want to switch between these modalities like maybe there's one where you just want to do it yourself maybe there's one where you're you would like actually a little bit of co-brows because you want like hey google maps like show me all of the restaurants in a particular area that i can afford and you want to actually be part of that experience instead of letting an agent run wild. And then finally, there are tasks where maybe you're part of the task. Let's say you're like getting ready for a holiday. This is the only time every year that you get to plan such a holiday. You want to be intrinsically part of that. But then there's one part that's like, go through all the TripAdvisor things and tell me like which ones are most valid to this

kind of query. Like you don't have to sit there for something like that. But you might want to be involved in something like the airline purchase or something like that because you want to pick your seats and stuff so what we're seeing is like keep more and more it's not just one modality and this is also the dream of the web the dream of the web was not just one experience ever or one person or one user or one type of commerce site content site whatever it's supposed to be for everybody at every level some people have high appetite for agents and people have low appetite for agents we have to keep the web safe and secure for all of those different pathways It funny as you mentioned that I literally have my other computer over here off screen And it is downloading a bunch of logos of companies because one of the things that I tasked my agent with doing is grabbing all of the photos from our recent MCP con and agent con in China and Japan. and then taking the speaker photos and uploading them to Canva so I can go and I can make the

thumbnails. But then I realized like, oh, I need the speaker photo, but I also need the logo of the companies that they work at. So it got the speaker photos and it downloaded them from Flickr and then it uploaded them from, it filtered out whoever, wherever there was photos that weren't speakers. And then it uploaded the right ones to Canva and then it put their names and the title of their talk so I can easily search it. And then I'm like, oh, but I need the logo. So now it's literally just popping up with websites and grabbing logos and then uploading them to Canva too. And that's something where I look at it as exactly what you're saying, where I am letting it do all of that monotonous, heavy lifting of downloading, uploading, searching for logos. But what I want to do is that creative part of taking the photo, adding the logo and then making it is it should be a little bigger here, I think. And that's where I actually find the joy in it. Yeah. And I think that that's like that is the most powerful part is that it should be tailored to what the user wants.

It should not be, oh, there's agents. So you're going to use an agent. It should not be, oh, I have to be in the loop. It should be exactly the experience you feel comfortable with, that you feel secure with. And we should be also tightening the security and privacy of each one. So what we have seen is that it's not going to be one thing that secures the web. We have to use multilayer defensive strategies because agents can be non-deterministic and they can do all sorts of things. And so you have things like special agent containment where you might say like, okay, me, Sarah, can look at all my passwords and has my credit card info and all of this stuff. But Agent Sarah, well, she can only go to these five sites and she only has access to this information about me and she can only spend $20 this month or maybe none or whatever. and that way you can like kind of define what you what experience you want your agent to have

another one is um origin sets so if you decide to go on like a trip or something and you're like okay plan this trip for me it should only be going to expedient united and those kind of sites it should not be going to your bank and to your like health site um another one is spotlighting so like let's say you have a large context window full of a lot of things, the things that you decide as a user are most important should be, like we use the term spotlighting, like more important to in that context window and have a greater weight than all of the other pieces of information. So I'm just talking about a few. There's so many ways that we can protect the web, but we have to use them all to make sure that we're creating good and safe journeys. I like how you got those agents on an allowance. You can only get $20 if you make sure to do all of your work. That's right.

And if not, you're not getting that allowance. I'll tell you what. The other thing that I was going to ask you about, which I always find fascinating, is after you've reached a certain level of engineering, engineer and this is more like just how you think less about your talk just um prefacing it i think there's we always joke like uh friends and i tend to joke about how you work with some engineers and they have like a really good nose or they can smell when there's a business problem that's worth solving and so i wonder how you think through that like how if you're gonna spend your time on something, how do you make sure it's that really impactful thing versus something that may not carry as much weight? Yeah, I mean, as a distinguished engineer, that is one of the parts of the job is being a strategic lead, like deciding, you know, I'm what we call an area tech

lead, meaning that I go over a large surface area of Google. And so I look over the AI and the web ecosystem for chrome um which includes a lot of different things web mcp is one of them on device ai all sorts of things um so when i think about things i do have to constantly because i get deep as well so if you get deep what happens is you naturally think about the problems in a very tactical tactical way too but i always have to bump myself up a level to also see things from a higher purview it helps that i can be across many many different projects and i actually pull different groups together so that they're aware of each other's work and kind of highlight and have the they do them do information sharing across groups too because people tend to silo real deep and then you forget to bump up at that level but i also encourage you know i i make myself write papers with that thousand foot view that could look like a premortem Premortems can look like OK the web failed in five years What what happened

And that forces me to look back and say, OK, well, this can happen. This can happen. This can happen. And then revisit like, OK, how do we make that not occur and trace backwards some steps? So that's one like way of thinking that helps. Another way of thinking is to have deep empathy for the users and the site builders and think through from their perspective. What do I need if I'm an, you know, if I'm smart and not technical, if I'm a doctor or a lawyer and I have to, I have to, I'm under HIPAA or I have attorney client privilege. What do I need in order to feel like I can even use an agent at all? like what kind of privacy and security would i need to be assured that my things aren't leaking and getting um used improperly or that i'm being suspect to a prompt injection attack or something like that and from there developing strategies of like okay well i'd probably need this i would

need this and you can extract from that what can make it easier to do those things like i mentioned finding these pathways to make web mcp work more like out of the box with existing systems that doesn't need to happen but i think it should happen because that also increases like the ability for developers to opt in without a lot of work for businesses to be able to use it without having to like rewrite in a whole new mental model because they could reuse existing infrastructure they've already built so anything that helps me kind of think through things and pose questions to myself at a higher level and walk backwards into the more technical work because you can start to extract the tactical work from those larger questions that's something that I force myself to do that's so cool and yeah the like thought experiments and thinking about how to come up with thought experiments I imagine has got to be a lot of the way that you spend your time just

because there's I can only think about how many different folks that are pulling you and trying to get you like hey this is important right now we need to we need to think about it and then you're having a balance between all of those different like demands that's right that's right And it does require a lot of reading, a lot of – so it also requires, I mean, honestly, time management because I'm in so many, like, different things that I have to always set aside a big amount of focus block. And Google – our love language and communication is docs and slides. So I have to have time every week to process all of that material, to understand what people are trying to build and make, go through CLs, like play with things myself, make sure that I understand in order to make some of those calls and decisions. like we were you know going through something the other day where i noticed that one of the

decks and docs was incongruent with how we were actually doing the implementation and you have to dig in deep enough to trust but validate the things that people are mentioning to you across teams it doesn't mean that you don't it doesn't mean that you don't trust the people it just means that like if we're strategically saying this are we really standing behind that in technical implementation and like making sure that it's the same can you tell me the story of web mcp like how did that even become a thing yeah so just to be clear i moved into this role in april so i haven't been here from like the beginning of the entire spec um okay the um the spec started as I understand it at Microsoft and Edge. I do actually think Alex Nahas, who does WebMCPB, originally started thinking about the idea, and then Edge started working on the spec proposal,

and then Google got involved. And now Google does a lot of the work, but it's always been like a cross-team, cross-system collaboration, which is the good thing, not a bad thing. So I joined in April. Before that, I was running core infrastructure for Google. So I ran web at Google, Android, iOS, multi-plat, experimentation, observability. I was also the reliability for all of core, which means that if anything went down across Google, I was on point to for our internal stacks. It was a little wild. I worked with Chrome before I had this job, but they were one of many, many people. I worked with, right? So I worked with Workspace and YouTube and Search, and those were all my customers. And then I moved over in April to this role. So I'm no longer doing the large-scale technical organizational leadership of core infrastructure at Google. Now I'm a distinguished

engineer, so I switched tracks, but not levels. So now I'm on what we call the IC track. Yeah nice So a little bit of a random one on WebMCP that I been really enjoying is can I build a mcp proxy or gateway to many different websites that would if i know how your website works because i'm just worried that half the internet is never going to build anything web mcp related because they either can't or they don't want to no you don't you you are the owner of your site's webmc webmcp implementation you're not making it for other sites that's not a a thing that you can do just like when you build your site you're building your site you're not building other people's sites it's the same kind of concept it doesn't go outside of the like site development things but you could imagine if it worked out of the box for react and

view and all you had to do is do a package JSON flag change that that might make it a little easier to adopt we're also exploring things to make it work out of the box without like the site builder having to do like you could just use this other thing that would like analyze your dom the problem with the analyzing like that that's a good it's a good stop gap if you don't have time to do a web webmcp implementation yeah but the issue with it is it tracks based on dom and dom isn't the state of your application so um i think that approach can be helpful in a transitionary period but i i do think that site builders should eventually explore webmcp yeah yeah yeah so it's it's like it works but it's not what you want to have working yeah and it's also like i think like Some people get confused because they think that if you have WebMCP, you're trampling on MCP or you're trampling on REST or GraphQL.

This is not the case. Anything that is exposed within your UX or UI that is a state that you'd already be exposing with React or what have you can just be the execute piece of WebMCP. and that includes allowing you to use MCP and GraphQL and REST and things like that. So I think that they come together. If you actually work with it, they actually play really nicely together. It's not like a either or. Yeah, at AgentCon, I heard someone talking about how at AgentCon EU, I mean, someone was mentioning how they built a WebMCP and the only thing it did was it linked to the mcp server but it was it's like ah it's not ideal again it's but it's something that will get you what you need to get to uh in a little bit of a hacky way yeah i mean you don't have to expose it that way either i mean you can what i would think

about it more like okay if you let's just do a cart checkout flow for a second yeah on the client you probably normally expose to the user your cart total, like things like whether or not you're, you know, navigating to the page, the like full thing. And then you do the cart checkout and then you have on the client like success, error, submission, things like that. And especially if you're in a co-browse environment, you want the page to actually be activated to show all of these things. Yeah. And so you would use both. And you could actually just do WebMCP, WebMCP, MCP, WebMCP, WebMCP. So you can also do just chained WebMCP and then one of them calls MCP. You can do it either way. But I do think that there's also things like off and things like that that you want to be exposed via the client to the user. So a lot of it has to do with like how your implementation is already working in your UX.

The other thing I'll say is I've had a lot of fun thinking outside of, you know, the first step is exposing what you already have on the site, like whatever you're already doing on the site. But the next step is I've been making tools that allow you to do batch processing. So I have a to do list. I can just say and this is good for accessibility, too. I can now say to Gemini or whatever, like, hey, close out my day. And it will automatically take all the to-do items and carry them over for tomorrow and also start my journal entry. Or it will allow me to, in a shopping cart thing that I've made in open source, it allows me to dump usuals into my cart. Like anything that I've ordered more than 10 times, it automatically puts them all into my cart. Like those are things that, first of all, manually you'd have to do differently. There are also things that you're kind of like exposing on the client to the user and you probably want them to be able to see that that's all working appropriately and what's in the card and things like that.

And so there's opportunities to go beyond just the like exposure of the simple actions into something that is potentially really good for people with accessibility issues, but also just people who are busy. I don't I don't I don't want to go through and like and it's doing that mental math for me of like, wait, I always get eggs. So that's good. It's an opportunity to delight the customer in ways that they weren't expecting. That's right. That's right. Brilliant. Sarah, this has been awesome. Thank you for doing it. Yes.