Chen Goldberg（CoreWeave）が語る AI ループ、Forge、エージェント時代のインフラ

It's not just going without a mission. We know what we want to do. We have a plan. We are executing on that, and that's something I really love about the COVID culture. What excites me is that I feel like AI can be used for problems that existed forever, and we didn't have the tools to solve. It can be in health, it can be in education, it can be in manufacturing. You lead around 1,100 or so folks across our product engineering roles. I think for engineers, several things are happening here. One, there is some identity crisis because you don't write code. I think I was wrong. The more tokens you use, the less you sleep. Pam Goldberg, EVP of Products and Engineering at Corvive. Thank you for coming on Thursday. Thank you for having me. I can't believe it took so long for me to be here, but yes, I should work on that. I'll do my best to ask you a few questions. First of all, about this crazy thing we're in. Behind us, there's a big Corvive thing, a blue wave, and we're in the middle of Moscone South in San Francisco for the fully connected event that started way back when

with some biases in smaller rooms. And now this is like a whole new thing. Do you want to address the magnanimity of what we're doing here a little bit for a second? I think it's unbelievable and it's overwhelming and it's humbling. It feels really strange to be in this situation because in our head we still see ourselves as a startup sometimes, right? And then you come here and you see thousands of people. And the sessions are full. And the content is amazing. And the vibe. The vibe is cool. It's fun. I had a few folks that came over for OpenAI's Dev Day just the day before our keynote. And then I can invite them here as well. You're already in the city. One person said that Dev Day was not that great for him. And this was the highlight. and he wasn't talking about the BattleBots which was also really this was crazy there's music and sound and loud announcements as well and it's actually it's the kind of things that

fits our culture and the thing that we like doing it's being exciting about technology and the fun things that you can do we're having fun all the time and Lucas Biewold the former CEO of Wagon Vices he built a bot have you seen this bot? He's like a little spider thingy. That was really cool. So this is just the fun part. But I think it's also fun. And I think I noticed this in you. When you announced the product lineup and Forge, for example, I would love to talk to you about, you announced a lot of pride. And here's the question for you, Henning Goldberg, EVP of product engineering at Corvid. Which one of the releases would you say was the easiest for you to deliver and get excited about? Wow. I wasn't ready for that. It's kind of a pick your children's question. I know it's annoying. You know what? I'm not going to put you in a hot position. I'm actually... The thing is, again, people don't know us. I talk on stage about speed and experimentation and learning and moving fast.

And that's what I love about what we're doing. There are so many things that, you know, when we start, like you are not sure that it's possible. And then it's there. And you're like, wow, I'm not going to say a word I shouldn't say. it happened first of all let's fucking go is kind of the company model yes and I think it's absolutely fine on Thursday I use it sometimes you can say I'm saying LFE yes let's fucking execute let's fucking because it's not just going without a mission we know what we want to do we have a plan we are executing on that and that's something I really love about the Corvish culture yeah I wanted to go back with you a little bit On the previous steps in your career, you led the rise of Kubernetes and serverless in Google. And now we're announcing Forge and the rise of agentic AI is happening, agents everywhere. Do you see parallels between those huge movements? A lot, actually.

So, two things. One, definitely the experience that I had with doing the transformation is very interesting. But there's other things that are very different from a PACE perspective. Like we've never seen before. But I'm going to touch something that I actually talked with Dylan Patel today on stage. We had a fire site chat. Dylan Patel is the CEO of Semi Analysis. And my biggest realization from this conference is that something that back in the day we were working very hard at Google and the Kubernetes community is to avoid fragmentation. What does it mean? And you want to create some standards and best practices and APIs that everybody is using. And you can see some teams, right, NVIDIA and outside, like things that they are promoting. But the problem right now that is happening is that everybody is so busy and have not just cycles in their brain but also time and things are moving fast.

But it seems like nobody is investing in that. And that worries me because I don't think that Kubernetes would have been as successful without its ecosystem. So the ecosystem is super important and the partners and the people can build on that. But here, what we also heard on stage is that people just have to rewrite everything every few months. The model is changing. The chip search, like everything is happening. And that worries me. So if you ask me what I want to see in this world, I want to see a world where it's not fragmented. I would love to see a world where, of course, front-end models and open-weight models live together and you have frameworks that gives you flexibility and composable systems. And that's something that I hope will see some similarities between Kubernetes and the agentic world. 100%. It is a lot that's happening at the same time. I have so many directions to go with you that I'm just going to start.

So you mentioned every few months, every new model. It feels like now... No, models every few days. Every few days. Literally last week we talked about the latest GPT sold and they just reached 6 sold Dev Day And I had a chance to chat with Sam Elfman about some stuff and then I asked could you guys slow down We can catch up. He's like, I believe in you, Alex. You can do this. And I don't know how other folks are feeling. So I would love for you. You lead around 1,100 or so folks across our product engineering roles. All the products and engineers, actually engineers, they're getting new superpowers essentially with those tools. I would love to hear from you. How do you see that? Have you played with some of those things? What are the stuff that you're seeing changing in the software engineer career right now? And how would it evolve in the future? There's so many angles I want to answer. So the first one I'm going to talk about, again, was today on the stage. John Jones, our CIO, was talking with Parag, ParallelWeb CEO and founder.

And actually, they were talking about the pace and whether Parag feels like he's up to date with everything. And he said he's not. And when you're asleep, you always feel behind. And it was actually nice to hear someone like him that you think about, like he's probably at the forefront of that, and even he feels behind. So imagine how everybody else is feeling. So I think that's number one. From an engineering perspective, we had a, both in the customer advisory board, and we also had a women AI launch today. And that came up. I think for engineers, several things are happening here. One, there is some identity crisis because you don't write code. Second, when you write code and you have innovative ideas, sometimes it's hard to be proud because people just take it for granted. Like, hi, you had an agent write it for you. So people feel also about that. They invested a lot. Even if they used, of course, in order to produce something well,

even if you're using agents it still requires a lot of thinking oh you dedicate a lot of your energy into producing this that's what I'm saying but it's hard because then someone maybe once you do that like maybe someone else can copy it from you I think that's hard for engineers I think that's an identity crisis for sure and the third thing that we talked about the people that I see most successful are the experts and why is that? because they understand the space they are critical thinkers they know how to communicate about those ideas. So because they're experts, they can also communicate great. With the prompts. With the prompts, right? Because they're experts. Yeah. But they became experts. How do people learn? Let's talk about it. We know that. It's like 10,000 hours. There's research that shows that through repeating and going deep and being hands-on, that's how we learn. Yeah. But now we don't do it anymore. That's like... Oh, I'm sorry. I'm usually super optimistic.

No, no, no. We're going to turn this around because I will tell you some stuff, okay? I decided to do this Yapping Career, the podcast, after I was an engineer for 20-something years. I started back in Israel, moved to Silicon Valley, led the engineering teams. At some point, early on in the AI, I decided I'm not going to be needed. So why would I do this? Go find something else. I think I was wrong. And I think that, like you said, experts are much more needed right now because now basically everybody is a PM slash engineering manager kind of with running like a fleet of agents. And expertise is needed. How do they get there? That's a completely different question. I think there is a way. So the first thing, we need to appreciate the expertise. And what does it take to build a system? I think, again, put aside for a second coding agent, which we have companies, of course, building them and so on. But let's say I want to build a fraud detection or a call center or whatever. Those are agentic systems which are always on. They need to be secured. They need to be reliable.

I need to make sure that I have systems there that will keep the accuracy intact and so on and so on. Those are super complicated systems. And it's not pipe coding. It's not tinkering. It's not like none of that is true when you want to build such a system. so the first good news is that you need to do it I think what we need to do especially experts is be thoughtful about how we are training people and how do we ask questions how do we trigger people's minds and I think what everybody needs to make sure is that they need to be curious and not lazy and by the way I'm really like echoing things that Rita she's one of the engineers on the inference team she said that that's Like what she's thinking a lot about, she's a principal engineer. How does she help lift the juniors? Yes. Because they no longer have seniors, they check their work as much. I want to, though, turn this into some of the stuff that we have talked about here.

So all of those production systems that folks are now using AI to kind of help them scale, they have problems in production. And we're introducing Forge. I would love to just hear from you directly. I know you reviewed this on stage multiple times. What is Forge from Corwee? What did it announce? and how does this help companies make their agents better? I love the story of Forge. People probably don't know that, but most of our product ideas and everything we do just always come from a problem we saw, some kind of painful, like everything was with a customer or a user. I'm super excited that all the talks that we had at Fully Connected, we continue with the transition from weights and biases, all user-centric. So how Forge happened? Okay, we had lots of assets, right? We had weights and biases models, and we had our inference, and Infernt Serverless, and we had Weave, and different types of... Open Pipe folks. Open Pipe. We had lots of services. And then, I want to say probably like a few months back, we had two things happening. We developed ARIA on one hand, which is a system for researchers.

And the other thing that we're doing in the context of inference, we were optimizing our inference service and building a speculative decoding workflow. and in both of those experiences what we found out that the human in the loop was very tired. Yes. Stitching everything together was horrible. I heard you mention his name on stage Aaron There is actually we didn show it on stage there is actually a presentation that Aaron did that shows the loop a few months back based on his experience But it wasn just him right You can also talk with people like Tim for example that built ARIA It actually started with us, and then we started talking with customers. And what's happening is that no longer those machine learning engineers are separated from the developers, and the scientific systems are harder. And this is just one part of the problem. The second part of the problem is that Cove believes in a full stack optimization, right? Purpose built for AI.

And it was always very easy to understand when we talked about training, okay? Because the training job runs on hundreds and thousands and sometimes ten thousands of GPUs. One complicated system, one problem impacts your job, how you bring it together. That, I think, all resonated. now inference have evolved to agentic system which are also always on and running on hundreds of GPUs and this is where we were like and of course you have now more people worried about the data and doing pre-training and post-training and so on we're like this is the world that we know how to scope so well because it is purposeful for AI like that's what we do so what Forge is doing one it's taking that loop from an experience perspective but it also ties all the the entire specs together so how vision control and how the observability works and my inference service and my training service just all together in that one workflow and the context

moves for you and behind the thing by the way we talk about speed I think that there are hundreds of people that had to work together in order to make Forge possible maybe it's an opportunity to say to all of them from marketing and legal and finance and all the engineering team and the public. That was like, let's fucking go. Let's fucking go. This is incredible. We brought all the pieces together in the Gente Cloud, which looks different than a standard cloud from before. We have specific things for that. And I expect that we're listening to folks and bringing more specific things to what they need. And I think this is incredible. By the way, everybody can try Forge today. Thank you so much. I love guests on the podcast, They're like, let's go over there. Okay, let's go over there. Everybody can try Forge today. This is new to CoreWeb. When I joined CoreWeb, I joined the Ways and Bias acquisition. The one thing that all my friends kept asking me, like, hey, Alex, what about doing GPUs? Like, what's going on? I was like, I don't know.

There is a salesperson you have to talk to, et cetera. And then Ways and Bias acquisition was like the consumer developer product to CoreWeb, which was also new. And now we're creating this across everything. And I heard, you know, not a rumor, this was like live on the show, that we're also bringing GPUs now to Sandbox's offering, which is also incredible. I forgot to mention Sandbox's. Yes, Sandbox's. We had Deox, the PM for Sandbox's, and we talked about this. I find that product incredible. I think it's very important. My question to you is how do we feel about a user offering that doesn't go through a wholesale cycle and invoicing cycle, and they can sign up and get an actual testing thing in their hands, like an API key very, very quick. That's a new thing for us. That's a new thing for us. One of the things folks don't know when I say LFE, let's fucking execute, the idea is that we are focused and intentional about how we build and how we go. And I think that focus part, and of course we learn.

So the focus part is we didn't want to get ahead of ourselves. Okay, that's fair. Like, you know, in order to do that, you need to have the right security controls. And so there are a lot of things that we built. And also we knew what were the problems. And candidly, I'm actually happy we didn't have it before because the pain for Forge has actually only evolved in the last 12 months. So we probably would have built something else and maybe we were not as ready. But we always knew we would get there. Okay? Yeah. But it's important for us to build great products, not just throw something out there. and I'm very proud with the result because it's not, again, it's not something we made up. On stage I was talking about it. In the ClusterMax report that was published last week, that was something that was known as like that we have a lot of experience and data with what's happening in this space. So this is like, I have goosebumps. It's amazing. I have goosebumps about this as well. I want to go and directly offer some of the services

to the people because you know what? They hear about CoreWeave. doing this deal with Entropic, and doing this deal with Openia. And all of the 9 out of 10, we say, right? 9 out of 10, like the big labs run on CoreWeave. And that's incredible. For the engineer that wants to test on stuff, that's not something they used to be able to get. They don't understand the Platinum in the CoreMax analysis. And now they will be able to. They will be able to just go approach and try via Forge. And it's important to call out that the ClusterMax report has changed. It's not like it used to be the first time. So every time the criteria changes, the bar is higher. Like in this time, I think they had several vendors that actually went down a tier because the bar is higher. So we keep raising the bar and we keep doing that. And yes, now they can definitely, everybody can access it. And, okay, another thing about the future that I'm optimistic about. Yeah, let's go. On stage, I had Brendan Huthman from Caterpillar, VP of Physical AI.

Caterpillar is a 100-plus-year-old company, by the way. it's a very old like services thing and everyone will be but when we like prepped for the conversations he said that they originally tried to like partner with someone and walk and they understood that what they really wanted to do is like to actually take their experience and monetize it like elsewhere and he was like game on we're going to do it ourselves and that's how they partnered with us they bring 100 plus year experience, data. From all the machines and all the sites They know the customer the best way And now okay that was the point that I made And what excites me is that I feel like AI can be used for problems that existed forever And we didn't have the tools to solve. It can be in health. It can be in education. It can be in manufacturing. So that's what excites me. You know, Mike was doing today the...

Mike Trader, the CEO of COVID. Yes, sorry. Mike Trader, the CEO of COVID. He did the hot take with Sean on stage. And that was something that he was also talking about. Those possibilities, how it's going to change things. And it's not changing like, you know, let's take my experience before with Cloud Native and Kubernetes. It was still a technology change. now it's impacting any part of our lives that's exciting I think it's absolutely exciting the whole thing we do on Search.AI is bring positivity about AI and I don't know if you remember last year during the offsite I stood up a couple of times and asked all of the execs their position about AI and doomerism and stuff like that back then I really loved what I heard and what I'm hearing now because I think that's very important specifically because for the past year,

there's a lot of change in public opinion about AI, specifically from myths about water use, et cetera. And I think that folks need to hear this positive message, and I think it's coming a little bit from the assistance world. So maybe we'll finish on this, and I would love to hear from you personally. First of all, how do you view this kind of switch toward the AI system, the Muses, the Dots that OpenAI just released, the Grogbots, kind of like the 24-7 assisting AI, and how that potentially, like, you know, how we can help send this message better to people about the positivity. What people are concerned about in the context of AI is valid. Okay, people are worried about, and we talk about security, we talk about my data and sharing that and expertise and innovation. All of those kind of things are definitely valid. What I do want to send maybe a positive note on that even before those assistants and agencies that what I'm hearing in the industries and across every industry is that if before there was a lot of like you had to maybe push something into the teams and you have to use it.

And what's happening now, even before that, there's a pool, meaning teams in every company, they want to experiment and try and understand it by not providing the tools, no matter if it's a Caterpillar or a Masterclass or a Canva or they want to use it. So that's like from a positive perspective. I think that companies like ours can help in making AI accessible in a secure way with guardrails and releasing some of those concerns. And we will probably see that happening, right? Always in those kind of technologies. Like you see things happening on the personal level as a consumer and then things are coming to the enterprise and becoming more. It should probably happen in a few months because that's how things are going. So this conversation probably will not be relevant in six months. That's why we're a weekly show, Hans. But maybe it will be interesting. Let's put a reminder for us in six months to listen to this episode and see what has changed.

And see where we are. I'm going to ask you a question right now. I'm going to ask you in six months. How many AI assistants do you have running right now? I know you have human assistants. Shout out to them. Thank you for making this possible, Audrey. and I have some personal and some at work. Do you expect the army to grow in six months significantly? Or are you getting to the limit of how much attention we can spread around them? Because I'm getting to my limits. I'm feeling that. Again, we were talking about that also on stage today that the more tokens you use, the less you sleep. Oh my god. Because you feel like if you used to be a bottleneck for people, now you're bottleneck for your agents and they can keep doing things while you're not there not true that's great I also think about you know not being wasteful I have other thoughts in that regard I hope

that we will see some kind of consolidation and more efficiency also with how people are using agents versus everybody building their own all the time. Yeah. Because I don't think that's the best use of humanity. So we'll see. I love that. And to end on a very, very positive note, we were the first company to put Vera Rubin and the L72 in production, and then Cognition, which offers a software developer thing. We're the first customer. I love Devin. I don't know what your thoughts about Devin, But when I saw that, I was like, let's fucking go. This is incredible. You know what? Maybe I will say just one thing. Instead of everybody just trying to tinker everything, I think we need to help the ecosystem grow. And use products and tools that people are building and where they bring their expertise. I know within our team, we have made a lot of tools available. I know Devin is getting a lot of traction, which is cool.

And Silas was happy to see that. And just having that relationship is always great. So on a very positive note, Corby also speeds up that. And I think the stat that he mentioned was a 4.8 improvement in the tokens that they send. So this means that everybody is much faster. That's compared to the Blackwell. Yeah. So the moves of AirRubin, it's amazing. So we're now a GPU company and a CPU company, a sandboxes company and a services company. All of that in the past year. I cannot wait to chat with you in the next six months and see where we're at it would be awesome I think so and thank you so much for coming on Thursday thank you so much for having me thank you