プログラマブルなデータセンター — SF Compute の Evan Conrad が語る計算資源市場の物理的決済
The Programmable Data Center — Evan Conrad, SF Compute
プログラマブルなデータセンター — SF Compute の Evan Conrad が語る計算資源市場の物理的決済
Latent Space: The AI Engineer Podcast
要約
Latent Space が AI Engineer NYC の会場で、SF Compute CEO の Evan Conrad に再び話を聞いた回。同社は当初のクラスタ転貸から、クラウドの運用機能一式を自前で担う「物理的決済」の事業へ進んだと説明する。注文板の設計、先物カーブの形、現金決済先行への懸念、NVIDIA などが同種の市場を作れない理由、InfiniBand のオンザフライ分割による「データセンターの仮想化」まで幅広く語られた。
この回のポイント
- ●SF Compute は転貸から出発し、顧客の要望に応えるうちにホテル全体を運営する形になり、クラウドの運用機能を一通り構築した。これを先物市場の用語を借りて「物理的決済」と呼んでいる。
- ●契約の標準化はまだ遠いと Evan は述べる。代わりに制約付きの注文板を作り、売り注文とマッチングする方式を採った。流動性が分散してスプレッドが広がる代償はあるが、技術が変わる中でも取引を成立させられるという。
- ●同社の注文板では総容量の約5%が掲載されており、取引量は小さいが流動性は集中していると説明。買い手が100ドルで買った容量を転売して平均25ドル回収できることを、実用性の指標としている。
- ●先物カーブは「こぶ」型になるという。直近は売れ残りを値下げするため安く、その後上がり、遠い将来はやや下がる。B300 は推論で時間あたり約40ドル稼げるのに調達価格は5〜6ドルだと指摘している。
- ●現金決済の指数や金融商品を物理的決済より先に作ると、操作されやすくベーシスリスクも大きいと警告する。LIBOR の不正操作を例に挙げた。
- ●NVIDIA、Crusoe、CoreWeave、Lambda、推論事業者は、自社の需給や事業と利害が衝突するため中立な取引所を作れないという見方を示した。InfiniBand の分割をオンザフライで行える機能は、データセンターを仮想化してプログラム可能にする例とされる。
章立て
SF Compute の現状とホテルの比喩
転貸から始まった事業が、クラウド運用全体を担う形に広がった経緯と、物理的決済の意味を説明する。
オーダーブックと流動性の現状
総容量の約5%が注文板上にあり、取引量は小さいが流動性は集中している。転売で平均25%回収できる指標も紹介。
標準化の難しさと制約付き注文
仮想マシンの性能差などの理由で標準化は遠く、複数の制約を付けて注文をマッチングする設計を採った。
中断・ストレージ・ハードウェア世代
現在の容量か将来の容量か、中断のされ方、ストレージやWAN、初期ロットの不具合といった価格に効く要素を議論する。
契約期間とバッチ/ライブ推論、価格カーブ
最長3〜5年の契約、中断許容度による価格差、先物カーブの「こぶ」と B300 の価格乖離について語る。
物理的決済と現金決済の関係
現金決済を先行させる危険、LIBOR 操作の例、Silicon Data など周辺プレイヤーとの関係を述べる。
運用スタックと中立性
サイト訪問からクラスタ設計、バックプレーン運用まで担う内容と、ハードを所有しない理由を説明する。
NVIDIA が作れない理由とプログラマブル化
主権系コンピュートへの展開、大手が市場を作れない構造、InfiniBand のオンザフライ分割による仮想化を語る。
解説記事
AI Engineer NYC の脇で収録された Latent Space の今回の回では、計算資源（コンピュート）の取引市場を手がける SF Compute の CEO、Evan Conrad が 2 年ぶりに登場した。テーマは、GPU の契約を実際に届けるには何が必要か、そして計算資源の市場が流動的になるには何が前提になるのか、である。
転売から「ホテル運営」へ
Evan は事業の変遷をホテルにたとえる。最初はクラスタの一部を転貸しようとしていた（当時の社名は Junark）。ところが客が部屋の掃除を求め、次にルームサービス、警備を求めるうちに、結局ホテル全体を運営することになったという。そこで同社は、データセンターにアクセスできるクラウド事業者のような立場に移り、信頼できる水準で計算資源を届けるため、クラウドの運用機能一式を数年かけて構築した。これを、石油や大豆の先物で現物が実際に引き渡されることを指す「物理的決済」と呼ぶ。
石油の価格がマイナスになった例や、電力市場でボイラーの冷却に時間がかかる事情が注文条件に織り込まれる例を挙げ、現物の制約が市場設計に入り込むのはコンピュートでも同じだと説明した。たとえば「中断されてもよいが、素早く復旧できるストレージがある場合に限る」といった条件である。同社は物理的決済と注文板を同時に設計したと述べている。
標準化せずにマッチングする
市場の成熟度については、取引量は小さいが流動性は高いという。総容量の約5%が注文板に載っており、買い手が100ドルで買った容量を転売して平均25ドルを回収できることを、流動性が顧客の役に立っているかの指標にしている。
契約の標準化はまだ遠いと Evan は述べる。同じ事業者でも仮想マシンの種類によって、GPU カーネル起動の遅さなどから約40%の性能差が出うるという。そこで同社は、時間帯やGPUの条件など複数の制約を注文に付けられる注文板を作り、売り注文と照合する方式を採った。技術が速く変わる中で標準化を待たずに流動性を作れる反面、流動性が分散してスプレッドが広がる点は代償だと認めている。
価格に効く要素としては、今すぐの容量か将来の容量か、中断のされ方、ストレージの種類（Weka や VAST など）と所在、クラスタ間の広域ネットワーク、さらに H100 や GB200 の初期ロットに firmware の不具合があった例のような、出荷時期によるコホートの違いが挙がった。GPU の世代も VRAM や NVLink ドメインの違いから無視できず、一般的な計算量の単位にはまとめられないという。そのため SKU を概念化し、「Blackwell なら時間あたり6ドル以下」といった条件を顧客自身が定義できるようにしていく。
先物カーブと価格の歪み
契約期間は、1時間から最長3年まで指定できるとし、市場全体ではクラスタの融資は3〜5年が一般的だと話した。バッチ推論は1時間前や数分前の通告で中断されても構わないが、ライブ推論は数か月前の通告が必要で、SLA 違反のコストも大きい。このため中断の価格は用途で異なるという。
先物カーブの形について Evan は「こぶ」型だと述べる。直近の容量は売れ残りを値下げして処分するため安く、その後上がり、遠い将来はやや下がるという。一方で、B300 は推論に使えば最大で時間あたり約40ドル（稼働率50%なら約25ドル）稼げるのに、調達価格は5〜6ドル程度で、どちらかが変わるはずだと指摘した。
現金決済を先行させる危険
Evan が最も強調したのは、物理的決済の必要性である。業界が物理的決済を整える前に金融商品化を進めると、市場が操作されやすくなり、ヘッジ対象の指数が実際の調達価格と無関係になるベーシスリスクが生じるという。番組では CDS や LIBOR の不正操作が例に挙がった。ただし現金決済そのものは否定せず、物理的決済から導かれるべきだとの立場である。Silicon Data や Compute Exchange は現金決済側で先行した存在として評価し、パートナーになりうるとした。基準づくり、残価保険、預託金型の融資など、周辺のエコシステムも育ちつつあると見ている。
中立性とプログラマブルなデータセンター
SF Compute 自身はハードウェアを所有しない。所有すると先物市場を操作できてしまうため、「スイスのように」中立でなければならないという。その上で、サイト訪問、部品表の設計、サプライチェーンの調整、クラスタ設計、ラックの展開、PDU・スイッチ・DPU・UFM・BMC を含むバックプレーン運用まで担う。電力や土地を持つ事業者をクラウドに変える「スーパーコンピューティング as a service」で、Evan は「計算資源版の Palantir」と評されたことがあると紹介した。
NVIDIA、Crusoe、CoreWeave、Lambda、Baseten などの推論事業者が同種の市場を作れない理由は、自社の需要や事業と利害が衝突し、中立性を保てないからだと述べた。市場が機能すれば金融コストが下がり業界全体が得をするが、作る側は大型のクラウド契約や推論収益を手放す必要があるという見方である。
直近の成果として、数億ドル規模の取引量を挙げ、InfiniBand ファブリックの分割を顧客が自分で即座に行える機能を紹介した。ハードを所有していれば得られる制御をソフトウェアで提供する、データセンターの仮想化にあたるという。
まとめ
コンピュート取引は金融の側から語られがちだが、この回の主張は、まず現物を確実に届ける運用基盤が要るというものだ。日本でも GPU クラスタの調達や主権的なAI基盤が話題になっており、契約の柔軟性、中断リスクの価格付け、ハードウェアの個体差といった論点は参考になる。一方、中立性や先物カーブに関する発言は Evan 自身の見解であり、市場の実態は今後の検証が必要である。
文字起こし（英語・自動生成）
NVIDIA is obviously like the 800-pound gorilla in the room. Why didn't they make SF Compute? So our impression is that NVIDIA can't make it, but benefits tremendously from it existing. The reason why they can't make it is because they would then not be independent enough for people to trade on it. Anyway, Evan, welcome back to Layspace. Thank you so much. Welcome back to AIE. Actually, first time at AIE. I think so, yeah. This is a very special one because I feel like you have been working on compute trading, compute markets for a while. Now it's like feeling of the moment. Jump basically announced that they are getting into compute markets, which is why I'm excited to sort of cluster people. You met Oliver from Semi Analysis, who is basically initiating coverage on compute markets is my read on this. so people are going to watch your talk
I think it's a very good explanation of both your origin story and why you need to exist and not the hard problems you work on done in a very simple engaging manner I'm very jealous of your writing skills because you showed the flyer that you showed at New Reps it's pretty straightforward and simple it's good congrats I appreciate that and also congrats on AIE It seems to... Y'all are rolling. We're rolling. I just came back from the site tour for next year's AIE. And then for... A hint? At Javits Center. It's the same place they do Comic-Con. Okay. Wow. Wow. Wow. Damn. But I think the finance focus is really hitting for folks. Yeah. So we're going to try to split up. Like, this is... Here, we just lump everything in finance. Like, your finance, their finance. Like, insurance is finance. But, like, they're all super different. Yeah. So we'll probably break out things more minutely in future because people want to go deep on hedge fund stuff or banking or personal wealth.
Especially in New York. Yeah, like Anthropic was like, yeah, we're all about RIAs right now. RIAs, like Registered Investment Advisors. Yes. So different from you. No, no, no. I'm aware of this. Yeah, yeah. Okay, so for listeners who don't know, we did a podcast two years ago where you talked about where SF Compute was. Today, you're much more in this sort of physical settlement. You talk about your back fabric. What do you call it? The back plane. Back plane, which obviously takes a lot of hardcore engineering stuff. Yeah. Can you give a status update on where is SF Compute versus listeners who last heard you on a podcast two years ago? So when we started, we were this company called Junark, and then we were just trying to sublease our cluster. And we started there in the technical stack. And a way to think about it is it's like you're in a hotel and you're trying to sell that hotel room. And then somebody asks you, hey, like, are you going to clean my room? And then you say, oh, I guess I probably also need the hotel room as well.
Or, like, I need the cleaning closet. And so you go get access to the cleaning closet. And then they want room service, and so you get access to the kitchen. and then they want security and so you get access to the security closet and you get this and you do that and you do this. And at a certain point, you're just running the whole hotel. And the moment you're running the whole hotel, you say, oh, wait a second. Instead, my supplier on this market is somebody that looks closer to a fund or a cloud that has access, in our case, a cloud that has access to the data center and we're going to level up. And so what we spent the last several years doing is building the entire operating capacity of a cloud in order to actually deliver the compute at a reliable level. We call that physical settlement. We're taking a term from the futures market, from futures markets, from other markets, where in oil or soybeans or something, physical settlement means you get delivered the oil and the soybeans.
Yes. But in some markets, like in oil and also in power, there's like a delivery network. In oil, you have to have pipelines. If you would like the oil to be delivered to you, there has to be a way for it to get to you. And for people interested in financial history, during COVID, oil went negative. So I will pay you to take oil from me because it is sitting there in the port. And I can't pay for this. Yes. And so I will pay you to take my oil. Yes. That is like the weirdness of physical settlement. Yes. There's a bunch of weirdnesses like this in other markets as well. So in the power markets and the grid, there's parts of the orders that you fill out, the buy orders and sellers that are there because the boiler takes a while to cool off. And you can stop the boiler and then it will, you can start it back up again within some period of time. And so that has to get factored into the
order book. This genre thing is the same thing that happens in compute markets. So we co-designed physical settlement and the order book at the same time. And that meant that we get to actually handle the real constraints of compute, which might include something like, oh, I'm okay with being interrupted, but only if there's like storage on the thing that I can recover from really quickly. Very similar types of problems. Yeah. So people can refer to your talk and you've done a couple other podcasts already. So I want to be like the sort of 201 level instead of the 101 level. And also because I have been a derivative trader, so I can give people a bit more detail that you normally don't get into. So for example, well, how mature or deep or liquid is the computer market today? Not very mature or deep or liquid. So what we've been doing is first we had to go get a bunch of supply online in the first place, which just takes a while. And then we had to build the entire platform in order to do this. And we just now got that working. And so now we move capacity over onto the order book, which makes it liquid. And so 5% of our total capacity
is on the order book. And we're consistently moving more and more. And then we're bringing on more capacity that's all on the order book. The nice part is that maybe I should have said the volume on the actual order book is small, but the liquidity on the order book is high. So for that small amount of capacity that you have, you have very concentrated liquidity. So a sort of standard metric, I would say, for someone who is using this is a customer who buys capacity from us, and they spend $100 to buy that capacity, will make back $25 on average from selling it back to somebody else. And that's the number that we – or this is one of the metrics that we try to track, because we want the liquidity to be useful to the commercial participant, and you can measure liquidity in a bunch of different ways. You can say like, oh, I just want tons of makers or something. And you can measure it in like high-minded ways. We try to measure it by like, is this solving the problem for the customer? Yeah, got it. And so last time we talked as well, there was a lot of conversation about standardizing the contracts.
Yes. Talking with like, does CFTC or whatever, what's the status of those things? So I think standardizing the contracts in general is still pretty far away. And what we were trying to do at the time was, or basically what we learned over the last several years was we just needed to operate the whole thing. Otherwise, you cannot standardize the thing. And so to get closer to something that can be standardized, you first need to make all the operations of the thing the same. Right. To give an example of what can be different is a single operator can offer two different types of virtual machines. one type of virtual machine might have a 40% performance degradation versus another type of virtual machine. And 40% is a lot. When you say performance degradation, are you talking like MFU type things or like amount of burn-in that you've experienced or what? It will affect. So in this particular case, I'm talking about GPU kernel launches and how long it takes for that to happen, which is a process that happens quite frequently when you're doing sort of either training or inference. the
that will basically the end result is that you get a worse NFU but it's the intermediate level so for anyone who's traded commodities before there's different grades of commodity it's not oil it's light sweet crude versus whatever the opposite of light sweet crude or it's like corn level A or corn B or whatever and so people are trying to you have to establish the equivalence of those things Correct, yes. And I'm wondering if they have been established. So what we did instead was we created an order book that had multiple different constraints. And you can say, I want to have something that's between this and that. It runs from this time to this time. It's like X to Y. It's a bunch of different properties that you can attach to your order. And then we match that against sell orders that have those properties in a big matching engine. And the reason why we did it this way is it allows us to generate liquidity without having to standardize before you know what the technology can do and because the technology was shifting so quickly we wanted to basically be able to do that The trade of this is that you have less concentrated liquidity
and so there's going to be more spreads between the buy and the sell. Yeah, it sucks. But on the other hand, it means there's lots of arbitrage to make if you can do that correctly. So if you have a product that can use lots of different types of capacity, then you might really like SF Compute because you will be able to use lots of different types of capacity and you'll make more money. You can take the spread. Okay. What dimensions matter in this grid of things? So a bunch of different things. One is dimensions, categories that I will say is like, is there a workload running on it right now or is it capacity in the future? So is it capacity right now? Capacity right now, in order for you to sell that, you have to interrupt your current workload. so you have to think, do I care if my current workload is interrupted or not? Capacity that's starting a month from now, you can buy and sell that all the time, and so you get liquidity on that really easily. There is some concern like, oh, no, well, I have capacity next month, but it's very different than if you're interrupting a workload right now.
The way that the interruption happens matters. So an example is if you're on a cluster and the cluster doesn't have as amazing storage as you want it to have. So an example is like, do you have a Weco or Vast system under the hood, and do you offer that and, like, what's the size of that? This changes how you respond to being preempted or removed off the cluster. Meanwhile, the location of that storage matters. So if you're able to move that data from one cluster to another quickly, then maybe you might be able to spread across multiple different clusters, and you can be interrupted over here and then pop back up over there. to make that work. You also need like a fast transit path between one cluster and another. Typically, this is implemented with a WAN, but most of these clusters don't necessarily have... Wide area network? Yeah, a wide area network. So basically cables that connect the clusters together. A private internet is another way to think about it. A lot of clusters that are being spun up today are like, they're not attached to...
Traditional CPU clusters were next to the customer for latency reasons. In GPUs, they're next to the power, wherever the, like, you know, you put the data center next to the power rather than next to the customer. This means that a lot of stuff is getting put in places that previously they weren't expecting data centers there, and so you may not have as excellent of a transit path from one to the other. So this starts to matter. Other examples are weird shit where we theoretically support but don't actually, like, this is the thing that we're thinking about but haven't done, basically. The early GB200s broke. The early H100s broke. And so if you were buying H100s or you're buying GB200s, you might really want the ones that were deployed after a certain time because you're not getting the first shitty ones off the rack. And so if you're a super sophisticated customer, you may be willing to pay more because for that. Oh, so it's about, like, what cohort they were produced, and they get better over time?
So, like, the first ones off the rack maybe had some faults or some problem. Right. And so if you were the ones who got the first ones from the first shipment of them, then, yeah, they would be broken in some way. I assume the sort of process is, like, look, there's defects everywhere. Like, it just happens, right? Totally. And so, like, you get a new shipment from TSMC or whatever, and you just kind of run your test, and you figure out which ones are bad, and you throw them out. Isn't that how it works? I didn't know about this element of cohorting where, let's say, Q4, 2026, B300s are way worse than 2027 B300s. Yeah, so I'll give an example. You're describing burn-in. Yes. So when you get a cluster, a good cloud, what it does is it tries to kill all the GPUs before they ever get in the hands of the customer. And then the GPUs that are broken or the nodes that are broken, you get RNA'd. You basically go and request it.
You return it, and you get a new one, and there's a lead time to that. But whatever, you'll get some back. And then the customer never has to experience this. The thing I'm referring to is that, like, in early H100s, there were, like, firmware bugs that may not even be related to the specific. Like, they may not be NVIDIA. They might be, like, Supermicro has a firmware bug in the box that you bought, and it applies to all of them. And it doesn't matter. Like, the solution to that was you need to get them to update their firmware, or they need to ship a new thing in every new box. Yeah. Commission had this for, like, the B200s that we got, and, like, you had to use that in to go fix it. Yes, right. So, like, different types of problems like this can happen. Interesting. Okay, so that seems small compared to the overall. I just want people to understand the shape of this market. You mentioned preemption, and we talked about spot instances and how the analogy is.
Yeah. And then there's also booking a month out. How far out do these markets go? And is real-time or sort of preemption more interesting to you versus being further out? To answer your first question, three to five years. Wow. we're starting to be... And, like, month by month or quarter by quarter or how's this? Are you asking how the cash flow works? Like, the contracts, like, the futures contracts, right? Like, again, like, I'm a former contracts, a derivatives trader. In the market itself, like, is in the broader market, clusters are financed three to five years. Okay. So the payments are available over that period of time. Yeah. There is also, like, blocks of capacity in our order book, which can extend any arbitrary time. Yeah. There's some stuff that we have to do for the cash flow on that. Right now what we do is we just ask people to pay entirely upfront for a block. We don't ask them to do that for a lease for a longer term contract. This gets into something that I definitely need a whiteboard to explain. But you had another question
I was about to answer. I forgot. Do you want real time or do you want more? What's more interesting to you, real time or longer out? So we care more about longer outs because we're trying to build a forwards curve that has a lot of liquidity on it. these are kind of the same thing and kind of not. So what we do is we can say you can buy an hour out. You can buy right now to the next hour. Or you can buy right now to the next three years and anywhere in between. That's wild. I know. And so the, you know, or finally, technically, Um, the, there's a certain difference that happens that's dependent for each customer. Basically, like, if I have a workload that is, um, uh, that's a batch inference, basically, where I can be interrupted at any point, I don't really care, um, then my ability to be cut off, um, is probably, like, I need an hour of time to know, um, that I'm going to be cut off,
and then I'm good. Or I can even be notified in, you know, a minute or two. But if I have live inference customers who are expecting me to be online, then you can interrupt me, but in a month, maybe two months, maybe three months. And then I can adapt to that capacity over that period of time. But you cannot interrupt me in a minute. And so those two things are priced differently. And the reason why they're priced differently is because what is the price you would pay in order to interrupt your batch job, it's very different than the price you would pay to interrupt your live job. If you have a live job, you are probably paying an SLA when you break for your customer. Does that make sense? Yeah, I see. So this is actually the market for breaking leases. These are the market for breaking leases, basically. Oh, my God. And then there's also the market for future leases, or like for leases that are... Oh, I thought this was like a subletting situation. This is like... Oh, the lease break. So one way to think about it is in power markets, there's actually two markets.
There's like the day ahead market and there's the live market. Compute kind of has the same thing where it has capacity that is coming online, or capacity that's about to arrive to you, and then capacity that is currently in use. And these have different price dynamics. Okay. Interesting. Backwardation or contango? Backwardation or contango? So is compute further out price higher than spot or lower than spot? Is compute further out price? Does it go like this? Does it go like that over, you know, over like one year out, two year out? Do oh this is both at different times I find it fascinating that you don know Yeah I mean both It depends on who you are Okay So I make the case Wait so you asking is compute the same unit of compute today Maybe not today. Maybe a month out versus one year out versus two year out. Is that curve going up or going down? We see like a hump.
Oh, my God. Okay. So explain. Like, I think this will make intuitive sense. is basically like, if I have milk and it is about to expire, I will lower the price on it. And so that's basically what we see. So it's the NVIDIA release cycle, basically, right? Well, we're more saying like, if there's a block of compute and it's approaching now, it can't be sold in the past. And so it has to drop its price in order to clear at some price. And so initially you see this like drop where stuff that's sold right now, anything that's on the market right now is something that hasn't been sold. And so therefore the price will drop. And then it goes up to where it wants to be. And then maybe compute in the future is worth less because people don't want, like, people would like to compute now. And so they're sort of paying for the shorter lead time. And they pay less for the compute that's off into the future. This is what I think a normal healthy curve looks like. the place where I'm like, oh, man, no shit, is that right now, if you took a B300 and you tried to run inference on it,
you can make $40 an hour or so selling that as inference. Any model, basically. And so that's if you use it at 100%, and maybe if you only have, like, 50% utilization or something, so, like, $25 an hour. Yeah. So $25 an hour. Meanwhile, you can buy B300s at, like, $5 to $6 an hour. One of these numbers has to change. Yeah, it's a huge overcharge. You mentioned this in your talks a lot. Yeah, so a bunch of people are basically looking at this and thinking, if I think prices are going to go up, I might be willing to pay more than what I previously thought in the future because I might think, okay, I can buy a five right now, but if I can't buy any more, then maybe I will be willing to buy even more in the future just to lock it in if it's going to come online later. So we see the other side as well. Kind of depends on who you are. Cool. So, is it, okay, and I just really want to make super clear on this. Of course. Do you care about the different generations of chips, or do you just care about overall flops, and you spread it among whatever chips you want? We care about different generations of chips because...
Yeah, you care. I guess your customers care. Yes. Right. Oh, wait, when you say different generation chips, do you mean like... So, like B300s, 800s? Yeah, exactly. Yeah. Or like even AMD or whatever, right? Yes, I think people totally care about the generation of chips. I think part of this is because of things like VRAM and part of this is because of larger NB-length domains and things like that. So there's just no way to abstract all this to some generalized unit of compute that you can just mush around whatever you like. I think when you were saying even oil isn't oil. Oil is actually just a bunch of different types of oil. Yeah, like jet fuel or car fuel, whatever. I think it's the same thing. You're going to get a bunch of different types. It is, but you have oil refineries. Effectively, you have black boxes. because you can put oil in and get whatever out. If computed is so heterogeneous, I guess is the word, you just have to expose the underlying complexity to your end customer, which is bad for you as a market maker. Maybe. You want to identify the one choke point,
or I don't know what the better word for choke point is, like a shelling point, where everyone agrees, okay, this is kind of the standard, and we'll all agree to this standard, and it'll be somewhat inefficient. Yes. So we've been able to reduce the problems to a certain level. Yeah, your job is dimensionality reduction. Yes, we reduce a lot of the dimensions, but nonetheless, there are still dimensions. It's still going to exist. Do you know how many SKUs you have? So like five times 100 times like 5,000 equals, I don't know. We actually made SKUs like a concept, and the goal is to make it so that somebody else can make their own SKU. So if I want to buy Blackwells, I can say I'm willing to buy Blackwells at $6 an hour or less, and you can match me against GB300s or you can match me against B300s, and I don't care which Blackwell, as long as it's less than $6 an hour. But you just said you care about memory and other stuff.
Totally, but not every customer does. So some customers just can work on BlackHalls, whatever it is. Some customers are super hyper-specific, and they need this exact thing, and they're going to pay more for that because the market's going to charge them more. Okay. And that's what we do. So people use your APIs to programmatically define what they want, and then you match that with whatever composition of sellers they might be able to fulfill. Yes. With the caveat that having a named query is the new thing we're doing, but the query system as a concept is how the order book works right now. Okay. Interesting. All right. So, you know, I think that's like a lot of, I guess, nitty gritty. Are there anything about, I guess, that you want to educate people on, whether it's financial people or engineers, about the state of the compute trading market as you've developed it? Because effectively you guys are defining it. Mainly just the requirement for physical settlement. Yeah. We think that if you – we spent three years learning why you need physical settlement. and our entire conversation today is that it is,
these things are not as fundable as you think. We're going to try and make them fundable. That's the industry's job to do that. But without that, you run a lot of risks. If you try to build too much financialization before the industry has had time to build out the physical settlement, then I think what you have is you have a market that's very easy to manipulate and a market that has a lot of basis risk, where the price you may be hedging against actually has nothing to do with the price that the market... Yeah, yeah, yeah. You can actually buy the capacity for. Yeah, so for listeners, we talked about this briefly last time, but people are often shocked to find out that, let's say, the market for credit defaults is 12 times the size of the bonds that they insure. something like that, or the markets for mortgage reinsurance or all that is larger than the underlying physical settlement. So that's why physical versus cash settled. And basically cash settlements are exactly the kind of side bet that you see in the big short,
where they may or may not have anything to do with the underlying thing itself, and it may be used as a proxy for overall other stuff. And you don't want that to go out of control? Yeah, so to be clear, we think cash settlement as a future concept is a great idea. We just think it should be derived from physical settlement because, as you said, it's a side bet. And it would be kind of like if you made a cash settlement future before you made the physical settlement, which I think people are kind of doing already. Yeah, totally. People are trying to make cash settlement before you make the physical settlement. People do that for all sorts of things, including startup stock. Yes. Right, like you can buy and sell a made-up share of OpenAI, but it's nothing to do with the actual share. Yes. And lots of those go very poorly. Like, it would be kind of bad if you made a – it would be bad if you started selling the derivative on a stock that doesn't exist yet. Yeah, yeah. So, I mean, one thing that we do – so I used to be a currency trader, right, and interest rates as well.
And one thing we do is you have a board that gets together and says, like, this is the London interbank offered rate, LIBOR. And we all agree, like, this is the monthly fixing, and we all trade to that fixing. Yes. Is that even meaningful at some point in three years from now? So critically, LIBOR was manipulated. Yes. Our concern at the moment is – Because there's 12 banks. Yes. Well, our concern is there's a very similar problem where you can manipulate an index. our impression is why you can manipulate that index is because a lot of the indexes are based on a very small amount of data that has not a lot of correlation with the actual price of buying a new reserve contract. That's what we're concerned about. Okay. Interesting. All right. I think that's the rough state of things. I think the other part of just today is you talked with Oliver, and you're doing an event with Jump and you're friends with Silicon Data. What are you learning from all these other players?
Who are the major players that you would want to call out as doing interesting things? Ooh. So we're big fans of Silicon Data and Compute Exchange. I think they were also very early to this, and I think they are doing a reasonable shot at any path that we think that we'll likely end up being partners with. Can you distinguish your different paths? So the generalist is they doing cash settlement We doing physical settlement We think both can exist and should exist And that they sort of mutually beneficial to each other And so totally fine So I think they are one of the earliest in the space. And I think they are, they're clearly took the bet well before anybody else did. And I think that's like very honorable to do. You mentioned Semi-Analysis, Jump. These are the players emerging. Like you were my first introduction to this. And then I see all these other guys coming in and I'm like, okay, like, something's happening here. I don't know the landscape as clearly as you do. Yeah, so the categories of folks that we've seen is there's folks that are trying to set better standards.
So Semi-Analysis, I think, is working to try and create better standards. There's folks that are trying to do residual value insurance and so on. There's a number of different companies in this space. I don't know any of them super well, but I know this is a concept, and I think that's important. There are folks that are trying to do deposit lending where they'll front your credit risk for the cloud. There's an ecosystem of companies that are forming around how do we resolve the financial bottlenecks of compute, and it is cool to see. I think effectively what you want in this ecosystem is buyers and sellers. You want an analyst. You want, we said insurance, or what's the? Residual value insurance. Residual, yeah. And we want this healthy ecosystem out there. But obviously, you guys are owning a problem of physical settlement, which is really interesting. Okay, so we can go into the hard tech slide because obviously you are also hiring for extremely cracked engineers. Describe your stack. We do all the things the cloud does
without being the cloud. As in, we don't own the hardware. The reason why we don't own the hardware is because if we did, we would be able to manipulate the forwards market. So we have to be Switzerland. We have to be independent, which creates a lot of sort of strange things about the company. But in order to do physical settlement, you have to do site visits. You have to visit the data center. You have to design the bill of materials and ensure that you have to basically do like supply chain consulting, where you have relationships with different OEMs that ensure that things get delivered on time. Then you have to do cluster architecture. So you're sort of designing the cluster itself. then you're rolling out racks or you're working with partners to roll out racks and supervising that. Then you're doing backplane management, so you're taking over the PDUs and the switches and the DPUs and UFM and BMC, and you're operating the whole harness around the nodes that you're selling.
When you say you as an SF Compute employees? Yeah. Wow. So when you buy a node from a cloud, there's the bare metal, but there's all this shit around the bare metal. Yeah, yeah. and in order to make that bare metal node work, you have to operate everything else, then we... Because I just, like, traditionally the job of the data center is to extract some of this away from you. Yeah. We're the ones that do that. Yeah, yeah. But you're not in the data center business. You're not in the sort of equinaces of the world. We're not in the data center business, sort of. What we do work with is... So we offer supercomputing as a service. And what this means is folks come to us and say, you know, I've got X, where X is, I've got, you know, X megawatts here. And maybe sometimes they mean I've got a data center that's, like, I would like to turn into a GPU cluster. Like, my options are I could sell the data center to somebody, or I could become the cloud.
And then we go to them and say, great, we will turn you into the cloud. We're the supercomputing part. You're the data center part. Let's partner together, and then we will sell and I'll make money. Um, sometimes it's people who have something that could turn into a data center, in which case we're going to basically project manage that up to the layer. Um, and then we will work with those far, we'll work with those folks to turn their powered land, um, or whatever they have, um, into a data center by working with partners. Um, and then we take over at the supercomputing part. Okay. This requires us to have some sort of data center expertise in house, but we don't self perform builds. So, yeah. Yeah. So it's somewhat modular because you have different scenarios you need to deploy under. Yeah. Someone once described us as like Palantir for compute because it's like each time it's this new rodeo. Yeah. And we deeply partner with folks on both sides. Yeah.
And primarily the U.S.? Primarily the U.S., but we operate around the world. Yeah. At some point, does SF Compute become a misnomer? So we're deploying clusters all around the world. And when we do that, we're working with people who themselves are trying to be clouds all around the world. They look like sovereign compute players. And our pitch to sovereign compute is amazing. Let's help you set up a cloud in your own country. And we will bring the San Francisco offtake. We'll bring the San Francisco expertise. We'll set up the clusters. That's our whole business. We're going to make it. Who are your biggest announced partners that you can name? Nobody I can name. Really? Wow. We will have some announcements in a week or two that we expect to come on. Yeah. All things sort of. Sure, sure. Good luck, yeah. No, I mean, I think this is really interesting for every country who's investing in software and AI to figure this out. and it's interesting that SF is still sort of like that brand name
that is very marketable. So I think one of the interesting things is just thinking through this market. NVIDIA is obviously like the 800-pound gorilla in the room. Why didn't they make SF Compute? And again, the answer is going to be market manipulation, but let's just really get into the incentives of the multiple players. Yeah, so our impression is that NVIDIA can't make it but benefits tremendously from it existing. The reason why they can't make it is because they would then not be independent enough for people to trade on it. If you don't trade on it, then you don't have enough liquidity for lenders to finance against the forwards market. This is also why Crusoe and Corrive and Lambda, why the hyperscalers can't make it, why Base 10 and Fireworks and Modal can't make it, because they have too much demand on that side. This is a thing that benefits the entire industry because it makes the financing costs go down. But nobody can make it unless they give up the ability to do all the big inference money and all the big cloud deals.
You have to kill your business in order to make this. That's why nobody does it. It's like structurally they're not set up to do that. Yeah. And you're basically reducing friction, and that's all your revenue. You've announced some milestones. Actually, I forgot to say that up front. Can you repeat what they are and just take a victory lap a little bit? We closed several hundred million dollars of volume. We have a significant number of new clusters on the way. And then the things that I think I'm most excited about is we've shipped a whole bunch of new features that make SF Compute very viable for platform-as-a-service companies. An example is we allow customers to subdivide their InfiniBand fabric without having to ask us. So many clouds, when you go to buy them, they will give you one Infiniband fabric. And then if you want to redraw the lines of that Infiniband fabric, you need to contact them and they will then do something for you.
We now can do that on the fly for customers, which I think is really great because it interfaces with the market really well, where you want to be able to redraw things in order to scale up and down your clusters. Would you describe this as like you're virtualizing the data center and making it programmable? Yeah, we're virtualizing the data center and making it programmable. So all the stuff that you would have gotten if you had owned the hardware, we're trying to virtualize that, which I think is exciting to us. It's funny, like the last time this happened was the cloud, which is really cool. And then finally, I think, what is your call to action for people? Is it engineers? Is it giving your supply? What is it? If you would like to own a GPU cluster and you want the residual value from it, you want to have it for your portfolio, we would love to work with you in order to build that GPU cluster. We will take over and we will do the supercomputing portions of that and ensure the things go right. We own the SLA. And then if it turns out that that was a bad idea that you need out of that cluster,
we can immediately sell that cluster to somebody else so it doesn't hit your balance sheet. Very simple pitch. Well, thank you so much for your time. You're doing a great service to the industry and I just am happy to be your friend. I appreciate it. All right. Thanks, sis. .
番組の概要欄（原文）
This was recorded on the sidelines of the new Compute Trading track at AIE NYC. Catch us next at AIE CODE, our last flagship for the top agentic engineers in the world. In the early days of cloud, former guest Martin Casado made something so disruptive that Cisco was threatened enough to pay $600m for it, and VMware outbid for $1.3B: That concept was software-defined networking, which virtualized the routing network by separating the control plane from the data plane. That revolution is now coming for GPUs - where previously running your own supercompute cluster required $50-100m upfront to get commitments and do the entire stack from hardware to firmware to kernels, SF Compute is now enabling you to not only buy or sell GPU hours now and in the future, but also to programmatically control everything from your network to memory. We last spoke with Evan about commoditizing compute. At AI Engineer New York, we went deeper: what does it take to physically deliver a GPU contract, and what has to be true before a market in compute can become liquid? From selling a room to running the hotel Evan describes SF Compute’s expansion with a hotel analogy. A customer wants a room, then asks for cleaning, room service, and security. Eventually you discover that delivering the room means operating the hotel. SF Compute began by trying to sublease a cluster; the company has since built much of the operating capacity of a cloud to deliver the compute traded on its market. That work is physical settlement: the buyer gets usable compute, rather than simply a cash payment against an index. It brings the physical constraints of a cluster into the market itself. Can a workload be interrupted? How quickly can it recover from storage? Which GPU generation, networking and software configuration does it need? Those details affect both the contract and the price. Making compute tradeable Rather than waiting for every GPU cluster to become interchangeable, SF Compute matches orders with different constraints. Evan argues that this makes trading possible while the technology changes, at the cost of spreading liquidity across more variants. We also discuss spot and forward pricing, batch versus live inference, and why an index can become disconnected from the cost of actually buying compute. Independence is central to Evan’s case for a market operator. He argues that companies with large hardware or inference businesses have incentives that make it harder to run a neutral exchange. We also get into the unglamorous engineering: visiting sites, designing clusters, running the backplane, and making infrastructure programmable. Near the end, he describes the ability to subdivide InfiniBand fabric on the fly as one example. Guest: Evan Conrad, CEO of SF Compute. Show notes * Evan’s AI Engineer NYC talk: How Compute Markets Work * Our previous SF Compute conversation * Why oil futures went negative in 2020 — EIA * How SF Compute customers resell unused GPU capacity * SF Compute’s $245M compute sale * Abel: SF Compute’s settlement ledger * CFTC: Barclays and LIBOR manipulation * Elastic InfiniBand partitions Timestamps * 0:00 — Why can’t NVIDIA build this? * 0:25 — Welcome back to Latent Space * 2:23 — From selling a room to running the hotel * 3:47 — Physical settlement: actually delivering compute * 6:00 — Liquidity and the GPU order book * 8:42 — Trading nonstandard compute * 10:00 — Interrupting and moving workloads * 12:20 — Why hardware cohorts matter * 14:32 — How far out do compute contracts go? * 16:19 — Batch versus live inference leases * 17:58 — The shape of compute prices * 20:33 — GPU generations, specifications and SKUs * 23:30 — Physical versus cash settlement * 27:00 — The emerging compute market ecosystem * 29:03 — What it takes to deliver a GPU cluster * 32:24 — Sovereign compute * 33:26 — Why NVIDIA can’t build this market * 34:48 — A programmable data center Transcript Automated transcript, lightly formatted; speaker labels have not been verified. 00:00 NVIDIA is obviously like the 800-pound gorilla in the room. Why didn’t they make SF Compute? So our impression is that NVIDIA can’t make it, but benefits tremendously from it existing. The reason why they can’t make it is because they would then not be independent enough for people to trade on it. 00:25 Anyway, Evan, welcome back to Latent Space. Thank you so Welcome back to AIE. Actually, first time at AIE. I think so, yeah. This is a very special one because I feel like you have been working on compute trading, compute markets for a while. Now it’s like feeling of the moment. Yeah. Jump basically announced that they are getting into compute markets, which is why I’m excited to sort of cluster people. You met Oliver from SemiAnalysis, who is basically initiating coverage on compute markets is my read on this. So people are going to watch your talk. 01:00 I think it’s a very good explanation of both your origin story and why you need to exist and all the hard problems you work on. Yeah. Done in a very simple, engaging manner. I’m very jealous of your writing skills because you showed the flyer that you showed at New Reps. Yeah. I was like, it’s pretty straightforward and simple and it’s good. Yeah. So anyway, congrats. Thank you. I appreciate that. And also congrats on AIE. It seems to, y’all are rolling. We’re rolling. I just came back from the site tour for next year’s AIE. then for You have like a hint? At Javits Center. It’s the same place they do Comic Con. Okay. Wow. 01:35 Wow. Wow. Damn. But I think the finance focus is really hitting for folks. Yeah. So we’re going to try to split out. Like this is, here we just lump everything in finance. Like your finance, their finance, like insurance is finance, but like they’re all super different. Yeah. So we’ll probably break out things more minutely in future because people want to go deep on hedge fund stuff or banking or personal wealth. Yeah. Especially in New York. Yeah. Anthropic was like, yeah, we’re all about RIAs right now. Yeah. RIAs like registered investment advisors. Yes. Like so different from you. 02:10 No, no, no. I’m aware of this. Yeah. Yeah. Okay. So for listeners who don’t know, we did a podcast two years ago where you talked about your, you know, where SF Compute was. Um, today you’re, you’re much more in a sort of physical settlement. You’re talking, you’re talking about your, your back fabric or what do call it? Uh, the backplane. Backplane. Yeah. Which obviously that takes a lot of hardcore engineering stuff. Yeah. Um, can you give a status update on like, where is SF Compute versus listeners who last heard you on a podcast two years ago? So, uh, when we started, we were this company called Junark and then we were just trying to sublease our cluster. 02:45 Um, and we started there in the technical stack. Um, and a way to think about it is it’s like you’re in a hotel and you’re trying to sell that hotel room. And then somebody asks you, Hey, like, are you going to clean my room? Um, and then you say, Oh, I guess I probably also need the hotel room as well. Or like I need the, um, the cleaning closet. And so you go get access to the cleaning closet and then they want room service. And so you get access to the kitchen and then, you know, they want, um, security. And so you get access to the security closet and you get this and you do that and you do this. And at a certain point you’re just running the whole hotel. Um, and the moment you’re running the whole hotel, you say, Oh, wait a second. 03:21 Um, instead my supplier on this market, um, is somebody that looks closer to a, fund, um, or a cloud that has access in our case, a cloud that has access to the data center. And we’re going to level up. And so what we’ve spent the last several years doing is building the entire operating capacity of a cloud, um, in order to offer, in order to actually deliver, uh, the compute at a reliable level. Um, we call that physical settlement. We’re taking a term from the futures market, from futures markets, from other markets, um, where in, you know, 03:57 oil or soybeans or something, physical settlement means you get delivered the oil and the soybeans. Yes. Um, but, um, in some markets, um, like in oil and also in power, um, there’s like a delivery network in oil. You have to have pipelines. If you would like the oil to be delivered to you, there has to be a way for it to get to you. Yes. Um, and finance, and for people interested in financial history during COVID oil went negative because, uh, so like I will pay you to take oil from me because it is sitting there in the port. Uh, and like, I can’t pay for this. 04:32 Yes. And so I will pay you to take my oil. Yes. That is like the weirdness of physical settlement. Yes. There’s a bunch of weirdnesses like this in other markets as well. So in the power markets and the grid, um, there’s parts of the orders that you fill out, the buy orders and sellers that, um, are there because the boiler takes a while to cool off. Um, and you can, you know, stop the boiler and then it will, you can start it back up again within some period of time. And so that has to get factored into the order book. Um, this genre thing is the same thing that happens in compute markets. So we co-designed physical settlement and the order book at the 05:10 same time. And that meant that we get to actually handle the real constraints of compute, which might include something like, oh, I’m okay with being interrupted, but only if there’s like storage on the thing that I can recover from really quickly. Um, very similar types of problems. Yeah. So, uh, people can refer to your talk and you’ve done a couple other podcasts already. Um, so I want to be like the sort of 201 level instead of the 101 level. Sure. Uh, and also because I, I am, I have been a derivative trader, so I can give people a bit more detail that you normally don’t get into. Um, so for example, well, how, how mature or deep or liquid is the compute 05:45 market today? Uh, not very mature or deep or liquid. Okay. so what we’ve been doing is first we had to, we had to go get a bunch of supply online in the first place, which just takes a while. And then we had to build the entire platform in order to do this. Um, and we just now got that working. And so now we move, um, capacity over onto the order book, um, which makes it liquid. Um, and so 5% of our total capacity is on the order book and we’re consistently moving more and more. And then we’re bringing on more capacity. That’s all on the order book. Um, the nice part is that, um, maybe I should have said the, the volume on the actual order book is small, but the, 06:21 um, the liquidity on the order book is high. So for that small amount of capacity that you have, um, you have, um, like very concentrated liquidity. So, um, a sort of standard metric, I would say for someone who is using this is a customer who buys capacity from us and they spend a hundred dollars to buy that capacity. We’ll make back $25 on average from selling it back to somebody else. Um, and that’s the number that we, or this is one of the metrics that we try to track because, we want the liquidity to be useful to the commercial participant and you can measure liquidity in a bunch of different ways. You can say like, I just want tons of makers or something. 06:56 Um, and you can measure it in like high-minded ways. We try to measure it by like, is this solving the problem, um, for the customer? Yeah. Got it. And so, uh, last time we talked as well, there was a lot of contract, uh, conversation about standardizing the contracts. Yes. talking with like, uh, the CFTC or whatever, what’s the status of those things? Um, so I think standardizing the contracts in general is still pretty far away. Um, and the, what we were trying to do at the time was, um, or basically what we learned over the last several years was we just needed to 07:31 operate the whole thing. Otherwise you cannot standardize the thing. And so to get closer to something that can be standardized, you first need to make all the operations of the thing the same. Right. To give an example of what can be different is a single operator can offer two different types of virtual machines. One type of virtual machine, um, might have a 40% performance, um, degradation versus another types of virtual machine. And 40% is a lot. When you say performance degradation, are you talking like MFU type things or like amount of burn in that you’ve experienced or what? It will affect. So in this particular case, I’m talking about GPU kernel launches and how long it takes for that to happen. 08:07 Um, which is a process that happens quite frequently when you’re doing sort of either training or inference. Um, the, um, uh, that will basically, the end result is that you get a worse MFU, but, um, like is the intermediate level. the intermediate level basically. Yeah. Yeah. So, so what I’m trying to, so for anyone who’s traded commodities before, there’s different grades of commodity. Yes. Right. There’s like, it’s not oil. It’s light sweet crude versus like whatever the opposite of light sweet crude is, or it’s like, you know, uh, corn level a or corn B or whatever. Uh, and so like people are trying to, well, you have to establish the equivalents of those things. Correct. Yes. I’m wondering if they have been established. 08:42 So what we did instead, um, was we created an order book that had multiple different constraints and you can say, I want to have, um, you know, something that’s between this and that. Um, it runs from this time to this time. Um, it’s like, uh, X to Y, it’s a bunch of different properties that you can attach to your order. And then we match that against sell orders that have those properties, um, in a big matching engine. Um, and the reason why we did it this way is it allows us to generate liquidity without having to standardize before, know, what the technology can do. Um, and because the technology was shifting so quickly, um, we wanted to basically be able to do that. The trade off of this is that you have less concentrated liquidity. 09:18 And so the prices are going to have, there’s going to be more spreads between the, the buy and the sell. Yeah. Um, it sucks. Yeah. But on the other hand, it means there’s lots of arbitrage to make, um, if you can do that correctly. So if you have a product that can use lots of different types of capacity, um, then you might really like SF compute because you will be able to use lots of different types of capacity and you’ll make more money. Um, you can take the spread. Okay. What dimensions matter in this grid of things? Um, so a bunch of different things. Um, one is dimensions categories that I will say is like, is there a 09:53 workload running on it right now or is it capacity in the future? Sure. is it capacity right now? Um, capacity right now in order for you to sell that, um, you have to interrupt your current workload. So you have to think, do I care if my current workload is interrupted or not? Capacity that’s sold that’s starting a month from now, um, you can buy and sell that all the time. And so you get, um, liquidity on that really easily. Um, there is some concern like, oh no, well I have capacity next month, but it’s very different than if you’re interrupting a workload right now. Um, the way that the interruption happens matters. So an example is if you’re on a cluster and the cluster doesn’t have as amazing storage as you want it to have. Um, so an example is like, do you have a Weco or Vast system under the hood? 10:31 Um, and do you offer that? And like, what’s the size of that? Um, this changes how you respond to, um, being preempted or removed off the cluster. meanwhile, the location of that storage matters. So, um, if you’re able to move that data from one cluster to another, um, quickly, then maybe you might be able to spread across multiple different clusters. Um, and you can be interrupted over here and then pop back up over there. Um, to make that work, you also need, like a, a fast transit path between one cluster and another. Typically this is implemented with a WAN, but most of these clusters don’t 11:06 necessarily have, um, like a wide area network. Yeah. A wide area network. So, basically cables that connect the clusters together. A private internet is another way to think about it. uh, a lot of clusters that are being spun up today are like, they’re not attached to, um, traditional CPU clusters were, next to the customer for latency reasons. In GPUs, they’re next to the power, um, wherever the, like, you know, you put the data center next to the power rather than next to the customer. Um, this means that a lot of stuff is getting put in places that previously they weren’t expecting data centers there. 11:41 And so you may not have, um, as, um, excellent of a transit path, um, from one to the other. So this, this starts to matter. Um, other examples are, um, weird s**t where we theoretically support, but don’t actually, um, uh, like this is a thing that we’re thinking about, but haven’t done basically. Um, the early GB 200s broke. The early H 100s broke. Um, and so if you were buying H 100s or you’re buying GB 200s, you might really want the ones that were deployed after a certain time because you’re not getting the first shitty ones off the rack. 12:16 And so if you’re a super sophisticated customer, you may be willing to pay more, um, because for that. Oh, so it’s, it’s about like what cohort they were, they were produced and they get better over time. So like, uh, the first ones off the rack maybe had some faults or some problem. Right. And so if you were the ones who got the first ones, um, or, you first ones from, um, the, the first shipment of them, um, then yeah, they would, they would be broken or, um, in some way. I assume the sort of process is like, look, there’s defects everywhere. Like it’s, it just happens. Right. Totally. And so like you get it, you get a new shipment offer from TSMC or whatever, and you just kind of run your tests and like you, you figure out 12:53 which ones, which ones are bad and you throw them out. And isn’t that how it works? And I, I didn’t know about this element of cohorting where like, let’s say Q4, 2026, uh, B300s are way worse than 2027 B300s. Yeah. So, um, I’ll give an example. Um, you’re describing burn-in. Yes. so when you get a cluster, um, a good cloud, what it does. Is it tries to kill all the GPUs, um, before they ever get in the hands of the customer. Um, and then, um, the GPUs that are broken or the nodes that are broken, um, you get RMA’d. You basically go and request it. Yeah. You return it and you get a new one and there’s a lead time to that, but 13:30 whatever, you’ll get, you’ll get some back. Um, and then the customer never has to experience this. Um, the, um, thing I’m referring to is that, um, like in early H100s, there were like firmware bugs that may not even be related to the specific, like they may not be NVIDIA. They might be like Super Micro has a firmware bug in the, um, in the box that you bought. Um, and it applies to all of them and it doesn’t matter. Like, um, the solution to that was you need to get them to update their firmware. Um, or they need to ship a new thing, um, in every new box. 14:07 Yeah. had this for like the, the B300s that, that we got. Um, and the IQ had to use Devin to go fix it. Yes. Great. So like different types of problems like this can happen. Interesting. Okay. So I’m like, okay, look, that seems small compared to the overall. Like, I just want to, like, I want people to understand that the shape of this market, um, you mentioned like preemption, you know, and, you we talked about spot instances and how, how that, uh, the analogy is. Yeah. And then there’s also, you know, booking a month out. How far out do this, do these markets go? And is, is real time or sort of preemption more interesting to you versus, 14:42 far being further, further out? Um, to answer your first question, three to five years. Um, wow. We’re starting to see. month by month or quarter by quarter or how’s this? Are you asking how the cashflow works? Like the, the contracts, like the, the futures contracts, right? Like again, like I’m a former contracts, uh, derivatives trader. In the, in the market itself, um, like is in the broader market. Yeah. clusters are financed three to five years. Okay. Um, so the, the, the payments are totally over that period of time. Yeah. there is also like blocks of capacity in our order book, um, which can extend any arbitrary time. Yeah. there’s some stuff that we have to do, um, for, um, the cashflow on that 15:18 right now. What we do is we just ask people to pay entirely upfront for a block. Um, we don’t ask them to do that for a lease for a longer term contract. This gets into something that I definitely need a whiteboard to explain. Um, but, um, you had another question I was about to answer. I forgot. do you want real time or do you want, like, do you want more, uh, what’s more interesting to you? Real time or long, longer out? So we care more about longer outs because we’re trying to build a forwards curve that has a lot of liquidity on it. Um, the, um, these are kind of the same thing and kind of not. Um, so what we do is we, since 15:54 a, you can buy an hour out, you can buy right now to the next hour or you can buy right now to the next three years and anywhere in between. That’s wild. I know. Um, and so the, um, you know, or if I, you’re technically, um, the, there’s a certain difference that happens that’s dependent for each customer. Basically like if I have a workload that is, um, uh, that’s batch inference basically where I can be interrupted at any point. I don’t really care. Um, then my ability to be 16:29 cut off, um, is probably like, I need an hour of time to know, um, that I’m going to be cut off and then I’m good. Or I can even be notified in, you know, a minute or two. Um, but if I have, um, live inference customers who are expecting me to be online, um, then you can interrupt me, but in a month, maybe two months, maybe three months. Um, and then I can adapt to that capacity over that period of time. Um, but you cannot interrupt me in a minute. Um, and so those two things are priced differently. And the reason why they’re priced differently is because what is the price 17:05 you would pay in order to interrupt your batch job? It’s very different than the price you would pay to interrupt your, um, live job. If you have a live job, you’re, you are probably paying an SLA when you break for your customer. Um, does that make sense? Yeah, I see. So this is actually the market for breaking leases. These are the market for breaking leases. Oh my God. And then there’s also the market for future leases or like for leases that are, um. Oh, I thought this was like a subletting situation. This is like a lease break. So one way to think about it, um, is in power markets, there’s actually two markets. Um, there’s like the day ahead market and 17:40 there’s the live market. Compute kind of has the same thing, um, where it has capacity that is coming online, um, or capacity that’s about to, um, arrive to you. Um, and then capacity that is currently in use. And these have different price dynamics. Okay. Interesting. backwardation or contango? Backwardation or contango? Yeah. so do, uh, is compute further out price higher than spot or lower than spot? Um, is compute further out price? Does it, does it go like this? Does it go like that over, uh, you know, over like one year, one year out, two year out? 18:16 Do. Oh, this is an, uh, both at different times. I find it fascinating that you don’t know. Yeah. I mean, both. It depends on who you are. Um, okay. So, uh, I’ll make the case. Um, wait, so you’re asking is compute sold right now? same unit of compute today. Yes. Uh, maybe not today, maybe a month out versus one year out versus two year out. Is that, is that curve going up or going down? We see, like a hump. Oh my God. okay. So explain, um, like, I think this will make intuitive sense is basically like if I have milk and it is about to expire, I will lower the price on it. Yes. Um, and so that’s basically what we see. 18:52 It’s like, it’s the Nvidia release cycle basically, right? Uh, well, we’re more saying like if there’s a block of compute and it’s approaching now, um, it can’t be sold in the past. And so it has to drop its price in order to clear at some price. Yeah. And so initially you see this like drop where, um, stuff that’s sold right now, anything that’s on the market right now, um, is something that hasn’t been sold. And so therefore the price will drop. Um, and then it goes up, um, to where it wants to be. Um, and then maybe compute in the future is worth less, um, because people don’t want, like people would like the compute now. And so they’re sort of paying for the shorter lead time. Um, and they pay less for the compute that’s off into the future. 19:28 This is what I think a normal healthy curve looks like. Yeah. The place where I’m like, oh man, no s**t. Um, is that, um, right now if you took a B300 and you trying to run inference on it, um, you can make $40 an hour or so, um, selling that as inference. Any model basically. Um, and so that’s if you use it at a hundred percent and maybe if you only have like 50% utilization or something, so like $25 an hour. Yeah. So $25 an hour. Meanwhile, you can buy B300s at like five to $6 an hour. One of these numbers has to change. Yeah. It’s a huge arbitrage. It’s huge arbitrage. this in your talks a lot. 20:04 Yeah. So, um, a bunch of people are basically looking at this and thinking, um, if I think prices are going to go up, I might be willing to pay more than what I previously thought in the future. Um, because I might think, okay, I can buy a five right now, but if I can’t buy anymore, then maybe I will be willing to buy even more in the future just to lock it in if it’s going to come online later. So we see the other side as well. Um, kind of depends on who you are. Cool. So, is it okay. And, and I just, I just really want to make super clear on this. Of course. Do you care about the different generations of chips and, or do you just care about overall flops and you spread it among whatever chips you want? 20:40 Uh, we care about different generations of chips because. you care. I think like, I only, I guess do your customers care? Yes. Right. Oh, wait. When you say a different generation chips, do you mean like. So like B300s, B300s. exactly. Yeah. Or like even AMD or whatever. Right. Uh, yes. I think people totally care about the generation of chips. Um, uh, I think part of this is because of things like VRAM. Um, and part of this is because of like, um, larger NBLink domains. Yeah. And things like that. So there’s just no way to abstract all this to like some generalized unit of compute that you can sort of mush around and whatever you like. I think like when you were saying oil, even oil isn’t oil. Oil is actually just like a bunch of different types of oil. 21:16 Yeah. Like jet fuel or car fuel, whatever. I think it’s the same thing. Um, you’re going to get a bunch of different types. It is, but you have oil refineries. Yeah. can effectively have black boxes. You can put oil in and get like whatever out. Um, if compute, it is so heterogeneous, I guess is the word. It’s that you just have to expose the underlying complexity to your end customer, which is bad for you as a market maker. Maybe. Like you want to identify the one like choke point or like, I don’t know what the better word for choke point is like a shelling point where everyone agrees like, okay, like this is kind of the standard and we’ll, we’ll all agree to this standard and, uh, you know, it’ll 21:53 be somewhat inefficient. Yes. Uh, so we’ve been able to reduce the, um, the, the problems to a certain level, but yeah. Like your job is like dimensionality reduction. Yes. We reduce a lot of the dimensions, but nonetheless, there are still dimensions. Yeah. It’s still going to exist. Do you, do you know how many SKUs you have? So, um, like five times a hundred times like 5,000 equals, I don’t know. Like what’s, we actually made SKUs like a concept, um, that, and the goal is to make it so that somebody else can make their own SKU. So if I want to buy Blackwells, I can say I’m willing to 22:31 buy Blackwells at $6 an hour or less. And you can match me against GB300s or you can match me against B300s and I don’t care which Blackwell as long as it’s less than $6 an hour. but you just said you care about memory and other stuff. Totally. But not every customer does. So some customers like, um, just can work on Blackwells, whatever it is. Some customers are super hyper specific and they need this exact thing and they’re going to pay more for that because the market’s going to charge them more. Okay. Um, and that’s what we do. So, and so people use your APIs to programmatically define what they want and then you match that with whatever composition of sellers, uh, they might, they might, they might be able to fulfill. 23:08 Yes. Um, with a caveat that having a named query is a new thing we’re doing. Um, but the query system as a concept is how the order book works right now. Okay. Interesting. Um, all right. So, uh, you know, I, I think that’s like a lot of like, I guess, nitty gritty. Are there anything about like, I guess that you want to educate people on whether it’s financial people or engineers about the state of the compute trading market as, as you’ve developed it? Because effectively you guys are defining it. Um, mainly just the requirement for physical settlement. Yeah. Um, we think that if you, we, we spent three years learning why you need 23:44 physical settlement. Um, and, um, our entire conversation today is, um, that it is, these things are not as fungible as you think we’re going to try and make them fungible. That’s the industry’s job to do that. Um, but without that, you run a lot of risks. Um, if you try to build, um, too much financialization before you’ve, the industry has had time to build out the, um, the physical settlement. Um, I think what you have is you have, um, a market that’s very easy to manipulate and a market that has a lot of basis risk where, um, the price you may be hedging against actually has nothing to 24:22 do with the price that, um, the market, you can actually buy the capacity for. Yeah. So for, listeners, uh, we, we talked about this briefly last time, but, um, people are often shocked to find out that let’s say the market for credit default swaps is 12 times the, the, the size of the bonds that they insure, uh, something like that, or like, uh, the, the markets for mortgage, um, sort of reinsurance or, uh, all that is, is larger than the underlying physical settlement. So that’s why physical versus cash settled. Yeah. And, and like basically cash settlements are like exactly the kind of, uh, the side bet that you see in the big short, um, where, uh, they, 24:59 they may, uh, may or may not have anything to do with the underlying thing itself. And it may, may be used as a proxy for overall other stuff. Yes. That, and you don’t want to, that to go out of control. Yeah. So to be clear, we think, um, cash settlement as a future concept is a great idea. Yeah. Um, we just think it should be derived from physical settlement. Yeah. because as you said, it’s a side bet. Um, and it would be kind of like if you, I mean, be kind of like if you made a cash of the future before you made the physical settlement, um, which I think people are kind of doing already. Yeah. Totally. People are trying to make cash on it before you make the physical settlement. And people do that for all sorts of things, including 25:34 startup stock. Yes. Right. Like you can buy and sell a made up share of open AI, but it’s nothing to do with the actual share. Yes. Um, and lots of those go very poorly. Um, like it would be kind of bad if you made a, um, it would be bad if you started selling, um, the, the derivative on a stock that doesn’t exist yet. Um, yeah. Yeah. Well, so I mean, one thing that we do, so I used to be a currency trader, right. Yeah. And, interest rates as well. And what, one thing we do is you have a board that gets together and says like, this is the London interbank offered rate LIBOR. Yes. And we all agree like this is the, the, the monthly fixing and we all 26:12 trade to that fixing. Yes. Uh, is that even meaningful at some point in three years from now? So critically LIBOR was manipulated. Yes. our concern at the moment. there’s 12 banks? Yeah. Yes. Um, well, our concern is there’s a very similar problem, um, where you can manipulate an index. Um, our impression is why you can manipulate that index is because a lot of the indexes are based on a very small amount of data that has, um, uh, not a lot of correlation with the actual price of buying a new reserve contract. Um, that’s what we’re concerned about. Okay. 26:47 Interesting. Um, all right. I think, I think that’s the, the, the rough state of things. Uh, um, I think the other part of just, just today is, uh, you, you, you, you talked about an event with jump and you’re, you’re friends with Silicon data. Uh, what are you learning from all these other players? Like who are the major players that you would want to call out as like doing interesting things? Ooh. Um, so we’re big fans of Silicon data and compute exchange. Um, I think they were also very early to this. Um, and I think they are doing like a reasonable shot at the, a path that we think, um, that we’ll likely end up being partners with. 27:23 Um, can you distinguish like your, your different paths? So, um, the general just is they’re doing cash settlement. We’re doing physical settlement. Um, we think both can exist and should exist. Um, and that they’re sort of mutually beneficial to each other. Um, and so totally fine. Um, so I think they are one of the earliest in the space and I think they are, very clearly took the bet, um, well before anybody else did. And I think that’s like very honorable to do. Um, um, you mentioned, uh, jump. Semi-analysis, jump. Like these are the players emerging. Like as far, like you were my first introduction to this. Yeah. And then, and I see all these other guys coming in and I’m like, okay, like it’s something’s happening here. 27:58 I don’t know the landscape as clearly as you do. Yeah. So, um, the categories of folks that we’ve seen is there’s folks that are trying to set better standards. So semi-analysis I think is working to try and create better standards. Um, there’s folks that are trying to do, um, residual value insurance and so on. Um, there’s a number of different companies in this space. I don’t know any of them super well, but, um, I know, know this is a concept and I think that’s important. Um, there are folks that are trying to do, um, uh, deposit, um, lending where like they’ll front your credit risk for the cloud. There’s an ecosystem of companies that are forming around how do we resolve the financial bottlenecks of compute. 28:34 Um, and it is cool to see. I think effectively what you want in this ecosystem is buyers and sellers. You want an analyst. You want, uh, we said insurance or, uh, what’s the, uh, residual value. Yeah. Yeah. and you, we want this like healthy ecosystem out there. Um, but obviously you guys are owning the problem of physical settlement, which, which is really interesting. Um, okay. So, I mean, we can go into like the, the hard tech side because obviously you are also hiring, uh, for extremely cracked engineers. Yeah. describe your stack. Um, we do all the things a cloud does without being the 29:10 cloud. Um, as in we don’t own the hardware. the reason why we don’t own the hardware is because if we did, we would be able to manipulate the forwards market. So we have to be Switzerland. We have to be independent. Um, which creates a lot of sort of strange things about the company. Um, but in order to do physical settlement, you have to do site visits. have to visit the data center. Um, you have to, um, you have to design the bill of materials and ensure that, um, you have to basically do like supply chain consulting, um, where you have relationships with different OEMs that, um, ensure that things get delivered 29:46 on time. Um, then you have to, um, do cluster architecture. Um, uh, so you’re sort of designing the cluster itself. Um, then you’re, um, rolling out racks or you’re working with partners to roll out racks and supervising that. Um, then you’re doing, uh, backplane management. So you’re taking over the, um, uh, PDUs and the switches and the DPUs and, um, UFM and BMC and you’re operating, um, the whole, um, harness around the nodes that you’re selling. When you say you as an SF compute employees. Yeah. Wow. Um, so, uh, when you buy a node from a cloud, um, there’s the bare metal, but there’s all this s**t around the bare 30:23 metal. Yeah. Um, and in order to make that bare metal node work, you have to operate everything else. Then we. Because I just like traditionally the job of the data center is to extract some of this away from you. Yeah. We’re the ones that do that. Okay. Right. Um, but you’re not, you’re not in the data center business. You’re not the sort of equinixes of the world. We’re, we’re not in the data center business sort of, um, what we do work with is, so we offer supercomputing as a service. Um, and what this means is folks come to us and say, you know, 30:58 I’ve got X where X is, I’ve got, you know, X megawatts here. And maybe sometimes they mean I’ve got a data center. Um, that’s like, um, would like to turn into a GPU cluster. Like my options are, I could sell the GPU cluster or I could sell the data center to somebody or I could become the cloud. And then we go to them and say, great. Um, we will turn you into the cloud. We’re the supercomputing part. You’re the data center. Let’s partner together. And then we will sell and I’ll make money. Um, sometimes it’s people who have something that could turn into a data center, in which case we’re going to basically project manage that up to the layer. 31:33 Um, and then we will work with those far, we’ll work with those folks to turn their powered land, um, or whatever they have, um, into a data center by working with partners. Um, and then we take over at the supercomputing part. Okay. This requires us to have some sort of data center expertise in house, but we don’t self-perform builds. Um, so yeah. Yeah. So it’s, uh, somewhat modular because you have different scenarios you need to play under. Yeah. Uh, someone once described us as like, um, uh, like Palantir for compute, um, because it’s like these, each time it’s this new rodeo. 32:10 Um, uh, but, and we like, we, we deeply partner with folks. On both sides. Yeah. Uh, and primarily the U S primarily the U S, but we are operate around the world. Yeah. At some point does SF compute become a misnomer? Uh, so when, so we’re deploying clusters all around the world. Yeah. And when we do that, we’re working with people who themselves are trying to be clouds all around the world. Um, they look like sovereign compute players. Um, and our pitch to sovereign compute is amazing. Uh, let’s help you set up a cloud in your own country. Um, and we will bring the San Francisco offtake. We’ll bring the San Francisco expertise. 32:45 We’ll set up the clusters. Um, that’s our whole business. Ah, um, we’re going to make it. Who are your main, uh, your biggest announced partners that you can name? Um, nobody I can name. Really? Wow. Uh, we will have some announcements, um, in a week or two, um, that we expect to come on. Yeah. All things sort of. Sure. Sure. Good luck. Yeah. No, I mean, uh, I, I think this is really interesting for every country to like, who’s, who’s like investing in sovereign AI to figure this out. Um, and it’s interesting that like SF is still sort of like that brand name and, uh, that, that is, that is very marketable. 33:20 So I think one of the interesting things is just thinking through this market. NVIDIA is obviously like the 800-pound gorilla in the room. Why didn’t they make SF Compute? And again, the answer is going to be market manipulation. But let’s just really get into the incentives of the multiple players. Yeah. So our impression is that NVIDIA can’t make it, but benefits tremendously from it existing. The reason why they can’t make it is because they would then not be independent enough for people to trade on it. If you don’t trade on it, then you don’t have enough liquidity for lenders to finance against the forwards market. 33:55 This is also why Crusoe and CoreWeave and Lambda, why the hyperscalers can’t make it, why Base 10 and Fireworks and Modal can’t make it because they have too much demand on that side. This is a thing that benefits the entire industry because it makes the financing costs go down. But nobody can make it unless they give up the ability to do all the big inference money and all the big cloud deals. You have to kill your business in order to make this. That’s why nobody does it. like structurally, they’re not set up to do that. And you’re basically reducing friction and that’s all your revenue. 34:30 You’ve announced some milestones. Actually, I forgot to say that up front. Can you repeat what they are and just take a victory lap a little bit? We’ve closed several hundred million dollars of volume. We have a significant number of new clusters on the way. And then the things that I think I’m most excited about is we’ve shipped a whole bunch of new features that make SF Compute very viable for platform-as-a-service companies. An example is we allow customers to subdivide 35:06 their InfiniBand fabric without having to ask us. So many clouds, when you go to buy them, they will give you one InfiniBand fabric. And then if you want to redraw the lines of that InfiniBand fabric, you need to contact them and they will then do something for you. We now can do that on the fly for customers, which I think is really great because it interfaces with the market really well where you want to be able to redraw things in order to scale up and down your clusters. Would you describe this as like you’re virtualizing the data center and making it programmable? Yeah, we’re virtualizing the data center and making it programmable. So all the stuff that you would have gotten if you had owned the 35:43 hardware, we’re trying to virtualize that, which I think is exciting to us. It’s funny, like the last time this happened was like the cloud, which is really cool. And then finally, you know, I think what is your call to action for people? You know, is it engineers? Is it like, give me your supply? Like, what is it? If you would like to own a GPU cluster and you want the residual value from it, you want to have it for your portfolio, we would love to work with you in order to build that GPU cluster. We will take over and we will do the supercomputing portions of that and 36:18 ensure the things go right. We own the SLA. And then if it turns out that that was a bad idea that you need out of that cluster, we can immediately sell that cluster to somebody else so it doesn’t hit your balance sheet. Very simple pitch. Well, thank you so much for your time. You’re doing a great service to the industry and I just am happy to be your friend. I appreciate it. All right. Thanks. That’s it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe
出典・この記事について
- 番組
- Latent Space: The AI Engineer Podcast（Latent.Space）
- 元の回
- The Programmable Data Center — Evan Conrad, SF Compute
- 配信日
- 要約作成日
- 公式サイト
- www.latent.space/podcast
この要約・解説記事は、元の音声を文字起こしした内容をもとに AI（Claude）が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について
関連エピソード
計算速度で回すラボ・イン・ザ・ループ — Ian Quigley（Leash Bio）
Latent Space: The AI Engineer Podcast
Dyna-2：100万時間の人間動画で学ぶワールドアクションモデルと、1時間自律で動く汎用ロボット — Jason Ma（Dyna Robotics）
Latent Space: The AI Engineer Podcast
なぜ AlphaFold はタンパク質フォールディングを「解決」しなかったのか — Pushmeet Kohli(Google DeepMind)× Sal Candido(Biohub)
Latent Space: The AI Engineer Podcast
合成超知能へ――半導体から超伝導体まで:Periodic Labs の Liam Fedus と Ekin Dogus Cubuk
Latent Space: The AI Engineer Podcast
学界は野心のためにある — Alex Zhang（MIT）が語るRLM・ハーネス・エージェントスウォーム
Latent Space: The AI Engineer Podcast
Dwarkeshは間違っている？Computer Useの現在地と、OpenAIがJev対抗のDecisions APIを1週間で作った経緯
Latent Space: The AI Engineer Podcast