現代AIワークフローの性能制約の内側――Everpure の Cameron Bahar 氏に聞く

Welcome, everyone, to the Emerge AI in Business podcast. Today's guest is Cameron Bahar, GM of Scale AI at Everpeer. Everpeer provides enterprise data storage and management software that consolidates block, file, and object workloads across on-prem and cloud environments. The platform focuses on efficient data reduction, consistent performance at scale, and unified operations across major cloud providers. Cameron discusses how the rapid shift from AI training to interference and agentee workload is training traditional storage systems, and why simplicity, scalability, and independent scaling of metadata and data have become critical design principles. He outlines what enterprise leaders across industries from autonomous vehicle companies to financial trading firms need to consider as compute and data demands continue to multiply.

Today's episode is sponsored by Everpeer. Emerge works with a select group of AI vendors to reach Fortune 500 decision makers through research, media and direct access. If you want to be considered, download our media kit at Emerge.com.ad1. That's Emerge.com.ad1. Now the conversation with Cameron. So Cameron, welcome to the show. Thank you very much, Dan. It's good to be here. Absolutely. We've got some good topics to be able to hop off into. I'm going to open it up in a way that will let you take us where you think we need to go here. AI storage architecture isn't a consideration that everybody's been thinking about for the last five years, but it's becoming increasingly important.

I want to get your take on what's important about AI storage architecture. What should leaders know, maybe, to start us off here? Sure. I mean, I think over the last two or three years, we've seen interesting workloads like training, inference, and now agentic workloads. and on the first phase we were training large LLM models and training was the dominant workload and given how expensive GPU farms are right holding them idle was a big financial burden on many people who were training GPUs and typically the data loading phase the throughput required for training is orders of magnitude higher than what typical enterprise storage systems were designed to deliver. Nobody really needed that many terabytes per second of throughput from one large data set. So it's often thousands of GPUs.

You're looking at 10,000 to 20,000 to OpenAI talking about 100,000 GPUs, Grok talking about 100,000 GPUs. And they often have one large data set, and you need to load that into the GPU compute fabric and then do your deep neural network to train the LLM. So the main architectural issue is how do you do that without it being a science experiment? And people have been leveraging traditional HPC storage systems that were used in government labs. Luster comes to mind. It's, you know, 25 years old and leveraging some of those fast systems to try to get a handle on the scale and throughput requirements. So really, it's about a scale that's an order of magnitude or more higher than what we're used to and literally feeding the beast, as the industry calls it, feeding the GPUs so that they're not idle. That's been the biggest sort of challenge.

And as you get into inference and agentic workloads, they end up being a little bit different than training, and they're stressing the systems in different ways from metadata, performance, latency, getting lots of agents to write lots of data, reading it back. There's sort of a memory problem. There's a memory wall issue as well that's hitting a lot of workloads. Yeah, so the fundamental shift from sort of a training-ish paradigm of how this stuff is being used most of the time to this inference-ish paradigm. We're hearing from leaders, it makes the use of compute hard to predict. What are we going to need? How much are we going to need? What are we going to be spending? These things all get complicated in part because training and inference are different, but also because when hardware changes, our power needs change, et cetera. What are you seeing leaders wrestle with to deal with this paradigm shift you're articulating? I think the challenge most leaders face is the unknown because these workloads are not very well understood and they're changing very fast. So every three months, things are changing. So having a system that is simple and flexible and scalable

becomes the dominant three pillars that people are looking for. Imagine you doing training and maybe you have a few data scientists who are training one large model and you flip that to inference and now you got a consumer play where there multiple thousands of people logging in and doing various inference workloads So the multi-tenancy aspect of this becomes an issue. And so these architectures end up looking more like cloud architectures and cloud storage solutions than appliances that are typically in the back office performing a single function. Yeah. Well, we're going to get into that analogy towards the end here for sure, because there's so much to mine there in terms of how to think about where this paradigm is headed. But, yeah, to your point, you mentioned kind of three pillars there. There's an overall notion of sort of nimbleness, feeling like it's what's required for the age that we're entering. To your point, you know, the models are changing. The hardware is changing. The kinds of workflows we can totally overhaul with AI are different today than they were six months ago for sure.

And of course, we presume they'll be different again in six months. So maybe you could reiterate those three pillars and talk about how they might serve to allow for that kind of nimbleness. Because a lot of the folks tuning in that are finally scaling this stuff, they're moving beyond these pilot projects. They're really bringing things to deployment. I think they'd love to be nimble and to make that real. And maybe your principles could sort of help to outline a bit. Sure. So if you think about simplicity and scale, you need the ability to scale different parts of the system independently. So, for example, one day you may need lots of throughput, and being able to scale throughput and capacity on demand is sort of one angle. And another workload like inference, it might be metadata heavy. So having a separate metadata system that scales independently and that you can tune that on demand to your workload becomes a key enabler. So when we say simple and scale and operational burdens that we'll talk about later, but these are some of the core tenets.

And so getting separate things, so the separation of metadata from data and being able to scale those independently. Having parallel access to data using standard protocols. Lots of people are inventing custom protocols and custom clients. And while that's great for HPC in the enterprise and in AI factories, people need standard. And the open systems flourish with standards. This is not any different. And also there's a security angle to shipping custom clients and running on compute with custom clients. So having an industry standard protocol like PNFS that provides parallel data access, and extreme throughput becomes an enabler here. And then you've got the third leg, which is, you know, what connects these systems together are networks. And high-speed networks leveraging RDMA become another pillar to build on that gives you this massive throughput and performance. Efficiency of each node in the system is also important.

So making sure, because power is the number one constraint in these data centers. And so, you know, you can't really over-provision and over-subscribe and throw a lot of hardware at this problem and have a lot of it be idle because you want to get performance. So a per-node performance that's efficient is important, is the other element, because power is a major issue and you need to get lots of throughput with as few racks as possible. Yeah, this is everybody's challenge. And again, as few racks for the workflows we have now versus few racks for the workflows we have in three months, six months, These are all things where, again, that nimbleness and simplicity, I think, are going to come in time and time again. I want to double click on sort of what's different about this way of thinking about AI storage architecture than maybe whatever the norm used to be. You talked about the shift from training to inference. One of the other things you mentioned was sort of that separation of data and metadata, scaling separately. This also feels unique, right? This was not baked into the early days of AI storage architecture by any means.

Maybe you could make that tangible and paint a bit of a mental picture for the audience as to like, what would that look like with a certain type of data? I don't know if we want to talk about autonomous cars or fraud detection in retail or whatever you want. And then how that balancing might help us achieve those goals of that efficiency, etc. that you were talking about. Often when you start, you don't start with 100,000 GPUs at the extreme end. You may start with a few thousand and do training. And so you size the system for a particular metadata performance and a particular throughput and capacity requirement, right? But as you, as I've observed customers, they tend to double their GPU counts and GPU compute every three to six months. So being able to, so as you expand and quadruple, you know, every year your GPU counts, it puts tremendous strain on the storage platform. And so, for example, if you find out that your mega systems might be bottlenecked one day, being able to add capacity on the fly to that system without taking it down is a critical feature that's needed.

That's typically not how enterprise workloads behave. It's well understood. You size the system for that workload and you amortize that over four or five years The rate of change is kind of slow Whereas in these environments the rate of change is really fast So being able to adjust the levers in the system is key right So I think that the separation of metadata and data gives you a nice architectural advantage where you can increase the performance of each independently based on demand. To your point, I guess demand, I forget the exact stat you had just mentioned, but companies even that start small tend to stretch footprint pretty darn quickly from what you're seeing. It sounds like you're seeing that across industries with different clients. I'm seeing that across industries. As an example, there was a car company that collects sensor data and they're a startup. And so nobody pays much attention to a small startup that's unknown and that maybe has 20 employees. But then they start consuming petabytes and petabytes of storage and running some workload. And we find out that it's an AI

workload, but they end up doing a 10x almost every six months and becoming a very large storage consumer as a startup. These kinds of data-hungry companies didn't exist five years ago. They're generating sensor data from lots of cars that are on the road, each generating terabytes a day, and they're training on that. So full self-driving Tesla is an example of that. But we see a lot of startups working on various verticals following a similar pattern. And so then for a company that's starting out, it's cash strapped, they don't have time to reinvent and build the storage fabric from scratch. So they need something that scales with them and is flexible enough and it can start small but grow to very large data sets. So we're talking about exabyte scale systems that are not typical in the area. Well, absolutely not. I mean, again, I sort of want to draw the distinction there, right? Because I think most of the folks tuned in sharp enough to know, okay,

Waymo or these other companies that are doing, you know, LiDAR plus video plus, you know, in the car, out of the car, night vision, whatever it is, right? We're going to be sensor fusion to make sure we're looking at everything at once or what have you. We're looking at unbearable amounts of data per vehicle, as you would say, terabytes per vehicle, whatever it is per day. You know, there's folks who are going to be tuned in, big financial services company, big life sciences company, et cetera, presumably their data might not pile on top of itself in the same grandiosity, but they are going to run into similar-ish bottlenecks. If there's folks tuned in who are like, well, maybe we don't have a lot of video data. Where does this still christened cross with their world, just so people can understand? Yeah, it's interesting. I do work with a number of high-frequency trading sort of financial services that are using GPUs to predict market movements, let's say, for example. They're all proprietary models. but, you know, speed for them is revenue. So having a platform that basically doesn't start up the GPU

and is really fast and low latency and gives you tremendous amount of throughput is sort of the game changer for that. That's the requirement. Because, you know, we're running at the speed of light. A millisecond makes a difference for us in market movements and so on. So we need to front run various strategies, right? Renaissance capital, you know, there's a lot of Jane Street. There's a lot of companies of that ilk that hire lots of PhDs in math and physics to create these models that are effectively simulations. And what we see is speed is critical. And when we say speed, they're like 10 terabytes per second type of speeds, right, which are not. It's approaching what the entire internet is doing in one workload at some company. and it's that order of magnitude change that's stressing the existing systems and you know from oil and gas exploration to you know financial trading to eda workloads people are designing new gpus using gpus and ai right so chip design with with ai it's in various segments

right and we see all the way from text processing where you're you know uh basically downloading the internet and training an LLM on it to answer questions to multimodal where images and audio and video are also added to it. And that breaks a lot of systems because the capacity and scale of these things then again does another 10x. All the text on the internet is a heck of a lot of text, but to your point, as soon as we get into multimodal, it's another stratosphere entirely, right? I mean, in terms of what storage requirements are. And to your point, anywhere where speed is not optional, speed is necessary for competitive advantage, well, you're going to be dealing with these kinds of efficiency and storage things, even if you're not doing video per se, same sort of concern. So you brought up before the idea of this being more like a cloud operating model than like an application, so to speak. This ties into the bigger paradigm where this storage architecture is kind of moving. Maybe you can kind of walk through what you mean by that in practice. In other words, hey, application was kind of like this. People thought about it, worked with it like this. Cloud operating model is more like this. Like I want to frame this for

leaders to say, here's how you should now think about the future. Like, where would you start? The main idea is that these, what we call data nodes that have lots of, let's say, NVMe SSDs in them that provide the storage capacity they should look like the field replaceable unit so that the customers effectively have choice to bring their own data nodes buy their own data nodes directly from the vendor of choice right And it intelligent software on these data nodes that will provide this storage fabric that we talk about. So far from buying a custom appliance that says single function appliance does block or file or object. This is really a soft distributed systems software that runs on commodity off-the-shelf servers with very fast NVMe SSDs. But these are off-the-shelf products that are the latest and greatest, you know, CPUs, memory, PCIe. And you want to be at the front end of this curve by being

able to leverage what's available from the server manufacturers, right? And with intelligent In software, you can have these nodes fail, and in a cloud-like environment, failures are often, they're daily. As your systems scale and you have many, many components and many, many nodes, failures are no longer a once-in-a-while phenomenon. You'll have failures daily. And when we talk about cloud architectures, the idea is that software will hide hardware failures, and the system continues to operate despite hardware failures, right? And so it's that architecture that allows you to have a multi-tenant system that's always up and running while the hardware underneath, you know, gets fixed, gets refreshed, gets exchanged, or it gets through a technology refresh cycle, gets updated. That doesn't actually take the cloud down. If you look at Amazon or other hyperscalers, when they are continuously upgrading their servers, their systems, fixing faults,

but from a service delivery point of view, you're not noticing any of that. You just get a service. So the key point is when we talk about cloud architectures, it's about consuming a service. And so what are all the features you need to provide from an infrastructure provider to you who's running an AI workload that makes it look like cloud-friendly solution that's always on. This constant availability kind of knits very well into what you were opening with around nimbleness, right? I mean, around being ready for whatever's coming. So it feels like it's almost impossible to do that from the old school application paradigm versus a sort of general cloud operating model. As we kind of close out on this theme and topic, we covered those ideas of nimbleness. We talked a little bit about what's changed and now sort of this paradigm of thinking about what always on looks like in practice, where things can be swapped out, but availability and use are still working in a multi-tenant system you'd articulated. Anything else would be important for leaders to know about where surveyized storage architecture

is headed or kind of nutshell points they should walk away with to be ready for the future that they're walking into as we wrap up? Yeah, I think, you know, what's really difficult is to operate these systems at scale in a simple way. And you should look for architectures and systems that simplify and are not a science experiment, right? Because as you scale, the operational complexity of running these things at scale becomes the, you know, tall pole in the tent, if you will. So I think a system that's well thought through and simple, I think is the key to have a successful outcome in a lot of these environments. Excellent. Well, I know the next time we're chatting, we'll be talking a little bit about scale. So we're teeing ourselves up quite well here, Cameron. I'm glad we were able to wrap up this first episode together and share some of your insights around where things are headed with storage architecture. Thanks so much for being here on the show. You bet. Nice talk to you, Dan. It's a pleasure.

Wrapping up today's episode, let's look at the three key takeaways from our conversation with Cameron. First, AI workloads are shifting from training-dominated demand to interference and agentic use cases that stress storage systems in fundamentally different ways, from metadata performance to latency. Second, separating metadata from data and scaling each independently gives infrastructure the flexibility to keep pace with GPU counts that often double or quadruple within a year. and finally treating storage as a cloud-like service one that stays available and multi-tenants even as underlying hardware fails or gets refreshed is becoming the standard enterprise leaders should design to it emerge works with a select group of ai vendors to reach 4500 decision makers through research media and direct access if you want to be considered Download our media kit at emerge.com slash A-D-1. That's E-M-E-R-J dot com slash A-D-1 to download your copy. On behalf of the team at Emerge, we'll see you on the next episode.