AIエージェントには新しいデータスタックが必要な理由

All right, so today we have my friend Deepthi Srivastava, founder and CEO of Snow Leopard, which you can find at snowleopard.ai. Taglines are empower your AI agents with live accurate data in minutes. Snow Leopard, with its automatic semantic data intelligence, provides 90 plus percent data accuracy out of the box so you can ship enterprise agents faster and with confidence. And with that, Titi, welcome back to the podcast. Thank you, Ben. It's super exciting to be back here. All right. So I guess the thesis is agents need data. In fact, agents need the right data at the right time. I think that makes sense. I think a lot of our listeners will agree with this. So agents need data, and data, for the most part, will have to be retrieved.

I think that also makes sense, right? So right now, I guess the current state of Snow Leopard is that it sits between your databases and the agents that need information from them. An agent sends a question through an API, and then Snow Leopard works out which database holds the answer, writes the queries, fetches the results, and combines them. That's correct, yes. And the emphasis here, DT, am I right to say the emphasis is on operational data, not analytic data like data warehouses? The emphasis is on operational data. I think analytics data is also can be used as part of operational data. But the way I think about this is operational data sits in structured storage, right? So structured storage is SQL databases, APIs like Salesforce or HubSpot or things like that, right?

So things that make your business run or are part of the critical business decision making. So, for example, there is a lot of data that is unstructured. And overall, you know, if you remember back to like 23, 24, 23, 24, 25, even, RAG was all the rage for unstructured data, right? Because it was the first time that we could actually sort of assemble and make sense of unstructured data. But really, the focus from Snow Leopard's side, from my side, has always been operational data that makes your business run. So it's stuff that is in SQL Server or Postgres or MySQL or Snowflake, which can be both analytical and people use it for, you know, operations data today as well. So like things like that and Neon and all of that and Sockle from Salesforce and those things, because they become critical to how you run your business. And they are therefore critical to how you will run autonomous agents. Right.

Because the whole point is we not only automate current workflows, but we create new workflows with the power of Gen.AI, to be specific, right? And so autonomous agents, AI agents, are meant to be that unlocking force. And for them to work, they need, like you said, access to the right data from the right place at the right time. So if it is a warehouse, we'll talk to the warehouse. So, but basically, so the foundation of Snow Leopard, so to speak, is data in databases. So it could be operational, it could be analytic. And then databases that use SQL. That's right. Is another assumption. Yes. So then one of the things that you do is that let's set aside how you write the SQL statement for now. But one of the things you do is let's say I'm an AI agent that's interacting with an external customer. you will, based on the inferred need of the customer, you will decide where the data lives.

It could live in multiple places. Write the SQL and then return to the customer what the customer needs. So implicit in this, I think lurking in the background is a dirty word called federation. Am I wrong? No, you're absolutely right. I don't think of it as a dirty word, but yeah, it is absolutely like federation. So federation has had a long history, right? Yes. So let's, I'll have you play devil's advocate first. So why do, why are people skeptical of federation? Because historically it hasn't worked, right? Historically, the way to do federation has been. Oh, by the way, maybe you should give a very, very high level definition of federation. Federation means you go to different data sources and fetch data from it. Whether it's live or not, it's a separate question. But the point is you have some data in one database, some data in another database. So Postgres and MySQL used to be the canonical. And then you don't know how to go get between data from both of them.

Your customer data can be in Postgres. Your product usage data can be in MySQL, for example, or the other way around. And so the question becomes, how do you combine that information in order to answer a question, even if it was for a dashboard? Like even if it was historical trend analysis, you still have to go get from both. And so the overall thought process in the past 20 years was you use an ORM and then you use like the base SQL, right, the common NC SQL standard of the time. And then you write every query in that form so that your LAMP stack type application would then go fetch from, you know, it would write the program, basically, would write the SQL query in the base SQL language. And so it could run anywhere, right? Because the idea was it was this standardized SQL, so it could run anywhere. And so you could write basic SQL and get data from, let's say, my SQL on a Postgres, right? Because both of them would support basic selects and basic joins. But what's implicit or explicit in that is that the basic feature functionality set becomes least common denominators.

You actually can't use most of the functionality that each database provides completely, right? So it becomes this like hobbled together system and you have to have exceptions all over the place in order to like really use the MySQL stuff versus the Postgres stuff versus the, you know, Teradata stuff versus the Oracle stuff. So that doesn't work. Also, federation used to be like really hard because you couldn't figure out a priori, which is how most software has been built until now. By pre-agents, all software is built a priori, which means all the logic has to be pre-programmed, which means you don't know which data silo you have to go to in order to get that information a priori. So putting together a federation became practically really, really hard, if not impossible, which is where this concept, which, by the way, I agree, in the pre-world, this concept of warehousing became so popular because now you dump everything into a warehouse. The warehouse is known in its boundaries. And now you can write your program very deterministically in order to get information from the warehouse.

But that's only good for analytics and reports. Yeah, so the current warehouse vendors will argue with you that real-time pipelining and sub-minute blah, blah, blah will give you that. But it doesn't. And all of your listeners who have ever done this know that it's not true, right? So you use the analytics style capabilities to run sort of real-time-ish workloads, and that doesn't work. There's always going to be a lag, right? There's always going to be a lag. And the problem is, at human speed, this was still okay because we would fill in the gaps ourselves, right? Like, I think this is the biggest thing that people are now sort of facing in real time, right? They're realizing as they build autonomous agents that some of that lag, when the human was the operator, we can fill in. We can go look up Salesforce and give you an answer if you're sitting in a boardroom. Or we can like the customer support example that you're saying. The human customer support agent can go look at the data source if they have access and give you an answer like, where is my order?

Even if it's not made it to the warehouse. I think the delta today is, and it's a really important delta, that an autonomous agent is not, if he doesn't have access to the right information, cannot give you that information, right? So a lot of support calls today that you call, I've done this even recently. If your e-commerce workflow fails at the wrong time where your user calls in, like, I didn't get my order number, so I don't know if the order went through or not, which is the canonical example I give because I faced it. The human agent has to, so you were going to talk to a chatbot, the chatbot's going to ask you for your order number, you do not have your order number, then it has to transfer you to the human to bridge that last 30% gap of where it doesn't have the information, right? And now you have autonomous decision-making agent, like how much marketing spend to have, right? Who is, which microtransaction and financial services to actually let go, like those things, you cannot intervene, right? But still, so why does federation work now?

Two things. In the era of Gen.AI, we can, and this is known, we can actually write text-to-SQL on the fly, right? And we have, because of Gen.AI, if you actually work with the system's understanding of how systems and databases work, you can actually do live federation across by making real-time decisions that you previously couldn't without Gen.AI, right? So that's actually what the premise of Snow Leopard is. This is possible now because both LLMs can do text-to-SQL at a base level, right? And we can make decisions based on live pattern matching, if you will, right? Which LLMs are very good at, so you don't have to have if-then-else conditions for every single thing that is out there. And that's why it works. So then in the example you gave, right, so you have customer data in Postgres and product data in MySQL. Well, first of all, I think one of the things you first do when you engage with someone is you understand the schema across all these databases, right?

So then I guess what people call the semantic layer, right? That's right. So a lot of people have this. Sometimes they have it only for their system. There's some AI to BI tools that have it across different analytic systems, right? So if you use Snowflake Databricks, BigQuery, they all have it. Yeah. So you're building it across these operational systems, right? So then that means you understand that customer data is in Postgres, the product data is in MySQL, but then you still have to do joins potentially, right? And we can do that outside those systems. That is the key sort of insight or value added to NoLapWord is that we can join across those data sources. And the reason is because the types of queries you're working with are not going to be just massive, big queries like, here's a billion row table, join it with this, right? So they're operational systems, right? So there's two parts to that answer, right?

The first part is yes. I think everybody that's in the data space goes into the like, oh, my God, but the worst case scenario. Yeah. Which is actually why the Federation also didn't work earlier, right? Side note, I literally recently wrote an article for the ACM, a research article about what's required for data architectures. I'll send you a link to it. Okay. Because it talks about what was wrong with federation in more detail and why it works now in more detail. But the point is we can build these semantics a priori, first of all, like at connection time, and you can use that at query time to write the SQL, right? So that's one part of the system. But to go back to your question of why this will work and how come we won't go into the pathological case, I think data shows, like, operational data queries are generally, generally speaking, point lookups or small joins, right? So when we talk about this customer example, it's one customer making 10 orders maybe. So those are not the pathological cases. So, yes, we work really well.

Well, in fact, you know, if you demo it, if you use it yourself, like we can answer these questions in real time, like very, very quickly. Right. Because we're doing all this stuff with point lookups and point joins and we can parallelize and we can sort of distribute and put the information together very quickly. So that's one thing. The second thing is any data person that has worked with databases, et cetera, knows like there are known optimization techniques. Right. So, for example, if we see joins that are large, we can break them down, first of all. We can optimize the query. The databases themselves do that. Secondarily, if we have had to do it once, like we can cache it. There's like multiple, like there's caching and there is sort of sub query generation. Like, you know, you can do a map reduce kind of thing with queries, right? Where you can break them up. You can pull the data in separately and then you can put it back together. This is what we did at Oracle for large joins and joins and things like that. You can create views and things like that. So there's multiple ways in which we already know how to optimize that part that we can apply. And these are known and solved problems.

So it's a matter of application. The hardest part is what Snow Leopard is solving today which is where is my data How do I connect it how do i get the right context so that the text to sql is highly highly accurate right how do we get governance right so that the agent doesn't touch the wrong data and how do we actually use the sql databases with the agents where the answer today is oh just mcp it and like i think we're coming out finally from the trough of disillusionment around mcp right like there was this whole hype the last time we talked it had just come out and there was this whole hype of mcp will solve everything doesn't solve governance doesn't solve um accuracy right actually burns a lot of tokens so all of these things that are practical issues that you and i know are hard right like data is one of those things it's like i don't know like hold or or things like that right where you can say oh i know how to solve it But the devil is in the details and you spend like 90% of your time trying to deal with the edge cases, outliers and things like that, which become very common.

Those are the hard parts. So this semantic layer, right? So this is key to what you do, right? So you go in to a company and then in order for you to do your magic, you have to infer automatically this kind of semantic layer, correct? So semantic layer, broadly defined, basically understanding what data is available, where is it, and what are the right permissions, and so on and so forth. But key is this column, which might be called, I don't know, 1, 2, 3, 4, 5, is actually customer ID. So how well does that automatic construction of the semantic layer work out of the box? So we ran a benchmark against Sonnet 5.6, Opus latest versions, and the GPT laser version at the time, like four months ago. These things moved really fast.

But we were 90 plus percent accurate, 95 plus percent accurate in our benchmark. And we didn't shenanigan it. We literally just ran us, so SnowLopper with the datasets, which is BigQuery and Postgres, and public datasets. We didn't shenanigan the dataset either. and these LLMs plus MCT, right? And we didn't do like 10 questions only, right? So we ran 100 questions three times. Like there is a benchmark that we published. So number one, we do really well. By the way, the operational data tends to live in these really ginormous schemas, right? So thousands of tables potentially. Right? So ERP systems are a mess, right? Yes. I think that all of that is true. I, again, think if you look at the sum total, right, of all the databases that people are even trying to use with autonomous systems today, like there's a bell curve, right? The meat of the bell curve is like maybe 5,000 schemas. Yeah. Like that kind of thing, right? And those are very simple. Like the way we've built the system is that it is scalable, right?

So it doesn't get trapped with we have, you know, 10 terabytes of data because we don't care. The schemas are large, yes. But I think I want to go back to like what does it mean to build semantic intelligence, right? what does it mean to like have semantic layer right because people focus a lot on ontology definition on the one end of the spectrum and schema reading on the other and snow leopard is built on the premise that you need both and more right so i think in order to have semantic intelligence you have to understand not just the schemas but also the system from whom you are reading those schemas so sql server is very different from postgres is very different from MySQL is very different from BigQuery, right, from Snowflake, etc. And I'm not even getting into Sockle. But each of these databases has a way in which you define the schema and the data. So the way Snow Leopard works is, first of all, we have ER harnesses that are custom to that database type, which means we understand that BigQuery has schemas a certain way, like names, for example, are differently put in BigQuery versus in Postgres versus in MySQL.

MySQL has schemas, Postgres has namespaces, and data sets, BigQuery has only data sets, right? So these things make it step one for accuracy. You have to know what the layout is going to look like, right? Step two is, yes, we read the schemas, but any fool can read the schemas, right? Like at this point. So that is not it. We read the schemas, but we actually have an LLM-based harness with judges and adversarial LLMs or systems that actually probe the data within the schemas, which is how we know that column 1, 2, 3, 4, 5 has floating point decimals and therefore must be a currency column or a discount column or things like that, right? Versus it has whole numbers and they're five digits, so therefore it might be zip codes, right? This is how we build that intelligence and we do all of this at the time you connect the data source. So that's number one. That's how we get our accuracy out of the box. And the number two piece is we completely agree that tribal knowledge today exists in people's heads. And so we actually provide you with an air phase to then give feedback.

We literally call it the feedback API to the system, to Snow Leopard, so that you can say, oh, actually, you thought column ABCDE was a discount number, but it's actually our price reduction number. Right. This is actually key. Your API is quite simple. Right. So you have response, you have retrieve, response, and then this feedback, right? Correct. And the feedback allows you to go from 95% to more and more accuracy, right? Correct, yes. And actually, this is interesting because one of the books I read recently, which I really like, is called Everywhere Millionaires. And it's really about two academic economists who, after grad school, went to the U.S. Treasury Department. And they were trying to understand small businesses. And they got stuck trying to understand it until one of them gave a talk somewhere in Utah and ran across the guy who built the system.

And all of the understanding around and the whole book and their entire research agenda flowed out of this guy explaining to one of them what these tables are, because it's just not documented, right? Yes. So this is my concern about things like catalogs and ontologies, because once you document it, it becomes stale, and it's always fragmented. Right. And so those are great as starting points, but in order to understand the system, you actually need to know what the live beating system looks like. So we do automatic schema detection, drift detection, and then update semantics as an example. These are like little things, but they are so difficult to deal with and require so much manual work even today. So to the defense of the rest of the industry, you're not the only one trying to do this kind of automatic semantic layer construction, ontology using AI, right? So have you paid attention to what others are doing and where do you think you shine in this area?

To be honest with you, I really don't think people are treating it as a holistic problem. So Snow Leopard is based, built on the fact that we need a new data architecture, a new data sign for AI. Well, for one thing, you're not a data management provider. So you're not assuming that everything we just mentioned, semantic layer construction, ontology construction, is limited to your data management layer, right? That's right. So we pull out from being part of any one data management system because that data management system is very optimized for them and don't care about all the other systems out there. And the reality is most big companies use many different vendors. That's right. Data silos are a fast of life, right? And so number one is that. Number two is we're actually looking at it as a holistic problem. So not just ontology and automated semantics generation, but also upkeep and maintenance.

So we have three pillars, right? We have the automated contacts, which is automated semantics intelligence, automatic and live federation, right? Governance. And on top of all of these is E-surfuse. So we care really a lot about most of the people on your podcast probably have this problem. We care a lot about how we make the lives of people who are managing these systems easier, which is why other – so other to your point about competitive analysis or landscape, people focus on just the ontology or just the schema reading or just the semantic layer overall, right? Build a graph database and you'll have semantic layer. Build it. Use this ontology system and you'll have ontology, right? But you need more than – DBT has a great way for you to do metrics. Google has a similar one, to define ontology, to define metrics, to define pipelines. But you have to do the work, right? With non-operative approaches, we'll do all the work that we can and then allow you to tweak where you need to. That's number one. Number two, there's got to be another system that will federate. I don't think any system is federating right now, by the way.

I've looked many times. Unless there's a verticalized solution where they're selling you a solution, right, an analytic solution. Most people that are in this space are selling an analytic solution. we are selling a platform on which you can build your own analytics solutions and you can build your own agent of any cunt right and so the third thing is governance which is then the governance becomes a third solution so the main point is you as the end user or the builder have to pick multiple different solutions in order to build the full end-to-end application you want and with snow leopard we're trying to make most of that like not something that you have to deal with like Pick this vendor for governance and this vendor for MCP and this vendor for semantics and this vendor for federation. So this is a hard question to answer. But how much of this unwritten business knowledge can an AI reasonably infer? I guess that's why you have this feedback API. That's right. Yeah. So we started with no human feedback is required. And then I very quickly got to SAP HANA and SQL Server and Salesforce are not that easy to read.

Right. And so, yes. So this feedback API, let's say I provide you this feedback, right? So there's two departments and they define active customers differently. But actually, department A's definition is more accurate. So then I give you that feedback. How quickly does the system adjust? Within a few minutes. Okay. So again, just because you've given me feedback or Snowy feedback doesn't mean we're going to take it, right? So we have a whole verification and analysis and judgment layer in the feedback system that will actually look at where it applies, whether it makes sense or not, and then it'll apply it. Right. And then we have like we apply it immediately to your session and then we see how the session is going. Right. And then based on that, we will bake it into the system. Right. So we are very careful because we don't want bad actors to like change definitions, obviously. But but we can take it and absorb it in real time. So this thing you gave me, just to pinpoint on that, like you said, right,

Department A's definition is different, let's say, from Department B. Whether it's accurate or not, the system doesn't know. But we can actually say this is Department A that's talking to the system, to the same system, right, to the same set of data sources. Well, you know based on my role. Correct. And so we can actually apply it for you and not apply it for the other one. So the technical solution here is we layer semantics. Well, what if I assert that actually it's universally understood by the rest of the company except for company B, that my definition is the correct one? You mean Team B, right? Yeah, yeah. Yeah, and then that's fine. If it's Team B, that's, again, we have roles. So if it's somebody from Team B asking the question or using the agent or the agent is assuming the role of Team B, we can answer that way versus not, right? So 95% accuracy sounds great. but that 5% might break my business. If I'm 5% inaccurate, it might drive my customers away.

Yeah, I think there's two answers. So then you have this feedback mechanism, right? I think, Ben, the answer is, let's be realistic. What are you getting today out of the box? Like for anybody that's doing this, right? Like if you just connect and go, which is what we're promising, which no love for it, is connect and go, right? the idea is again like you can see how far the system can get and then you can give feedback right that is the point and like you can do acceptance off the system as the administrator like we give you the tools to do the acceptance on your own but without having to build everything right and so no probabilistic system is going to give you 100 determinism out of the box our aim is to marry these probabilistic engines with the deterministic outcomes reaching as close to 100 as possible Like even humans make mistakes right So obviously we not promising anything that is like foolproof or bulletproof but we trying to make it as close to that as possible and i think today it even more possible with llms and gen ai than it used

to be even three years ago even when we started two years ago so then now i have let's say i deploy snow leopard right so now i have my agents having access to the right data at the right time but now I have a single point of failure, which is snow leopard. If snow leopard goes down, I'm screwed. So what do you say to listeners to say, well, I'm going to wait until Veepty and company have more of a track record. I know that this is hardened and so on and so forth. Yeah, I think nothing I say will make somebody change their mind about what risk they're able to take or want to take. I will say try it. Right. And the other thing is we have been with financial services companies that have been using us. And so I think the point is we work with you. Yes, we are an early stage company, but we work with you in order to get you to have the kind of reliability you want. Right. And then because our solution is BYOC, we can be inside your cloud. We can be on our own cloud. Right. Like so we aim to be the strongest link in your chain because clearly with data, there is a chain and there are weak links and strong links.

I want us to be the stronger link on the chain, not the weak one. So what are some of the DevOps numbers that people want to know? So uptime, latency. Latency depends on the source system to a large extent. Concurrency. What if you can only support two users at a time? Yeah, I don't think that's a matter of, like, that's not the case, right? Because we are a, from the name of the app, built to be like cloud. resilient, right? And so we have high concurrency. We can support hundreds of users at the same time. The on-prem deployment, again, concurrency is a function of how much resources you give to it, right? And so the on-prem deployment, we can parallelize. We have built a system to be parallelizable and highly efficient from the ground up, right? We are a bunch of systems engineers and AI engineers to start. So then we've built it that way. I think in terms of latency, See, like we add like some microseconds, if at all, to those responses.

We are faster, by the way, 10x faster than MCP directly. And you don't burn all these tokens, right? And we are half this token. Yeah, I was going to say that. Like we're 10x faster than MCP and half the tokens on average. Again, it depends on what you're doing, but we're half the token on average. But also you have this federation, which is key to, I think, a lot of making agents useful moving forward, right? Yes, I truly believe that without having federation in some format, people will not be able to use autonomous agents for critical workflows, right? Because you really do need to have access to the right systems for the agents to really be autonomous. So since you've made federation respectable, I have to point out that Databricks, who I'm an advisor to, has also made another thing respectable, which is, what is it called? The hybrid transactional analytic systems, right? So unifying OLAP and OLTP. So let's set that aside. It works well, but you have to be in the Databricks world, right?

So another approach besides this unifying transactional analytics storage is, I guess, streaming? Are people trying to do this with streaming? I think they're trying to. From the people I speak to, I spoke to a head of data and AI in MarTech. It's the second largest MarTech firm in the U.S. And they were like, it just doesn't work, right, for them, like for a lot of use cases. So maybe explain to our listeners what we mean by streaming in this context. Yeah, so the idea is, going back to Kafka, and I think all of this is pre-world, pre-AI world is what I'm talking about, right? Like pre-gen AI world, to be specific, where consolidation was the only way to do a bunch of things because we hated federation for various reasons. And so one of the things was you stream data out of databases and into a warehouse or into a lake. And in the stream, you can do a bunch of cool things like attach SQL on top of the stream. So you could like, you know, make queries happen for dashboards or things like that.

Right. And then you have high, like faster streaming or broader streaming. So you can have multiple databases that are pushing to the same data warehouse or lake. And then you can then do sort of transformation in those streams so that you can like basically have views on this like data in motion. So you can like make faster decisions than you could with like only the warehouse, right? How does this help me in the context of an agent trying to look up something like an order? How to be honest with you? I don't know. Because like how reliable is the stream? So you're doing a bunch of manual work, and it makes for those people who are doing a bunch of manual work to build functionality around these streams, like maybe it works for them for specific use cases, right? I think the point I'm trying to make is... But it's very hard to every use case you have to do it. That's right. It's custom. It's brittle. And you have to, like, any column that was added to the table, source table, now you have to change the stream, because streams have definitions,

to add that and then put it in the warehouse. Well, I have an AI to do the semantic layer for the stream. Yes, yes. And to like not have to, I think the point is people started using streaming as the way to query because putting it in the warehouse was taking too long. So this is again, another way to say near real time, near real time. It's not real time. And also you define streams again for specific workflows. Like data pipelines are designed to be specific. So if you try to use a data pipeline or a data stream for everything at all times, it's going to break because it's like, you know, it's like round peg square hole situation. Right. So the way to build applications on top of streams became a way to do sort of fresher data. Right. Like not have staleness because it's stale by the time it gets to the warehouse. But the other problem with that is it's transformed. And so you've lost meaning in the pipe because you had designed that pipeline to be for analytics, for example, for historical trend analysis, for example.

So you've made the columns and rows picked specifically for that use case, right? And so now you don't have the full picture. So if you're going to put something that is a decision-making agent on top of that stream, it's not going to have the full picture. It's going to give you wrong answers, like just point blank, right? So what I like about what you do is you're not making copies of the data, right? Yes. So you're hitting the data where you need it. And then you make the reasonable assumption that, hey, data is not going to be conveniently located in one system. It's going to be located in multiple systems. And I need the freshest data. So I don't want that data to be copied first and then before I hit it. All of that is well and good. But obviously, one thing that people have been reading about is these agents going rogue scenario. Right. So now I have a scenario where these agents can interact with an application that's hitting my live systems and making all sorts of potentially nefarious requests.

So, yeah. So what I guess what you alluded to this earlier, you have the notion of governance and all of that. So what are some of the things you can say to reassure listeners that this doesn't open up their systems to rogue agents? Yeah, I will say, like, if you're using MCP today, you're already open to rogue systems, right? Because MCP, by definition, on its own, is just like HTTP, right? It doesn't have any of that governance built in. So I actually feel scared when people are using MCP on top of their databases, right? And then, because once you have MCP, there's just two APIs, right? So all it does, it has access to all of your data and all of your schema. Unless you do custom work inside the MCP or on top of the MCP, right? So, again, now we've gone to custom work. So just as like a rule, I think by itself, MCP doesn't help you solve this problem. So then people are, as I was saying earlier, vendors sell governance on top of MCP or governance built MCPs, right? That then you have to configure. So it's no leopard. The point is, because we sit outside of all of your data sources, like in between your agent and your data sources,

we can actually be that person, sorry, the system in the middle that can arbiter, like I can route things to different places and only when it's allowed. So one of the things I talk about in this ACM article is like governance on not just roles and access control, but who and why. Because an agent can impersonate. In fact, that's how most agents work today. You impersonate somebody else's role. I mean, people are going to start using agents like Muse and Dots, right? Yeah. I mean, the thought is scary without governance. And so the point is, because we sit outside the systems, when an agent makes a call, the agent itself can tell us and needs to tell us, like, who it is and what it's trying to do, right? Team A versus Team B in your earlier example. And based on that, we can, like, federate maybe Team A only has access to… Or what about if I'm a Muse agent acting on behalf of Ben? Yeah, same thing, right? Like, because, again, if Muse has to interact with Snow Leopard and you… So there's two things happening, right? Like one is Snow Leopard knows it's Muse and therefore has access to Ben's credit cards, but maybe not direct bank account and direct sell.

And so we will not let Muse do that, number one. Number two, so what that means is we arbit are where you're going to go, or we have the ability to arbit are where you're going to go based on who is asking the question, which agent and on behalf of whom. Second is you can actually set up Snow Leopard to have access to only certain set of data to begin with, right? And because it's just a matter of connect and go with Snow Leopard, you can just add or remove not just data sets and data sources, but tables. So, for example, you can give today access to Snow Leopard to one of your, let's say, BigQuery data sets and some of the tables in that BigQuery data set, right? So we use a combination of existing data-based access control as well as agent-based, like, access control of, like, what are they trying to do and why are they trying to do it? and then push that down to get the right information, right? So because this stuff is outside, even if Dan has access to all of their financial systems, all of your financial systems,

like maybe the agent news does not. And so we can enforce that because we're sitting outside. So as I opened this podcast, I used the word retrieval. So that implies read only. Correct. So do you think that there will be a future where Snow Leopard can update these databases or never? 100% yes. It's just a matter of giving read access versus write access. I think right now, based on where the world is at, read itself is hard enough and scary enough. It's scary enough, yeah. Yeah, I think the scary part is more important. Like, select versus update are just permissions, honestly, to a database. The way I think about it, like a build database, is just about whether you can do select statements or whether you can do update statements. And right now, we're not allowing Snow Leopard to do update statements, but we can. It's just a bit set. Again, these things are known. I just don't think there's enough trust in touching, not trust in Snow Leopard, but just trust in AI and autonomous agents in general. And we can, this is just a technology problem, and that's very easy to solve. I think the trust problem is a different problem.

So this is probably something that is outside your system. It's more of a question for security teams, right? But when an agent uses Snow Leopard to retrieve information across these systems, whose permissions is it using? Right now, the permissions are either granted through a service account or through a role that Snow Leopard is given access to. So, for example, for BigQuery, we use OAuth and SSL. And so then it becomes whoever the user is. But if you set up a service account, then we can use the service account. Right. So it's up to the user administrator, really, to set up the right security role right now. And I don't think the security problem is solved. I think what I'm excited about for us is that we've built a system in a way in that we can actually add on to security and governance practices as they evolve for agents. Right. Because, again, we are not tied to one system. So we don't have to have if you are in this world, then you can do this. Otherwise not. Right.

Like we are very easily evolvable. Like I believe in evolution because AI agents and autonomous agents themselves are evolving. And so I believe in evolving the system to that. And I think the way we've set it up, there's a lot of room for us to evolve as the industry evolves. So this semantic layer construction, automatic semantic layer construction, it seems like, so do you think most companies have systems in place where when you come in and you try to infer their data, the semantic layer automatically, do you think their systems are well architected enough so that out of the box, the semantic layer is reasonable? Yeah. Because the reason I ask is there these teams in companies One of the most awful jobs in these companies is this thing called master data management Have you heard of this Right So it like a manual A lot of people doing all of these things manually, right? Yes. So then, to me, that implies that maybe because their underlying systems are a mess.

So if their underlying systems are a mess, then how well will this automatic inferring of the semantic layer work? Right? Yeah, I think this is a common refrain from people who are on the data side of the world, right? From people who are trying to build agent side of the world, they're like, just give me something because right now I have nothing, right? So yes, data is a mess. I think it's messier than most people think it is. But also, there's a lot more we can do. Like it is surprising to even us what we can do with the harnesses and the way we've built them for Snow Leopard, right? So we can actually get a long way through. And this is why we added feedback, because whatever the last mile is, because nobody is guaranteeing that data will not be messy. I think there are two things. One, getting started is the hardest part with this data, because if you're going to have a two-year-long project to move everything onto a system of choice. So you guys come in, you infer the semantic layer, and then I assume the first set of users are people in the IT group.

Yeah, the administrators, yes. So then they're trying to give you feedback before they deploy, right? That's right. And the thing is, you can continue to adjust, right, very easily because the APIs for the agents are very simple and the user interfaces for human beings are also very simple, right? So if a marketing department, if IT approves something for the marketing department, the marketing department can then go off and fine tune it. The other problem with these master data management jobs is that there's few people trying to address the needs of the entire company. Right. So if you can set it up so that each team at least can use something without you having to worry about it and without you having like a huge queue of work for each of those systems, I think that's a huge unlock for the company as a like overall. Right. And then the other thing is, like, I want to just sort of bring it down to, yes, data is messy, but we can do go a long way. right with that without too much human intervention and then you can add the human intervention to tweak which is a time saver versus start from scratch second you can allow

because of the way the apis and and the feedback is set up even people who are not in the it department can give feedback in order to like adjust semantics etc right so you're not constantly going back to the it department very quickly i'll give you an example like again this fintech company we worked with we worked with the founding engineer who is also like basically the head of data managing all of this stuff and he could reduce his workload because he could just point people to the snow leopard based agent to answer a bunch of questions instead of him having to build dbt pipelines and then writing the sequel and then iterating over the sequel and then giving them the answer like and then having to do that for every team and every question all the time as i As I understand it, you have some sort of tool for explainability, right? So in other words, when I first deploy you and I'm in the IT team, not only am I giving you feedback, there's a certain level of explainability. So I'm reassured that, okay, so it's hitting the right systems.

It's writing the right SQL. That's right. Yes. And that sort of built-in explainability is sort of right in front of you. So if you look, so the API returns it, by the way, so that even if you build autonomous agents, agents can verify if you put the verification loops in there, by the way. So we have APIs for both the agents and the humans. The humans can see, this is on our website too, the human can see the SQL being generated, the data sources being targeted, and the reason why and how we built, like, so it's national language explanations of the SQL query we built and why we built it, right? So you're talking to your agent or your agent is talking to Snow Leopard and it can see what happened. So you can say, oh, it builds SQL with this table and this column, but actually it needs to use this other table and this other column for this specific question on churn or on profit. Right. Because these can be very, very ambiguous terms, as you said, different for different people. So what feedback have you gotten about this explainability tool? Is it good enough or do you have plans to tweak it and improve it?

Both. We have gotten feedback that is better than what they've seen. So they're excited about it because they didn't even know that you can get this kind of like detailed information. And then the common one I get is, can you draw it as a picture? Can you build an ER diagram? Right. Right. And I think that's like or a graph or something that says we went here, we went here, we went here. So that's something we are considering adding. We can also do it custom for like, you know, especially for messy data or for large teams. Like it makes more sense to just visualize it very quickly, especially the ontology part of it, the semantic layer part of it. Because it is hidden and a lot of data people are like, I don't like hidden. So, you know, showing it in a visual format is something they're considering for sure. But I want no feedback. And if people are more interested in having it one way or the other from your audience, like I would love that. So please let me know. So if someone makes an adjustment to the schema or gives you feedback and therefore you have to make an adjustment, so some adjustment is made, right?

So can you go back in time and understand, okay, so we showed Ben this last week, but we should have shown him this in light of what we know now. Yeah, that's a good question. So we have auditability because, again, I come from the day world. I don't like it if I don't know what the steps were. So, yes, we have like auditability where you can see even today in the cloud product today, we have some auditability where you can say this person gave this feedback and it was applied to this database. Right. And once it's applied, it means it's there. Right. Now, do we explain or have the whole stack exposed? I think we have it exposed to us in the back end from a debugability perspective, from DevOps perspective. I don't think we show all of that in the front end. But again, we are early, we're very open to feedback on how people would want to see it. But again, we've built a system because I know our people want to see more and more and more and have more transparency. So Snow Leopard is basically all about SQL databases and data and SQL databases.

So any plans for other types of data? Yeah, it's not just SQL databases anymore. It's Salesforce as well, which is SOCO, is an API, and, like, graph databases are not SQL. So we will add as many connectors as we need based on the customers. So far, we've had mostly folks who want to connect or join across sort of one of these databases, right? SQL databases and Salesforce seems to be the most popular API, but we are absolutely set up for APIs as well, structured APIs, not blob APIs. Never unstructured data. I just think there's a whole body of work in research and vendor. Vendors out there that are doing that. I think this is a hard problem, and I have expertise in this. I just want to solve this problem. So if you look three years ahead, what will the data stack look like? Obviously, Snow Leopard is right there. Yeah, I think the data stack needs to have agents, a data, this is another bad word, middleware, but data middleware, right? and then the data sources at the end, right?

So one of the things that I didn't say earlier is, you know, we are not saying no to warehouses. If the data exists in the warehouse, we can go get it from the warehouse too. Really, it's just a question of where is the data? And like, if you have a RAD vector store, basically for RAD pipelines, like we can query from that too. That's not a big deal either. I think the point is, federation is key because you need to know where the data is and how to get it from there, right? So I imagine the stack having governance as a pillar where the governance is around both the data as well as the agents. So not just what, but who and why are one of the key colors for the data stack for AI agents. The other is just federation has to be key. And so it needs to be really, really easy. It cannot be consolidate everything into one place. I just don't think that works. I mean, I love I'm not saying what Databricks or Snowflake are doing are not great. I think it is, but not everybody's data everywhere is in one place. And so federation has to be key, live federation. And then the third thing is this machine-readable automatic understanding of the business context, which we call the semantic layer, but really it's automatic understanding of the business logic.

Has to be a part of it with human feedback, whatever form that takes, right? And regardless of no effort, I think these three are the key pillars in order to make autonomous decision-making agents work. And by the way, if agents actually start writing more code, so there's no reason why they wouldn't write code for the data teams and the data engineering teams. That's right. So which means actually the data stack itself will change constantly because maybe, you know, because now you're coding, you're using coding agents. Oh, let's rebuild the schema in the database tomorrow, right? So fire up agents. Next week, we have a completely different schema and whatever, right? Or we never used this piece of infrastructure because we didn't have in-house expertise. Now we can use it, right? So then people might be constantly making adjustments to their data infrastructure, which means this middleware might actually be super important, right?

As long as the middleware constantly infers all these changes, right? But this is my point to you. I think we're talking a lot of the concerns that you brought up, which I hear these are legitimate concerns, are from a world where humans are curating carefully all of this stuff, right? But we are going to an Asian first world, right? Some of us are already there. Others are getting them, right? And in a world where people are throwing away code. That's my point. In an Asian first world, it does not matter what the pipeline is, what the code is, because it's going to change every time. And so that's why we have automatic semantics and automatic schema drift detection, right? Because we have to actually update based on what the agents are doing, right? So if the agent is changing the data underneath you, like, you didn't even know what it was doing, right? Because you care about outcomes. Muse is about outcomes, for example, right? And so to your point, if everything is changing, the middleware has to understand that something changed, semantics changed context for you know getting data from the right place has changed and so

all of this just has to be like automatic at machine speeds not human speeds and with enough verification where governance comes in right so i think the agent first world requires us to be in this sort of three pillar system right for data especially and yes like we will get there in time and so we do need to have human interfaces. But agent first means agent interface, agent speeds, machine-readable stuff, and not worrying about everything always being hand-curated because it cannot be. And static, right? Because I guess the world I'm imagining is these data teams will become bolder and move faster. That's right. And they will throw away pieces of their infrastructure more quickly. And then so you need you need this kind of snow leopard layer so that your agents can access the data as the data team is constantly ripping out and replacing things underneath you.

Right. Yeah. And you have to be efficient with that. Right. So I think static is a great word. Like I just don't think we are going to go from a static world to a dynamic world. We're already going there. If we believe like agents are going to create and undo databases as a result of whatever work they have to do or as part of whatever work they have to do, then these things become all automated. Right. Like the middleware has to exist and has to have strong boundaries and automation in order to enable the data teams to do that. And with that, thank you deeply. And you can learn more by visiting snow leopard dot AI. Thank you, Ben. You can follow the work of Deepthi Srivastava online at snowleopard.ai and you should definitely check it out. I think it's going to be one of the more useful things you will be trying as far as building agents. Please join the thousands of people who subscribe to our newsletter, which you can find at gradientflow.com slash newsletter.

Thanks for joining us. If you like the show, please subscribe and rate us through Apple Podcasts or Overcast or TuneIn.com or Spotify and never miss an episode. The Data Exchange Podcast is a property of gradient flow. And I'll be back next week and we will do this all over again. Thank you.