クラウドの時代は終わったのか？ハイブリッド基盤とエージェント時代のデータガバナンス｜Sergio Gago（Cloudera CTO）

Generative AI is transforming industries at an unprecedented pace. But as AI changes how you or your team work, one thing is clear. Your skills also need to evolve. At DataCamp, we offer everything you or your team need to adapt and thrive with AI. Whether it's business users looking to get the most out of ChatGPT and Copilot, or developers and data scientists looking to fine-tune models, you can learn the entire AI skills spectrum on DataCamp. Power your AI transformation today. Start learning at datacamp.com. banner in that. Sometimes you want to run AI in your own data center for privacy or cost reasons.

That means organizations can end up with compute and data scattered across different platforms. Today we're going to figure out how to make that work for your human colleagues and for your agents. But we see more and more, 66% out of 1,500 respondents, CIOs, CTOs, CTOs are saying that they're moving workloads into their data center. But the reality is that if you start to give agents to all your employees. You don't have 10,000 people in your company anymore. You have a million people in your company that go deeper, faster, and can collate and coalesce more tables than ever before and can identify things and other stuff and patterns that no human has done before. Our guest is Sergio Gago. He's Chief Technology Officer at CloudEra. Sergio leads technology strategy focusing on trusted data infrastructure for enterprise AI. He has over 20 years of experience across AI, machine learning, quantum computing, and data architecture. Previously, he was Managing Director for AI and Machine Learning and Quantum at Moody's Analytics. He's also held CTO roles at Bakuten, Capacity, and Xenio.

Let's learn about managing messy infrastructure. Hi, Sergio. Welcome to the show. Hi, Ruti. Thank you very much for having me here. Super excited to finally be on the show that I've been listening in for quite some time. So thank you. It's great to have you as a listener. Yeah, dreams come true today. Awesome. So the last couple of decades, one of the big stories has been almost every company moving everything to the cloud. Has AI changed all that? I think it has completely changed the whole paradigm. Not just someday on how we think about infrastructure, but about how we think about where beta resides. And I was one of those CTOs some time ago, some six or seven years ago, who said everything's going to be on the cloud. Who wants to manage data centers? Let's move everything, right? You know, make your pick on the hyperscaler, but it makes your life way easier. Scalability, sustainability, all those things.

It is phenomenal, right? And the hyperscalers do a great job at making it simple for you. And as a company, we started losing a little bit our ability. to manage the data center. In fact, go and ask any sysadmin or DevOps under 30 or 35 how a rack actually looks like or how to change a blade in the rack, and they would be oblivious to it. So for a long time, many people, and myself included, thought, that's it. Sure, the cloud is someone else's data center, but everything is so well managed that you can focus on your business, on your domain, on creating value with it, and not necessarily on the bare metal, nitty-gritty details of the systems. But now the world, especially with the push of AI, has changed that story completely. And enterprises and public institutions, not necessarily smaller companies and small startups, but large companies are having to rethink how their whole infrastructure looks like and how to essentially play with everything together in a nice way where they complement each other.

The good parts of the clouds, the good parts of NeoClouds, the good parts of sovereign cloud providers, and we can talk about that if you want later. And also the data centers, the on-premise, the private cloud elements that are fundamental for many, many enterprises when you think about data privacy, AI privacy, sovereignty, and many other considerations, including actual cost and cost management. Oh, man, it sounds like things got a lot more complicated. I mean, you mentioned like five different locations for your data as well as for your compute. How do you decide between your cloud, NeoCloud, so for an AI, on-premise, whatever? What's the decision process? That's a phenomenal question. And for a long time, data companies like CloudEd ourselves, right? But essentially every data platform, we all had the premise of what I call data hoarding. Just bring your data to me, to my system, whatever it is, right? On the data center, on the cloud, on one hyperscaler, SAS, PASS, you name it,

depending on the flavor of what you want or what you need. Just bring everything to me, to my storage, to my catalog, to my metadata system, to my lineage system, and everything is going to be blissful. Everything is going to be phenomenal. And it is, right? But over the years, we have moved our data platforms, everyone in the industry, from point solutions, databases, components, you know, the Hadoop systems, the Impala systems, the Spark jobs and so on, all the way to holistic data management platforms that help you extract value from your data so that you can power your agents. And that is true for DataBridge, Snowflake, Microsoft Fabric, for Cloud Data, for everyone in the field with our differences. I am not sure you can choose anymore where your data is going to be. think about a typical enterprise you've had years decades of amassing data in different places and then your large bank goes and merges with another bank

that comes with a completely different data state with a completely different footprint tables and so on it's not going to be possible for you to merge those things because those are multi-year endeavors but often times end up in failure So the world came out of ThoughtWorks with the data mesh concept, right? And all right, let's federate things and let every business team, every business unit kind of figure their things out. What is the implication of that at the grounding layer, at the infrastructure layer? So all in all, I don't think people can anymore make those conscious decisions without the push from the world. What do I mean by this? For a long time, we moved a lot of our workloads into the cloud. It's scalable, it's flexible, it's elastic. The trade-off was that the governance and the cost control was not as easy. But we are okay with that so that we don't have to invest heavily on our CAPEX,

on our data centers, and so on. But now AI comes and offers us a total cost of ownership that is orders of magnitude cheaper when it comes to GPU management and LLM management on the data center, right? So it is just natural that a CIO comes and says, all right, maybe we need to have a hybrid solution. Now, what do we do with the data? Do we proactively keep it on our S3 buckets or whatever, or do we duplicate it? every single company or every single enterprise that has been on massive cloud migrations, because many companies are still on their cloud migrations, most of them did not remove completely all their data center footprint. So they ended up with two universes. They called their cloud universe with their specific platform engineering teams, DevOps, and so on. And their data center universes, typically under IT,

managing and running their own stuff in there. What has been happening in the last few years is that CIOs and CTOs are realizing, hey, hold on, we still have a lot of compute available here that is up there for grabs, and it's useful, and it can be utilized perfectly fine. Why are we still consuming OPEX here with growing costs when we can leverage all these data centers here? And that is not even considering when you have specific legislation that are forcing you to give you data on a certain region or in a certain country. That's simply a mandate that you get for these things. So to answer your question in a long way, in many cases, it's not even optional. You end up having an absolute mess of data states with different forms and factors and shapes, different ways of management, different skills and knowledge that you require in order to support that. And there's no way out of that. So companies are trying to figure out, all right, how do I keep a data platform that allows me to build agents,

to really leverage all what AI can give me and the competitive advantage that I can deal with it. But with an agent, it's able to access all my data, not just a chunk of it, not just a bunch of it, not just a vector database that I have hydrated somewhere, but everything that I have on AWS, on Azure, on GCP, on these three data centers, and a small rack that I have in the office with the CRM that was live in the late 90s and has critical customer information. If you as a human can access that data somehow, even if it's with a USB stick plugging in there and downloading the data, an agent should have those same capabilities. And that's where the true hybrid component of not going back to the data center, make no mistake, is about considering all your data state, all your infrastructure as a holistic layer of data access. Okay, well, I agree with your point that trying to synchronize things like changing from one data warehouse to another, it's really hard.

like uh data camp did this like back in the early days we we switched from redshift to big query and it was a whole thing that took months and that was when data camp was like a really tiny company so i imagine like if you're like a multinational company and you've been around for decades it's going to be a horrible process trying to synchronize everything uh so how do you deal with that if you can't synchronize and standardize your platforms you do end up with this like really heterogeneous infrastructure like how do you even approach managing that uh once you bring agents into the picture? I think that the key is not thinking about having to move everything to one system or having to hydrate different federated systems and try to synchronize your policies. When it comes to agents, the challenge is the trust. Probably we all used OpenClaw when it came out and we all went nuts with it. It's amazing and so on. And then we all faced the risks and limitations on access controls and so on. And then we took a couple of steps back and said, all right, let me use it without giving

it access to all my life, all my email, calendar, bank accounts, and so on. And then we realized, oh, well, if the agent doesn't have the data, it becomes pretty useless. So how do you consolidate those two things? Right on one hand, as we're saying, your data is all over the place. If you're lucky, you have a nice catalog that at least tells you about your metadata, where this stuff is, and who has access to it. If you're lucky, those companies that did their homework in the era of machine learning, they already have, many of them, kind of a layer of data governance or a data fabric, if you will. The companies that are trying to jump in the bandwagon of AI that did not have a data strategy before, they're up for a challenge because you cannot do this without having proper inventory control lineage and essentially visibility of your data state. Again, whatever that data is, right? The only difference with agents is that when you need to access any piece of information

as a human, you can always pick up the phone and call John from the HR department and say, can you give me access to that database? Can you give me a snapshot of that? Can I plug in a few ETLs so that I can dump all the data on my data warehouse? You know, the usual things. But an agent doesn't necessarily have that power anymore, right? So the only thing that agents and AI is doing is essentially pushing from behind, making sure that if you want to do this, then you have to do all these things. In the same case as OpenClaw or any other agentic system or any other agentic evidence, it's all about the access controls, the permissions, the governance in general about the system and on the other hand, the data access. We just released in Cliletta a report a few weeks ago We call it the great AI re And we were asking our customers 1 of them essentially that question What is the one thing that is really

humpering your innovation in agents? And it's not anymore the procurement with the AI companies, with the labs, or the access to the LLMs and so on. People are building their experiments and so on. It is, can my agents access the data? and can I give them the access to that data in a safe way and in a safe environment? So you put all these things together. And then you realize, well, if that is the case, the actual database, the actual data lake or data warehouse, or even the engine is not that relevant anymore. And don't get me wrong. Spark is super important. and your engines, your click house, your star rocks, all those, your DAX-to-be, all those things are fundamental, but they are just one piece in the middle where the agent is just going to throw queries at your trino, for example, for federating queries. Now, how do you make sure that that trino that federates against SQL servers,

oracles, Impala, Hive, can have a proper orchestration level around them on what I call the second derivative model? So what do I mean by this? Imagine you have a couple of data lakes, just two to simplify things, and then a bunch of data warehouses, either Telavidas and Oracles, you name it. The usual thing. And then you have a bunch of things on the cloud because you've been on a five-year journey on moving things to BigQuery, for example. So you are in what I call the rough oceans of ever-migrating strategies. Every company is in a continuous migration because you never migrate everything to the cloud. And then there's new things popping up. And then you have shadow IT or shadow data teams popping up, new things up and down. And that's the universe that you have to manage and control. Your internal stakeholders, your lines of business inside your company, they would want to use the latest systems, the latest engines.

Some of them will come and will say, I have this time series use case. I really, really need to use InfluxDB. Maybe your data team does not support InfluxDB for that case. It doesn't matter. They're going to use it, whether you like it or not. And if you don't support it, they're going to set it up in their own cloud account, and they will hydrate it, and it will be completely out of your data fabric system. So, of course, in that situation, if an agent goes and uses that system, it's outside of your data fabric, of your control. So the agent will not know, can this user access this data? In other words, it doesn't matter what you deploy and where you deploy it, as long as you have a whole umbrella that has that visibility and control. There hasn't been another moment in the history of data where your data fabric, your governance layer is more important. Now this is the number one thing that everyone needs to build. and then underneath that

if you want to use this query engine or that query engine sure that's going to give you some milliseconds of performance or it's going to give you something or not but that's it's becoming a commodity as we move forward that's fascinating and again it sounds like the whole situation this bit with how you manage it how you process data it's got a lot more complicated now that we throw agents into things now you mentioned the term data fabric a few times and so this is one of the big ideas I guess rewinding a few years back before agents was like if you've got federated data like there's lots of different data all around your organization you can either deal with it with a fabric or data mesh so these are the two big ideas do you want to explain like the difference between them and like what actually works now so the beauty of the data fabric is that there's one definition for its data professional in the world today, right? And so I can say anything I want. The way I believe, and Cloudera, we think about the data fabric is precisely about that

governance layer that helps you control what's underneath. Typically, we thought about data platforms as a collection of storage, data formats, and data catalogs, and engines that access your data. And then artifacts that you can create on top of that. Your artifacts could be pipelines, workloads, ETLs, machine learning algorithms could be BI passwords, you name it. But that was your typical left-to-right funnel of the data pipeline. When the concept of Data Fabric was introduced, it was more about how do we control where the data comes from, how it is transformed along the way, and how it is consumed by that and by whom. Essentially, how can you trust the data that you're consuming? If you're a CEO and you have a BI dashboard for revenue or for revenue forecast, you need to trust that the origin of the data is correct. You need to trust that the transformations of the data have been correct as well, that the data has been manipulated in the right way.

You need to make sure that you have the right access to that data. If you're the CEO, probably the answer is yes, but if you are a mid-manager looking into HR information, then that probably has some fine-growing access control that needs to be added into the system. And then you need to trust that the models applied, in the case of a forecast, can be trusted. So those are the four elements of trust that you have on traditional data pipelines. Now, when you add an agent into that equation, essentially, it is no different than that. An agent is propagating a human, but there's one difference that I call the second derivative. And what do you mean by the second derivative? If I go and ask my chief HR officer, hey, give me the information about my last payroll, she will be able to give me the answer, right? Because she has visibility on everyone's salaries and everyone's payment information and so on. She needs to have that information for her work.

But then she's going to give me, Sergio, the information about me only. If I go and say, can you give me the information about Richie? So you'll say, no, you don't have access to that info. All right, we have proper access controls on the system at the human level. If I build a little dashboard that gives me information about my, you know, tax, etc., about my 401k stuff, I can build a dashboard that follows that set of access controls and permissions easy to easy. We've been doing this for many, many years. But now if we build an agent that supports HR with those type of questions, think of an HR chatbot, right, or an HR agent. And the same can probably apply to a customer support agent. This agent, to perform its work, needs to have access to the whole information, to the whole universe, but needs to make sure that the person that's speaking with has a set of access permissions.

In that case, I speak with an agent. I can ask about my salary, but not about which is salary. And the agent needs to have everyone's salary information in mind in order to have that information. Now, it gets even more complicated when the agent is not speaking with Sergio, but is speaking with Sergio's agent. Or with a collection of agents that are collaborating together in a cloud code environment to build something, right? So the system of permissioning, access controls, fine ground access controls, data privacy, data tokenization, all those elements that we solved very well for the unidirectional data pipelines that we created, now they have become multidimensional and multicyclican, not acyclical anymore. That's why the data fabric needs to evolve in that level. And then the engines that are underneath. if one agent asks a question

and that question needs to create a Python script that runs a SparkQL query that has a fine ground access control over data that you have on Iceberg on an S3 bucket or an Ozone bucket that is irrelevant at the end of the day important but not very relevant but that's actually kind of terrifying Because with the HR example, I mean, it's such a high risk thing because if you get the permissions wrong and the agent starts blabbing confidential HR information, you're going to get sued at the very least. And there's going to be all sorts of repercussions. I love using that example precisely because it is the one thing that compliance teams and cybersecurity teams and legal teams are always saying, oh, this use case sounds fantastic. It's the whole factor of generative AI, right? You can save up 30% of customer support team

or HR operations team or this and that. That sounds phenomenal on paper, but then what if, what if, what if, what if? And that's when we've seen these many Gen AI cases that never see production. That's the reason why, right? Because there is not a true data fabric around it that puts those controls in place so that we can safely run those environments, again, at large enterprises, large companies, and public institutions. Absolutely. So if I understand correctly, basically the permissions are going to be a property of the agent and also who the agent is talking to and I guess also the task as well. So how do you manage all these different dimensions and make sure you get it exactly right? Like what do you need to do to set all this up? So this has to be part of the orchestration system itself, right? It requires a re-architecture of your system, both at the access level and at the permissioning level and at the scalability.

So the first thing is considering agents in a similar way that a human. When you go to an office, you would have your key card or your pass, and then that would give you access to certain areas of the office and not to certain other areas. We have that nicely created generally from an IT perspective and security. the agents need to have exactly the same type of hardness. The second part is what we call bringing AI to the data and not the other way around. In the last four or five years, the strategy has been, hey, I have my data here or scattered in the case of companies who didn't have a proper data management figure, who didn't figure out their data management, but they had all that stuff scattered around and then they brought the data to AI. They would create a nice chatbot with a RAC system, with an agentic framework and so on, and they would start, you know, hydrating open search clusters or elastic search clusters and pine cone databases with vector databases

and knowledge graphs and so on. Beautiful architectures, but that didn't have any controls or considerations with data access policies that in most cases were already in place in their data platform. So the first part is bring your AI to the data and not the other way around. Second part is making sure that the agents have bounded controls and accesses in the same way that you would a human. So when you create your policies, what there is on your typical systems, right? And in Colorado, we have what we call unified data fabric system that considers agents almost similarly to a human. And once you have that, every query is not only audited, but also follows the right access controls for who the agent is and who the agent is talking to. And that's baked in the whole system of what you create. So when I talk about orchestration systems,

it's about can you access all your data regardless of where it leaves? and for that we have great tools on data access, on query federations and so on. Open catalogs and open table formats, fundamental to this. So on a strategy perspective, you can either go your way and create private systems and licensed systems that are gatekept or that on open source elements, on open catalog formats and open table formats, that essentially make you, the owner of the data, the final responsible for that. So you can manage that by yourself. That's the orchestration layer, where the agent needs to live, essentially. Okay, so I like that. So there's basically a governance layer that, I guess, is fundamental below the actual agent. So any talent agent wants to access the data they got to go through the governance layer in order to actually get the right permission Is that about right Fundamental So tactically speaking for a long time we used tools like Ranger for example to store your policies

And whenever you wanted to run a query on your data warehouse or in your data lake, the system, Ranger itself, will say, who are you, what are you accessing, and where are you going to post it? And where is the data located? And, oh, by the way, you're accessing PII, so let me tokenize it so that you can get a masked, on this column, you're going to get a masked version of the customer email because you are not supposed to see the final email. And many companies have built really good capabilities at creating those catalogs in such a way that they can ensure that safety. That's how major financial institutions, oil and gas companies, public institutions and governments can trust different people? We were talking about data mesh strategies before. How do I make sure that this line of business in my company, a company with 200,000 people, I give access to 20 data scientists on the other side of the world, the business-specific

data set for them to do their work, because I want to empower them, but I want to make sure that they don't create a mess, because suddenly they have access to all the salary information or critical information or material information about my company. Think about public companies, right? Many people are not allowed to see the final financial results, even internally, until the day of the earnings call, right? Those systems exist today, whether it's with Ranger and other tools. We have to make sure that we expand these tools so that they understand the second derivative process of not who is accessing the data, but who is serving the system that's accessing the data, and that everything is properly audited in between. And there's plenty of solutions in Cloud. Our flavor is called Unified Data Fabric, that is essentially a way to create that fabric thinking about agents, the systems that will start accessing your data or that are accessing your data, orders of magnitude

more and deeper than humans. Yeah, that's that. I think we're basing you at a point where agents are doing a lot more analytics than humans if they can do it so much faster and get into more depth. Okay, so who's responsible for setting all this up then? Is this like, is it an InfoSec team thing? Is it a data team thing? Like, who gets the blame when it all goes wrong? Typically, in large companies, that's the central data teams, the ones who set up the platform and create the permissioning system and so on. in the same way that you have your SSO systems, your OctaS and whatnot, right? Who can access what? So you have a central team that controls and coordinates that. I'm talking about large enterprises, of course. So that's what we see mainly, and that's their main responsibility. Now, the challenge is that now they have a lot of pressure to deliver value from their data with AI. What do you mean by this? In the past, you would have, I don't know, you would need to extract some data, some revenue forecast.

You would call your central data team. You would say, can you put that together for me? And maybe they need to hydrate a new database or add a new data set or do some transformations, maybe create a new model for a forecast. And in a few weeks, if you were lucky, they would go and create that for you, right? So they would be responsible for extracting value from the data. When companies move more towards the data mesh model, essentially the data team had the underlying factory, but it was that line of business, the one saying, just give me access to the data, I'll create my models, I'll do my stuff, and maybe I will add some additional engines and so on, but I will figure out myself what do I do. The orchestration layer, the governance layer that we talked about, that's still under the central team, that has not moved. Cyber security will check that the system is secure. Governments and compliance will check that the permissions are set at the right level. But managing that tool and platform has traditionally been part of the central data team.

Oftentimes under the CIO or the chief data officer by himself or herself or under the CIO, sometimes under the CTO. Now, what has changed is that now people don't have the patience to wait two weeks for a dashboard. now they want to run an agent right now and suddenly that agent doesn't have access to that database that no one has looked at in years, right? So now data teams are under a lot of pressure to extract value to the data at agentic speed and they realize that the systems that they have in place today may not cut it for the reasons that we already mentioned either governance, scalability at the infrastructure level saying how many quiddies per second can you actually hold or simply access controls? So it sounds like the role of the data team has shifted a lot, and I think there's been a recurring conversation there in a lot of the data frame dev sales this year, is that if agents can do your analytics for you, what does the data team do now? And it sounds like the answer is making sure that all the data is available

with the right permissions for the rest of the organization. Does that sound about right? That's the main thrust of the new data team? That is indeed the number one job and responsibility, I think. The whole software world has changed completely. It's not that it's changing, it has changed completely. I often mention one of the cultural shocks that I had probably last year around this time. I came to the office one day and, you know, sometimes you go to the engineering area And even though we have a lot of remote, we're remote first and so on, but we have a decent amount of people showing up in the office. And typically you go to the engineering area and we engineers wear headphones in the zone, focus on our thing. But that day I came in and it was super noisy. It was a lot of chatter. I'm like, what is going on? Am I in the sales area or what's happening? And I realized that everyone was talking to their own cloud or cursor or whichever weapon of choice they had, right?

So our engineers are now talking more than typing at all. And that makes sense, right? And then you have your different environments. We have a bunch of people with the lights and the things when the agents are done, working, ready for input and so on. The same happens with data teams, right? writing an ETL. If you're still doing it today, as in September 2026, that doesn't make a lot of sense in terms of value added, right? If you're still trying to debug a DAG pipeline that is broken manually, same thing. If you're trying to optimize a query, sure, there's a lot of artisan work that we love how to do. And many of us, myself included, we came into building software because it was a craft that we loved how to do that. So it has been a huge shift that essentially changed the ethos of what it means

to be a data engineer or a software developer, right, or a software engineer. But the reality is that the vast majority of the artifacts that we are creating now are going to be ephemeral. we would work for a few days putting together pipelines and create an extremely little application for someone to consume and then make decisions based on that that would be used once and killed the example of the dashboard that I was put in before a CEO comes and asks hey I need to know for the board meeting next week the revenue forecast for the APAC region on this specific category of our products and then you scramble and get everyone together and trying to find a way. Oh, we don't have the data from APAC because they used the different ERP on the acquisitions that we did two years ago. How do we do this? All right, let me just build a really dirty script that will hydrate from those oracles that they have in there that are super old and legacy.

And then you go and do it because good people in that achieving things that are technical challenges, that's what we look to solve technical challenges. and then we build a dashboard and the CEO is happy and then presents it and so on and consumes it on Monday consumes it on Tuesday and that's it that dashboard goes into the graveyard of dashboards that get updated every Monday at 9 a.m consuming compute but that no one cares about anymore together with the other 10,000 dashboards that the company has created many oftentimes with data that is rotten columns that don't exist anymore, you name it. This is our life as data engineers. So what if the CEO can, by herself, go and say, I want this, and then the agents that are already set in place with the right permissions, with the right systems, with the right data access, with the right networking connectivity to those legacy Oracle servers in that region,

in that place that came from an acquisition that no one cared to integrate properly, five years ago, and then they put that together, they put the dashboard together, they give it to the CEO, he's happy, and then all that gets killed and doesn't consume any compute because it's not going to be used any longer. And hey, if it needs to be used again, it took 30 minutes and $50 of compute to put all that together. That's definitely cheaper than four data engineers trying to scramble in and getting that data, right? So it is a fundamental shift on us humans. How do we change the place where we create value for the company, right? And that's making sure that the agents can do that at scale. That suddenly a human could make X amount of queries on their databases, and now an agent can do 100 times that.

So how do you build an infrastructure that can support orders of magnitude the employees that you have? So what I mean by this, imagine you have a company with 10,000 people. And these 10,000 people are essentially your stakeholders for your data. You create your dashboard, your databases, your query system, so they can go and use your data there. And then you have scaled and built your system to support 10,000 people. That's different than supporting 100,000 people. or a million people. But the reality is that if you start to give agents to all your employees, you don't have 10,000 people in your company no more. You have a million people in your company that go deeper, faster, and can collate and coalesce more tables than ever before and can identify things and other stuff and patterns that no human has done before. So if you have, say, 1,000 nodes for running a data lake before,

start procuring new hardware. Okay. Yeah, I mean, it's really fascinating that the dashboard has been like a staple of analytics for decades now and they're becoming increasingly ephemeral in many cases. So for people who are wanting to get into analytics or data science, like what do you need to learn about then given that the job has just changed so much in the last few years? Honestly, I think about that every day, right? And even for a long time, my kids, I have two daughters. They're 10 and 12. So like many fathers in our industry, I wanted them to learn coding, Python, and basics of software engineering and, you know, tailoring them towards data and so on. And now I'm not sure anymore. I think it's all about solving problems, being very close to the domain. This is the most important time where the domain knowledge with the industry, knowledge is so fundamental. Because your pattern recognition on, hey, I found a bug on the pipeline,

that is not that relevant anymore, right? Now is, how can I go deeper and further? So we are creating a bipolar system where, on one hand, we need very deep experts at the engineering level. Someone builds the LLM, someone builds the orchestration systems. that's very hardcore knowledge and that's what we do at data companies right at Cloudera we have people who are essentially building the database systems with super deep knowledge on distributed systems federated data and so on that needs to exist so that's going to be one trade and then another one is more product engineers and I know you guys talked about this in the podcast many times before product engineers that have both the engineering skills but the product and the domain vision of what is what we need to build with this and including the orchestration using the systems And this chasm in the middle is where it slowly disappears, right?

Either you become a super hardcore engineer, whether it's on data, on distributed systems, on infrastructure, networking, super important, or you become the best product engineer ever. And what do you need as a product engineer based? More than ever, you need to know what works and what makes people tick. That's a great and big change, right? Especially for us hardcore engineers. Absolutely. This case requires understanding. There's a lot of psychology in there. There's a behavioral science and things like that. And a bit more social. You've got to speak to other people, which is traditionally not. not a thing that goes along with engineering. So, yeah, it's interesting that there is this divergence there. So, since you mentioned at the start how a lot of people, like if you're a sysadmin, you've never seen a rack before, has the world of the infrastructure changed then? Do you need new skills there? Are there new job opportunities in that region?

There are new opportunities indeed, because in some cases it's going back to the basics. As I was saying, if you're in the middle, You can either go the hardcore side or go on the product side. But we see more and more in our report, actually, 66% out of 1,500 respondents, CIOs, CTOs, CTOs, are saying that they're moving workloads into their data centers. And make no mistake, that doesn't mean, hey, we're moving back to the data center. No. It means we need to have a complete hybrid and holistic approach so that we can go and leverage the private cloud in exactly the same way as we leverage the cloud. We do not want to get rid of the benefits of the cloud that are plenty. I want to make sure that I have a cloud-like UI or console but to manage my own private data center, right? And if I go and I need to use... I'm sovereign cloud provided in Europe. I come from Europe. I'm Spanish.

And sovereign clouds are all over the place at the moment. most enterprises today in Europe need to use and to have a very clear strategy on sovereignty. Essentially answering the question, what would happen if we cannot use any U.S. provider anymore? Sure, you can have many, many political arguments about that, but forgetting about the politics, if that is a constraint from a business continuity perspective, a data team needs to make sure that with one click or one API call all their workloads move from hyperscaler A to public or sovereign cloud provider B or to data center that you have in Frankfurt or in Paris or in Madrid with one click without a five-year migration process back into the data center but what every single enterprise in Europe is saying is I do need to have that plan in hand,

but I definitely want to keep using AWS. I want to use spot instances when required. I want to have my reserve instances. I want to use, you know, Redshift when is the right tool or BigQuery because those are phenomenal tools and we want to be able to use them all. So at the end of the day, that orchestration would create a new trade, right? And our bet at Cloudera is trying to be the company that helps enterprises do that very thing. Think about your data platform holistically, a single pane of glass, if you will, regardless of where your staff is. Data tables on S3, OSU on storage on the data center, Rastafest, Vast data, GPUs on your data center with your NVIDIA H200 so that you can run inference, but when you run out of use, you go and push them, push compute on OpenAI or Azure or Bedrock. that way of seeing holistically your data state. But then your orchestration platform not only knows what data you have

and who can access it, but also where that data can run. So imagine I am a large bank in Europe. I have a certain type of data that needs to stay in my country, say in France. So I use my data center with my GPUs to run a lens on that data. And that never leaves my environment. so an agent knows that and runs the pipelines or the agent, the LLM on that place all is good there's a person next to me who's running a credit risk model based on public news that's public information it's news, it's an experiment it's perfectly fine to use the latest Oboos under bedrock and run that on the cheapest spotting sensors on AWS that today at this hour happened to be in U.S. West 2. So the system needs to be able to orchestrate all those things and say, yeah, this can go there so that we leverage our capacity.

This is a lot of phenops as well. But this specific data set, this specific use case must remain within this sandbox environment. How do you create such type of orchestration system in the data center so that you don't have to think in terms of bare metal. And that's what some of the software cloud providers are doing. The new clouds are doing as well, creating that abstraction layer. That's what we, Cloudera, do. If it is your own data center, you definitely need skills to go and put the servers in the racks and do the networking. Many of the data centers out there can do that for you. And also, the industry has changed a lot. Even when we think about Kubernetes. five years ago it was very different from today by the whole collection of open stack to to value from bare metal to you running a smart job it could take you weeks or months

i i oftentimes say this example internally with our teams for you three four years ago You need to have an open stack team. You need to have a whole virtualization team. And from bare metal to a spar job, it could take you two months and 15 to 20 people. The last customer that we had with a similar environment, we went from bare metal to running a Mistral model in their data center with GPU within an hour. And that is obviously thanks to the technology that we have today that the community builds, right? That's what you get by having open source blood and then creating enterprise solutions around that. The maturity of Kubernetes, the maturity of all the open stack solutions and so on, and the maturity of the data platforms as well. But the world is indeed changing. Absolutely. It sounds like data migration then is going to be like the hot new career perhaps, or at least the hot new technology to build on.

All right. So just to wrap up, I know earlier this summer you gave a talk to the UN about AI for humanity. Do you want to give us a quick summary of what you told the UN delegates? A hundred percent. That was a high on my career, right? Being able to be on that stage talking about this. And it was phenomenal because just three days before my keynote, that was when the Fable and Mythos models became unavailable for non-US nationals or companies. So my job went way easier. That week I had countless calls from customers and friends from all over the world, not just in Europe, saying, okay, this is the first time in a long time that we realize we are at the mercy of one single government. And it is challenging because on one hand, the world wants to use these technologies,

wants to leverage Anthropic and AWS and OpenAI. We definitely want to keep living in a globalized economy. But that was kind of a wake-up call to say, well, what if? And we've seen these problems happening left and right, right? We've seen this AWS region in AWS blowing up completely with no data recovery capabilities. And that's no one's, quote-unquote, no one's fault is the current geopolitical situation that we have in the world today that's extremely complex. So, what I wanted to share with the delegates was AI is becoming or has become critical infrastructure, as important as Internet, water, and electricity. not only at the company level so that enterprises can keep competitive advantage but it is it is a sovereign conversation so that countries don't get left behind and can participate in the new

economy that we're all building together and you can think about slowing down development stopping the research and so on like the conversation that we see before that for me is more red herring conversation. We can get the main players in the US agreeing to slow down and then China saying, that's phenomenal. Keep doing that. I like that. Please stop. I won't. The whole conversation is, what if? What is the risk mitigation, but at the same time, we keep collaborating and keep working. How do we make sure that AI works for everyone in a fair way? That it is not central to a specific ideology, a specific country, a specific politic mindset, but essentially that can serve citizens. And that was my main message, right? You want to implement AI in your tax agency, in your hospitals, in your police departments, because it's good. It's a great tool like the card was a great tool,

but you want to make sure you don't kill many people or anybody at all as you do that, right? And it feels like when we were talking about machine learning, there was a lot of development about that and ethics and how to implement safety systems inside machine learning and so on by making many mistakes along the way. but in many cases with AI it felt like we started from scratch instead of leveraging all the work that many companies have done in data and machine learning and apply them into the system and it was a call for the delegates to say first of all, every government needs to have a very clear AI strategy that includes but is not limited to sovereign and private AI components Every company, every country and company needs to understand how AI impacts them in their own language, their own culture and so on, and how it is going to be applied to their citizens.

So one of the examples I used when I started the pitch was, hey, imagine in your company, you can only hire people from four different companies because of the regulations or whatever, only from those four places. And these four companies, they come from the same country, from the same culture and mindset, and they can overnight change how much they charge you. They can deny service. They can change the terms of service overnight. Would you do that? And obviously, everyone said, of course not. Why would we do that? But essentially, that is what we are doing today. So the whole message was, first of all, Europe especially, but also South America and Central America, Middle East, Asia Pacific, China is obviously very well advanced on that. We need to have our own story and we need to have our own control of our own AI. And that doesn't mean going our own way and building our own things. It's still collaborating and working together,

but making sure that our AIs as critical infrastructure are at the service of humans and not just a handful of humans. That's a very good point. I mean, having introducing dependency on a few American companies, that's going to be a big risk for any business, for any country, and AI really does need to be for absolutely everyone. And I talk from the perspective of Cladera, which is an American company in a few ways. Absolutely. All right. Super. Thank you so much for your time, Sergio. Lovely. Thank you very much, Ritu. It has been great being here. you