サンドボックスから2万5,000ユーザーへ：Bayerはいかに AI をスケールさせたか

This podcast is presented by NXAI, your partner for Time Series Foundation Models and Physical AI. Hello everybody and welcome to a new episode of our Industrial AI Podcast. My name is Ruhl Weber and my guest today is Jan Tönesen from Bayer. Jan, welcome to the podcast. Thank you, Robert. Nice being here. Yeah, you are a listener of our podcast and you wrote us an email and anyone who writes us runs the risk of being invited to our show. Before we start talking about buyer AI and scaling, please introduce yourself briefly to the listeners. Okay, yeah. Happy to do so, Robert, and thank you for the invite. Yeah, you wrote us an email, so you are now in. Yeah, nice opportunity. Yeah, Jan Tennyson. I'm now over 18 years with Bayer in various roles over the last years.

I very initially did some C-level IT support, then went actually more into consulting for process transformation, larger transformational projects. I was with Bayer five years in China to roll out an operating model and global IT system. And the last decade, I mainly focused on the question of, like, how do you actually transform processes and businesses and functions with intelligent automation and now with AI? So, until recently, I had an IT delivery platform in Bayard's global IT organization delivering IT pieces from our largest Gen.E.I. and now Gen.Tick internal product called MyGenesis, 25,000 users, 60,000 assistants. We also just recently delivered with the team an AI gateway to consume all your LMM and MCP needs. And we have an agent hub to register all your agents across the company. Wow, wow, wow, wow. Okay. But I also was busy with things like in a team effort, setting the frame for responsible AI at Bayer.

And we also built up a community of now 19,000 Bayer enthusiasts across the company that come together every month. Actually, this morning was our September call. There's up to a thousand people in the call and they're all sharing what they're doing, the latest and greatest. Today we had internal products. We had colleagues that talked about CoPilot 365 and what's coming new. We had foundation colleagues talking about a platform they're building for agent delivery. So that's me. So CoPilot is still the hottest shit at your company? It's part of it, I would say. We have a toolbox. And, yeah, since September to close us out, my role is to accelerate the actual transformation. So it's not a question about what we need to build or buy, but really, like, how do you actually transform now the business process? And maybe also how we bring that together with the larger things going on at Bayer, like our new operating model. Can you explain a little bit how to scale AI in a company like Bayer?

So it's a huge company, right? So pharmaceutical industry, heavy processes, very strict processes. How difficult, how easy is it to scale AI in such a setting? Yeah, good question, Robert. And I think there are probably two sides to it. The one is, does the technology scale? And that's probably in many cases the easier question to answer. If you set out to roll out Copilot 365 or, you know, any off-the-shelf product, I think it's fair to assume that in most cases that scales rather easily in terms of like an increasing user number. Our internal products, there were obviously a few learnings. We started very small with a few hundred users at the beginning of 24. The more noise we made and the more good, you know, like feedback we got and people talking about it, the faster the scaling was. So there were a few moments, you know, where we had to refocus on stability and performance, but that can be managed with an experienced IT team.

The other question, obviously, is, like, how do you bring people to use it? Because the tool is only as good as, you know, the usage at the end and really what people do with it. So we spend a lot of time, for example, with our internal genesis product to tell people about what it can do, also what it cannot do, right? So people can easily judge on whether it helps in their daily work. We have training sessions. We did prompt-athons. And then through the community, we constantly, like, provide new learnings, new kind of like knowledge nuggets to people to really understand what is possible with the next iteration. So more and more people are able to actually adopt that technology. From my point of view, so having a big company like Bayer, they want to try out. You have people that want to do fancy stuff with Gen.A.I., LLM technology. You need guardrails for this whole thing, or am I wrong? No, absolutely. You need guardrails. That's why we also started with setting the frame. I think the interesting thing is once you get a bit into the details,

and, I mean, a good trigger, for example, is for most of us the EU AI Act, right? There are certain things that need to be done. You figure out you need a register of things that are going on. For example, you need to understand who builds an agent, right? who puts an agent in production, what is the agent doing, and so on. But you also need to understand, like, what can you use, which L&Ms are okay to use, which tools are okay to use, and so on. So while setting out this frame, which, like, sets the guardrails, as I said, for everybody on a very high level first, you also figure out that you can actually support people on that journey because the more you talk about it, the more questions also come up. Is this safe to do? Is this safe to use? so at the end of the day your guardrails don't necessarily mean slowing down but rather passing the way you know that is easy for people to go and then they can run very fast this you know their new solution so what is your experience or what is your advice building guardrails before or after the first people came to ask questions

so how do you organize that so with JTBT came on the market the people tried out in their offices and you said, oh, shit, we need to build guardrails now. What is the idea behind that? Yeah, I think that in all those words, I think that went a bit in parallel. So, you know, end of 23 and beginning of 24, JTBC came, a lot of other topics, mid-journey for pictures and, you know, all these tools. And we saw, like, a very strong, like, uptake in, like, bring all these tools in, right? not always can you expect that people run through the necessary processes making sure everything is safe and sound so we very early on gave guidance to people what it means to use such a tool right and what are the minimum requirements that we as Bayer want to see when people use tools with actual Bayer data right and Bayer use and so on if you just go out and you want to try it with some fake use case mock data or whatever that's one thing but once you put it

into a company context then you need to comply with certain regulations. And what I always say, they didn't really change fundamentally between AI and agentic and pre-agentic because also with other applications built in the past, you need to think about the data, the access rights, the controls and everything. But I think it's kind of like good IT practices that still apply. That's interesting. So you mentioned the EU AI Act, and it's often seen like a hindrance, But you said it's a catalyst more, the AI. Why is it a catalyst for you? Yeah, because it made us think about what we need to put in place so that others don't have to replicate it again and again. And then with such an enterprise mindset, you figure out, okay, like a solution like the agent hub, where people can easily go, we already have thought about what is an agent, what is an agent card, how do you mostly automatically

register and so on they don't have to do that step anymore then this whole topic of how do you assess your application once you think it's through a lot of these steps can be mostly done automatically you have a questionnaire some people bring their solution idea and you can actually have a very automated interactive discussion about the solution and out you get kind of like an initial scoring if it's green go ahead first and then once you get more into the details you come back. If it's red because maybe a use case sounds like something forbidden according to the UEI Act, you can very early on have that discussion with experts that help you shape your use case in a way that you stick and harvest the benefits but you don run the risk of doing something that is non Can you please explain me a little bit the process Imagine I an employee at your company and I want to do something with the agent Let's stay in the office environment first and I have a nice idea to do something.

What do I need to do in the next step? Okay, let's stay an HR employee or procurement employee in our company and you want to build for your team an agent that maybe helps the team to answer questions or produce some dashboards every month, etc. I think the first step would be to think about what is the best tech stack to use. I would say if it is mostly with data that you have in your SharePoints, emails, etc., I think there's a good benefit in using Copilot Studio or Agent Builder because then you can harvest this tight integration. If it is more about building something from scratch, maybe with your own data sources, then our MyGenesis product is also a good platform to do this. What is that? Can you explain a little bit? Yeah, MyGenesis is that product we started building end of 23 that now actually offers you a platform where you can go not just to prompt, but you can actually build authentic keyset assets that can be reused and shared across a company. So, for example, a little widget or an agent is a certain master prompt and data context or a small workflow that invokes various agents in a row.

And all of it can be programmatically accessed from outside of the platform. So you can also embed it into something bigger. And that's where we had over the last two years these 25,000 colleagues, I would say, in the months, around 20,000, 22,000 or so that do something actively there. Yeah, and that gives you a safe environment, you know, to do your things, to build your things, and you get a lot of support. So it's a sandbox approach? Yeah, it initially started as a sandbox, you're right, because we wanted to try out things. But soon after the sandbox idea, we figured out that a lot of people across the company, maybe pharma R&D or consumer health, marketing and sales, they actually use it in their daily work. So it has outgrown that sandbox idea. And now it's part of our, let's call it, low-code, no-code toolbox offerings next to Copilot 365 with Copilot Studio or Power Platform. And then we also have some other platforms, for example, like NNN, which do pieces of this.

But they're all of their special area of strength. Don't get me wrong. It's an internal tool you develop with certain guardrails, with certain security issues. Or what was the idea behind that? because there are a lot of tools on the market that are maybe comparable to your internal tool. Yeah, so I think there were two initial triggers. One was really this topic of security. When the first JGPT came to the market, it wasn't that JGPT enterprise that you can get today, right? But everybody wasn't sure what happens to the data. Is it used for training? Where does it flow? Where is it stored? And so on. So my genesis, even in the initial version, consumed the LMM just from our hyperscaler accounts, where we turned off all the trainings, all the checks that you sometimes have. So we were relatively sure that this is good for our data handling. And on the other side, it still turned out to be quite cost-efficient compared to buying licenses for like 100,000 employees across the company.

Now, with Copilot, we still decided it makes sense to bring Copilot 365 because we are the last mile, I would say, is Microsoft. So we are a Microsoft company when it comes to your workspace. And of the tight integration, you get a big benefit if you have Copilot activated. Yeah, but that was the initial starting point. Coming back to our use case. So I checked now I want to do an agent in my environment. I checked Copilot. I checked your internal platform. what else do I need to do? Yeah, so I mean as long as you stay within a certain, I would say limit or freedom in the frame that we give you Token limits or what? No, I mean in that sense really like if it is an agent mainly for you or for a team and you don't use any restricted or secret data and so on we don't actually ask much more of you. The registration between this environment happens automatically so your agent is automatically registered at our agent hub, not yet for everything in Able.net, but where we're heading, right?

So there's transparency automatically created and through the policies that we impact in these environments, whether it's Microsoft or N8N, you actually can't do any wrong things, right? On top of that, we provide you with that guideline I talked about out of the responsibility eye context, right, where it clearly says what use case you can do because that's a hard piece to check, right? You could imagine only taking an exit sheet or PDFs of resumes, uploading them to an LMM and say, pick one is forbidden, because you're not allowed to make automatic decisions, for example, on hirings. So that's where the educational piece comes in. You mentioned the agent hub, right? How many agents are running in the moment in the hub? So the hub is more like a telephone book, right? They're all, they sit on different platforms. So we have all the hyperscalers, GCP, Azure, AWS, the Power Automate Environment, MyGenesis. So they all run some agents, but the idea on the agent hub is to create transparency, right? So you register your agent, visit agent card, which tells everybody what the agent is doing, who is the owner, where it's actually running, and so on.

And that helps. But because otherwise it becomes a jungle, right? Absolutely. And what we're now combining with this is, so let's say, our governance processes, right? Because when you build whatever IT solution they are, you need to run your idea and ultimately also your solutions through IT security, data privacy, trade regulation checks, and so on. And depending on your geography, there might be local regulations as well. And if you would now build also a more sophisticated agent solution, then you will also need to bring your agent solutions through this process. And that's where we now work on integrating these. So ideally, that all goes hand in hand and you get like one pair or one round of questions for your use case and your solution. And hopefully, you know, if it's not too complex and not too challenging in terms of regulations, you can easily go ahead. Do you work only with OpenAI models or Gemini or Claude or what kind of models do you use? Yeah, so across the company, I would say there's nearly everything.

Okay, so QAN models too. CRAN models as well, absolutely, and DeepSeek, and they have a wide variety on purpose. I would say the most used ones across the company are probably still from OpenAI and Anthropic, but then you see, I would say, you know, like open weight models picking up, because they can be run quite cost-efficiently, and they become actually quite powerful in certain tasks, right when it comes, for example, to coding. And just today, I had one of our developers saying he uses one of the FrontShare models to plan his tasks, and then he uses one of the open rate models, self-hosted, you know, to do a cost-efficient execution. That's interesting. That's interesting. Can you share maybe, as you mentioned, the developer, can you share maybe three use cases you saw in the last four weeks where you said, wow, this was impressive, this was interesting to see? Yeah, so I would say one use case really is around supporting research.

So you can imagine if you are looking for a new molecule, that takes a lot of time, a lot of research, literature, and so on. And I think the numbers that were given were like, usually you search in sources like PubMed, where you have millions or gazillions of documents, right? You filter down and then you still end up with probably tens or even hundreds of thousands of documents, which might be relevant. to what you're looking for, right? And that usually took the colleagues weeks to go through this and limit it down maybe to like a dozen of really relevant research documents, research papers. And they can do this now in a matter of hours or a few days because that's where LMMs are specifically good, right? Unstructured data, figuring out and measuring it to your search terms in terms of the prompt. So this was the first one. Yeah, that was the first one. Then I see a lot in terms of communication and slash marketing and sales. I mean, we had a lot of agency business or business with agencies.

So, put it that way. You still have agents, but these are different agents now to agencies. Exactly, right? So agencies really in the sense of external agencies that if you marketing and sales stuff or fine your messaging and so on and so on Yeah newsletter stuff etc Exactly right And obviously they cost right For a good reason because they very expensive people But if you take away the first 80% and create your messaging, your marketing sales material and so on, even if it's not perfect, it saves a lot of time in terms of discussion and iterations and so on. It actually can save you hard dollars. going forward. And then maybe one internal use case. We have a new product just launched a few days ago. It's called Friday. And Friday is an agentic system for every employee. And it basically brings together two sides of the internal house. The knowledge because we have this saying if they are new or bear knows it would be much smarter, much faster.

And the other piece is there are so many things that you need to do in the course of your day, different systems, different approvals, different requests, and so on. And we kind of unify this on Friday. So it's an agentic system that we give everybody in terms of either online access, but you also have a standalone application now for the laptops. And it basically gives you kind of access to this universe of knowledge and backend systems. And it makes it very easy for you to say, okay, maybe you start this morning with, I need some information about my leave policy and then you want to put some leave days in to I need to actually partner with this company. So how does a potential standard contract look like to then ultimately ordering maybe a service from this company, right? And not everything is there yet, but we rolled it out to already 1,000 users and get very good feedback. Oh, that's a very interesting, very nice approach. I want to come back to our token discussion because it's a huge discussion in every company. What about Bayer?

Is it also a huge discussion when it comes to tokens? Who manages the tokens? Do you have KPIs for token use? What is your strategy on tokens? Yeah, you're right. Token consumption and with that also cost for tokens were high on the rise the last month. And we took a few measures. So, I mean, one reason for everybody to bring in Copilot 65 was also that it's rather a seed-based license and not a consumption-based license, right? So we prefer people for, like, easy things, find your word document, draft your email, boss, et cetera. Just use this, you know, instead of using any other solution which is consumption-based. So I think that keeps a bit the lid on all of this. We were initially a very early adopter for GitHub Copilot. We now are moving more to Cursor. For us, it seems more cost efficiency. And the idea is you consume your LMM, your tokens, through our AI gateway.

And that would give us then full transparency on more or less the whole token consumption of the company. And it enables a few extra benefits, right? So it's not only the transparency and making sure that, you know, we know which models are used and so on. But we can also, for example, with some of the OpenAI models, we have kind of like a pre-bought, pre-paid models, right, where we get better SLAs, better performance. But we also know, as long as you stay within like a certain limit, that is all covered, right? And for the main models used in most of the applications, we can do this, not for all, but, you know, like for the most popular ones. But we can also look into now caching some of the requests very early. We're doing some experiments on this. And the last piece we're looking into is smaller language models, specifically if you build them into your solutions, because we also still see, I would say, a trend that everybody tries to get the biggest and brightest model. And then we ask, like, what is it?

Is it an invoice or is it, you know? This is coding, right? And obviously you don't need to ask Astra for this. You can totally use a much smaller, much more cost-efficient model or maybe even train or fine-tune a model with the data set that you have. And good example for cybersecurity, where we have a lot of data on cybersecurity incidents and we can train a model. Or service management, where we have millions of tickets from the last years and we can easily train a small language model to decide what kind of ticket it is. so that's interesting so token is a topic self-hosted models become more attractive for your company right absolutely and is there a certain discussion in your company how this company spends so much tokens we are more efficient than them these are the KPIs for tokens I think this discussion will follow in the next coming months because you want to compare human work to AI work

and you need some KPIs at the end Or am I wrong? No, that's right. The interesting piece is how it developed over the last months, right? It came from, like, everybody tries to maximize their token usage. We thought that was good. Now everybody tries to minimize their token usage, be more efficient about it. So that's the new gold standard. We are actually taking token consumption when it comes to building solutions as part of our investments. All right. So we're now actually saying if you build a solution, and formerly you would have maybe used, let's say, 20 engineers. Now you do maybe the same work with 10 or less engineers, but we are adding the cost for the tokens. So we're keeping track about the investments, actually also counted as a capital expenditure in our case. And obviously that then gives you the full picture on your use case. And, you know, ultimately we want to see that a use case in a certain period of time returns on your investment. When do you see the first results to this strategy? Because that's a lot of documentation stuff, right?

So when will you see the first results, tokens compared to human engineers? I think we already have now. I mean, we have a few, quite a few platforms that are doing engineering, also powered by our digital hubs, which are locations, you could say, where we build up really engineering talent and capacity. And I would say across the board, all our engineers are using AI-assisted coding from the point where they start doing the Figma blueprints to the actual development and later also the testing. And what I can see from some of the products that were on my platform until recently, that saves us a lot of excellent spend already because we can achieve more with less. Now, I think we're still discussing a bit on the question, what is the factor, right? Is it a factor of five or ten? What is your feeling? My feeling is it depends a bit on the nature of what you want to do, right? I would say testing and so on, you can mainly hand it over,

right? If you prompt it in the right way and you give it the right instructions and so on, I think testing can be done mostly fully automatically. When it comes to spinning your head around your solution design, your architecture, iterating maybe also on different ideas and different designs, it's probably rather on that 3, 4, 5x because it just multiplies or magnifies the impact of our experts. And in the middle, depending on the complexity of the task, it's probably somewhere between 5 to 10. Okay. I want to come back to our agent discussion at the beginning because You mentioned you have guardrails for using your LLMs or the ChatGPTs and the Gemini, etc. What about guardrails, building guardrails for agents? Because we hear this terrifying stories, sandbox attacks by agents. Do you also build guardrails for agents? Yeah, absolutely. Interesting what I can share because it's security relevant.

But let's say we have an idea about how an observability and governance there looks like, right? And there are different pieces. So initially I had the feeling everybody was thinking about like this is one tool that does it all. But I think the truth would be there will be different tools for different purposes, right? So we're going to look at transparency. Do we know what agents are there, what they're doing, who owns them and so on? That's a bit the agent hub idea. Do we know what they consume? Then we have, I mean, a longstanding experience, decades with data management. So we can track who has access, which agent wants to access and so on. And then there are probably some newer pieces when it really comes to, like, where's the runtime and how you, you know, like, make sure it's in the sandbox or not, depending on what the agent is supposed to do. And I think there was also looking into how to improve this now with partners, like Microsoft is one of our strategic partners. but also with others and also our IT security colleagues use some specialized tools you know to guard against pump injections and personal applicable data leakage and so on

So I think it's a very broad topic if you want to make it safe end to end. Who's at the end in the lead, Jan? Yeah, that's a good question. From top to the bottom. So CTO, who's in the lead for this initiative by your company? So our CIO, B.J. Saga, is passing away, having regular exchanges with the supervisory board, the board of management, the business unit leads, definitely visionary. Then we have different platforms, or let's say layers in our IT organization, thinking about the different things. So there's cybersecurity, there's our head of engineering, there's IT Foundation, which looks at infrastructure and hyperscalers and so on. And I think the key now is, or the challenge and the key, it all needs to come together, right? You have the right setup on the hyperscalers, you need to have the right data management, you need to have the right access management and so on.

So I think that's the IT side. But I think ultimately what we also want to have is that if you put something in production, right, I mean, the idea of everything we do in IT is creating business value. So there needs to be ownership on the business side, right? So if you ask me, let's take some IT agents aside. Whatever agenting solution we build, it needs to be owned by somebody on the business side who also understands, you know, the implications on what data we let it lose, right, how it's embedded in the process who should have access to it and it's really from a business design side I want to switch a little bit to our discussion because we now talked a lot about the office environment right and now the next wave is coming with physical AI and we will we discuss agents near to the shop floor that's framed like this will this be more difficult or can you build on your experience? Or do you see the wave coming that also the guys on the shop floor are saying,

oh, Jan, we need something when it comes to 1, 2, 3, 4, 5, or we want to solve something with AI in this case or on the convoyer belt or whatever. So what is coming in the next wave? Yeah, I have to admit I'm not the expert on our shop floors because there are quite a few. different ones. But what I already have seen is the connection of data on the shop floor. We have a very cool use case where you can actually talk to your production line. I think that's just the first step. I could imagine that if you want to really automate things in terms of when you talked about this robot, I thought like in my head I saw some robots switching product lines or cleaning or whatever. I think that's probably going to take some time at the end talking about often medical products you know it all needs to be really validated and secured so we know what the robot is doing, it's traceable

it's auditable and so on so I could imagine it's an interesting next step and pretty sure the colleagues are working on something, also the lab of your future for example but as hardware is involved it actually needs to change more than just on the software side and that probably takes it exactly and I think The office world and the shop floor world are coming closer, right? So you use now chat GPT or LLMs with MCP connection to a database on the shop floor. Ask a time series model, can you share the lightest production information? As you mentioned, talk to the factory, talk to the shop floor. So the whole thing is getting more complicated, more agents, bigger environment. Now we add shop floor, we add production lines, we add logistics. So it's getting bigger and bigger, and that means you need really a good structure to start from, a good basis to start from, that you already have the guardrails to run this and to scale it, really, as you mentioned.

Absolutely. That's why we early on started with this topic of Responsibility AI. We're now also re-looking at our data again. And so we have a very interesting team working on what we call a unified consumption layer. So how do you actually expose your data assets across a company? You know, in the past, we were very restrictive in who even knows what data assets exist. Now, we turned that around a while ago into if it's not necessary to restrict it, then you have access. But that doesn't mean, you know, your agent understands the data. So we're also talking about semantics, ontology, and I think that's the next barrier to take. So data becomes accessible for all these solutions. And then obviously we also need to think about what is actually an agentic operating model, right? Who runs this? Who observes this? What if it breaks? Who fixes it? Who is responsible? Interesting. At the end, you work now for a big company, huge corporate infrastructure.

Let's imagine you work for a machine building company, let's say 1,000, 2,000 employees. What would you recommend how to start something like the agent hub or giving the company structure when it comes to AI? What are your three takeaways, what you learned from your company? Okay. I mean, first of all, I would say gather your coalition of willing colleagues, right? Because you will need, you know, to partner with some of his colleagues. And it's good to have, like, a small coalition. You know, they can pull things off together. I think that's one thing. And you can build it into a community later. So it's a good starting point. Second thing I would say, choose your use cases, your first battle carefully, right? Because it's easy to get caught in, you know, too many pieces and a lot of potentially compliance discussion whatsoever. error. So choose something that convinces because it's not too easy, right? But also don't

choose the most difficult one because then you potentially get that. Do you have an idea? I mean, I would pick something that is impactful but not critical. So don't maybe go to a meeting with product supply or supply chain. Yeah, invoices would be great, right? Invoices, relationships, IT is also, you know, a lot of use cases they can support with identifying your service ticket support. And the last piece is, I think whether you build or buy, whether you use open source or proprietary stuff and so on, it's difficult to copy it because it really depends on quite a few factors. Do you have actually the ability to build? Do you have the time to build? Do you have the appetite to run it yourself later and so on? So I would say from that perspective, think about kind of like a decision tree first. so that when you execute you can just go through and I think it's totally okay to go to something off the shelf if it fulfills your needs and the pricing point is okay

Great, Jan, last question what is on your technical agenda in the next 2-3 months? Yeah, so we want to continue shaping the build environment so we have Agent Hub, we have AI Gateway now we just started with elements. Now we are adding NTPs and we're going to build further out this governance, transparency, observability layer. So I think that's definitely on the agenda. Then we also, of course, want to go stronger on this data piece, finalize a few elements there. And I would say the last piece is probably a bit around really focusing on some rather impactful use cases that we have identified not only in our back office, but also really in R&D and product supply so that we can showcase a bit across the entire enterprise that this is possible. It's all about KPIs, right? Absolutely. I mean, and that's maybe a good message, yeah? I think what Stuka said, you can't steer what you can't measure,

so you need to measure first, right? So think about how you actually measure your KPI before you start. Yeah, exactly. Jan, it was a pleasure. we keep our fingers crossed for your technical agenda and we are really proud to have you as a listener and now as a contributor thank you very much and greetings to Leverkusen thank you Robert what a pleasure bye bye bye bye