AIエージェント2万体を計画していますか?Capital Oneからの警告|Abhijit Bose

Are You Planning 20,000 AI Agents? Capital One Has a Warning | Abhijit Bose

Eye On A.I.38分
AIエージェント2万体を計画していますか?Capital Oneからの警告|Abhijit Bose

AIエージェント2万体を計画していますか?Capital Oneからの警告|Abhijit Bose

Eye On A.I.

0:0038:37

要約

Capital OneのAI/MLプラットフォーム部門を率いるAbhijit Boseが、10年以上前のクラウド投資と中央集権的なプラットフォーム戦略がAI拡大の土台になったと語る。損失予測や不正検知の高速化、ディーラー向けChat Concierge、2万人超のコールセンター支援RAGシステム、ガバナンスの枠組みMacauなどの事例を紹介。エージェントが5つなら不要でも、2万規模ではプラットフォームが不可欠だと警告している。

この回のポイント

  • ●Capital Oneは自社を「銀行業をするテック企業」と位置づけ、約10〜15年前のクラウド移行(主にAWS)と全社標準プラットフォーム化がAI拡大の基盤になったと述べている。
  • ●プラットフォームは大規模なCPU/GPUとデータの調達、エージェントやMLパイプラインのオーケストレーション、開発・モデル作成、手作業を減らすデプロイ層で構成される。
  • ●損失予測のモデリングは従来数週間かかっていたが、新基盤の分散並列計算でエンド・ツー・エンド約1週間になり、不正検知モデルもエンジニアを介さず素早く本番更新できるという。
  • ●オープンソース、外部ソリューション、自社開発を柔軟に組み合わせる制御プレーンを採用し、不正検知など中核領域は独自データ・独自モデルを使う方針。
  • ●顧客向けエージェントとして、ディーラーサイトで24時間稼働するChat Conciergeと、2万人超のコールセンター担当者を支える生成AI/RAGシステムを展開している。
  • ●ガバナンス枠組みMacauは、タスクを専門エージェントに分割し、実行前に検証エージェントが合成データでチェックし、ガードレールと監視を重ねる段階的なリスク管理を行う。

章立て

  1. Boseの経歴とMLXの役割

    Boseが自身の経歴(IBM Research、Google、JPMorgan、Facebook AI Research)とCapital OneのAI/MLプラットフォーム組織MLXでの役割を紹介する。

  2. 規制環境でAIを拡大する条件

    リスク管理とイノベーションの両立を課題とし、10〜15年前からのクラウド基盤、データ、人材、プラットフォーム思考が必要だと語る。

  3. プラットフォーム構成とクラウド賭け

    AWS上に再構築した基盤、オーケストレーション、デプロイ層の構成と、2016〜2017年にクラウドへ賭けた判断の背景を説明する。

  4. チーム統合と具体的な効果

    ML基盤チームの統合の重要性と、損失予測・不正検知がプラットフォームでどう高速化したかを紹介する。

  5. 2万エージェントの課題とMCP基盤

    個別導入では拡大できず、監視の一元化とMCPプラットフォーム整備が必要だと述べ、内製と外部利用の使い分けを語る。

  6. 本番稼働中のエージェント事例

    Chat Conciergeと、2万人超のコールセンター担当者向けRAGシステムの導入経緯を説明する。

  7. Macauによるガバナンス

    タスク分割、専門エージェント、検証エージェント、ガードレール、リアルタイム監視による段階的リスク管理を説明する。

  8. 導入企業への助言

    制御プレーンなど基盤設計を長期視点で決めること、複雑で価値の高いコーナーストーン事例から始めることを助言する。

解説記事

Eye on A.I. の今回の回では、Capital OneでAI/MLプラットフォーム部門「MLX」を率いるAbhijit Boseが、司会のCraig Smithに同社のAI拡大の舞台裏を語った。Boseは過去にFacebook AI Research、Google、JPMorgan Chase、IBM Researchなどに在籍した経歴を持つ。話の中心は、パイロットで止まる企業が多い中で、なぜ同社が本番規模のエージェント運用に近づけているのかという点だ。

土台は10年以上前のクラウド投資

Boseは、規制の厳しい環境でAIを拡大する際の本質は、リスク管理とイノベーションのバランスだと述べる。ガバナンスや意思決定の品質では妥協せず、同時にLLMの進化速度にも追随する必要がある。同社の問題設定は「AIの最前線で、しかしwell-managed(適切に管理された)速度で革新する」ことだという。

そのために、ガバナンスを適用する中央の仕組みを作り、一度LLMのホスティング方法を確立すれば1万のアプリケーションに繰り返し適用できる状態を目指している。Boseは、これは今日始めて間に合うものではなく、10〜15年前からのクラウド基盤、社内のエンジニアとサイエンティスト、データエコシステム、プラットフォーム思考の積み重ねだと強調する。

同社は自社の規模の金融機関としては珍しく、従来型データセンターを持たず、主にAWS上へ技術スタックを再構築したと説明する。2016〜2017年当時のクラウドの経済性は今ほど魅力的ではなく、数字だけ見れば見送る判断もあり得たが、ソフトウェア開発ライフサイクルを加速できるという認識が転機になったという。あわせて「プラットフォーム企業になる」と宣言し、全事業部門が使うべき少数の標準プラットフォームを経営層が定め、AI/MLもその一つになった。

プラットフォームの構成と効果

Boseによれば、基盤の最下層は大量のCPU・GPU・データをオンデマンドで確保する機能で、その上にエージェントやMLパイプラインを呼び出すオーケストレーション層、アプリやモデル、特徴量を作るオーサリング層、開発から本番へ手作業を挟まず移すデプロイ層がある。月間では4桁規模の社内ユーザーが利用し、そこで作られたアプリは1億人の顧客に影響するとのことだ。

具体例として、四半期ごとの引当金を決める損失予測を挙げた。数十のモデルをアンサンブルし、経済シナリオ別のシミュレーションも必要な難しいタスクで、従来は学習だけで数週間かかっていた。新基盤と分散並列計算により、全体で約1週間になったという。不正検知でも、更新頻度が日次、時間単位、数週間とモデルごとに異なるが、エンジニアを介さず素早く本番反映でき、新しい不正ルールの一部は生成AIで自動生成される段階にあると述べている。

組織面では、就任後1年以内に主要なML基盤チームを自分の組織の下に統合したことが決定的だったと振り返る。文化や納品速度、人材レベルが異なるチームの統合には最初の1〜2年の投資が必要で、マルチテナンシー(部門ごとのデータプライバシー要件を分離すること)などの設計水準も上がったという。

内製と外部の使い分け、MCP

同社は、オープンソース、外部ソリューション、自社開発を混在できる柔軟な制御プレーンを設計の核にしている。Boseは、誰でも再現できる領域では外部を使い、中核は独自データと独自モデルを使う方針だと述べた。例えばLlamaを一部用途で使うことはあっても、不正検知には必要なデータがないため自社モデルを使うという。Jupyter Notebookは採用しつつ、強いガバナンスと独自のMLOpsプロセスを重ねている。

MCP(Model Context Protocol、エージェントが外部ツールに接続する規格)についても、個別導入ではなく一つのプラットフォームとして扱う方針だ。MCPサーバーの登録・発見、構築用スタック、監視、どのAIエージェントがどのサーバーやツールにアクセスできるかを管理する層を整備し、社内で数百のMCPサーバーやサービスに拡張できると語った。

本番稼働中のエージェントとMacau

顧客向けの事例として、まずChat Conciergeがある。自動車ディーラーのサイトやアプリで24時間稼働し、在庫検索や予約を自然言語で支援する。複数のAIエージェントが連携し、人間と話しているように感じられる低遅延で動き、ディーラーごとの事業特性や在庫に合わせて調整されている。

もう一つは、2万人以上のコールセンター担当者向けの生成AIシステムだ。ナレッジベースの検索を支援するRAG(検索拡張生成)アプリケーションで、最初は少人数に提供してフィードバックを集め、検索手法を改善した結果、担当者のほうから参加を求めるようになり、全員に展開したという。なおここでの「2万」は人間のオペレーターで、AIエージェントの数ではないとBoseは明確にしている。

ガバナンスの枠組みがMacauだ。Boseは、昨年のGTCでJensen Huangが紹介したと述べている。複雑なタスクを小さく分け、専門のLLMを持つ専門エージェントが各々を解き、実行前に検証エージェントが合成データを使うシミュレーション的な確認を行う。さらにプラットフォーム側のガードレールでデータ漏洩などを検知し、リアルタイム監視でデータや運用の変化を見る、という段階的なリスク管理である。

導入企業への助言

Boseは、プラットフォームを始める企業にまず制御プレーンなど基盤部品の長期的な設計を考えるよう勧める。オープンソース、内製、外部のどれを選ぶかは数年後に大きく響く判断だという。次に、複雑で価値の高い「コーナーストーン」ユースケースから載せることだ。簡単な案件を多数集めれば利用者数は稼げるが、会社に本当の価値を出す基盤にはならない可能性がある、と述べている。

まとめ

この回の主張は明快だ。エージェント5つなら個別対応でも回せるが、2万規模ではオーケストレーション、監視の一元化、アクセス管理、ガバナンスを担うプラットフォームが前提になる。日本の企業にとっても、個別のPoCを積み上げるだけでなく、データ基盤、共通の制御・監視機構、組織統合といった地味な土台づくりが、エージェント活用を本番へ進める鍵になるという示唆が得られる。なお内容はBose個人の説明に基づくもので、数値や効果は同社の自己申告である点には留意したい。

文字起こし(英語・自動生成)

Capital One was very early among banks in the AI space. You know, we consider ourselves as a tech company that does banking instead of the other way around. We have, I would say, like one of the best rates of fraud prevention in the industry. 80-90% of that is because of the way we use AI to fight fraud. I talk to a lot of people about the penetration of production-grade agentic AI in enterprises, and there has been written over and over again. There are a lot of pilots, but very few of them make it into production. What sort of scale are you at now with agents? Why don't we start by having you introduce yourself and give us your career journey, how you got to Capital One and what you're doing at Capital One to scale AI. I lead the AI ML platform engineering org called MLX at Capital One.

So we not only build and deploy our AI and ML platforms, but we also build a lot of the major AI applications for servicing, for fraud, and for other applications as well. Especially in the Gen AI space, we are building a bunch of assistants and deploying a lot of coding agents of the company. I have been in this role for the past five years. Before Capital One, I used to lead the research engineering org at Facebook AI Research. And then before that, I worked at Google, JPMorgan Chase, and IBM Research. the connection to Westchester County is that I started my research career at TJ Watson Lab you know in Hawthorne and Yorktown Heights I spent roughly five years there and that was right after my PhD but pretty soon I decided I wanted to be an engineer building

things because I just love building and seeing the impact of you know my work on users and customer's lives. So that's how I became an engineer rather than staying on the research track. Here's a sentence that would have sounded pretty strange a few years ago. Your company probably uses AI agents to buy software and has a token budget to keep track of AI usage. That's one of the biggest findings from G2's latest buyer behavior report and it's changing the economics of software. 84% of companies have consolidated or cut at least three tools to fund AI tools. 70% of buyers want shorter contracts due to the pace of AI innovation. And vendors that offer deals based on outcomes or tokens are winning more favor. In other words, we may be watching the traditional SaaS model get rewritten in real time.

If you build, buy, or invest in software, G2's 2026 Buyer Behavior Report is worth a read. Check it out at g2.com slash BBR. What are the biggest prerequisites for scaling AI? I mean, you're a big bank with a lot of customers. So can you talk about scaling? So, you know, modern AI, we all realize that it's not just a promising technology, but it's a fundamentally different way of doing things. It's disruptive as well. but when it comes to taking you know LLMs and JAI technology scaling that within an enterprise and especially within a regulated environment like Capital One it's really about balancing risk

management with innovation so we have an incredibly high bar on governance you know risk management quality of our decisions, you know, operations. We are not going to compromise on any of these, but we also need to move faster in the age of, you know, LLMs and how fast the Gen AI itself is moving, you know, in the external community. So you can say our fundamental problem statement is that how can we innovate at the frontier of AI, but at the speed of well-managed and that's really you know a big focus of our team as we build platforms you know we are constantly looking at automating well-managed tasks how can we build central points of application of governance so that once you have figured out how to host an LLM you know it can

be repeated 10,000 times, you know, for 10,000 applications. But what I will tell you, Craig, is that it's not something that you start today. You know, you have to start like 10 years, 15 years back, you know, building the right cloud platform, building an in-house team of engineers, you know, and scientists. It's about building a really solid data ecosystem, you know, building a platform kind of a mindset in the company that, you know, a lot of companies build one of use cases that they will have a capability for AI in marketing, capability for AI in fraud, you know, but how do You connect all of these things together through central platforms. So we started doing that, you know, 10, 15 years ago as part of our cloud migration. So that's what is helping us to scale AI now.

I see. It's the centralized platform. That's an orchestration layer or a governance layer? Can you describe that? It's both. So at the core of our platform is the ability to bring in massive compute and data. You know, how do you provision that while still building on the cloud? And I think you know that Capital One is one of the probably the only financial institution of our size that's fully on the cloud. We don't have traditional data centers. So we had to rebuild all of our tech stack primarily on AWS. So the bottom most layer of our platform is basically being able to harness huge amounts of CPUs, GPUs, data on demand.

Then we have an orchestration layer where you can build agents. you can, you know, invoke agents, you can invoke machine learning pipelines. We have an authoring layer where you can build applications, you can build models, you know, build features. And then we have the deployment layer, you know, in runtime when you are ready to deploy that application from your development environment to runtime production setting. How do you do that without a lot of engineers in the middle doing manual kind of deployment. So we spent years studying this, you know, refining it. We built some earlier generation systems that did not scale as much. Then we landed, you know, what we call our AI platform today. And it's scaling. We have been able to scale to thousands of users within the company.

Really, a big four-digit number of users actually use our platform on a monthly basis. But the applications touch 100 million of our customers. They're the applications people build on our platform. So it's a very high responsibility to be able to do this at scale. Yeah, I am aware that you guys are an industry leader in the cloud and data transformation. What was the turning point that made you realize the company needed a fundamentally new platform approach to scale AI? Was there, you reach a certain size or data centers are too costly or not flexible enough? I mean, what was the sort of trigger? We saw the cloud coming. I think we were probably one of the very few companies like Netflix and, you know,

few companies like ours, Intuit probably, that really saw the cloud coming. and we took a bet on the cloud because the economics of cloud at that time, you know, back in 2016 2017 was not like what it is today If you had looked at the economics you would probably say no on is still great Maybe we experiment in the cloud. But I think the turning point was that realization that you can harness and accelerate our SDLC by moving to the cloud. So once we started doing that, we had to re-engineer our stack based on AWS services and really build some of these fundamental capabilities ourselves. But that was kind of the impetus. But part of that was also the realization that we have to be a platform company.

Along with the platform declaration, you know, we basically identified a set of a small number of, you know, platforms that would become the standard for the company. And they were declared, you know, by our most senior leadership that these are going to be the few platforms that every line of business, every user has to use. so AIML was part of that and since then we can talk about how that journey was for us for AIML but once all the ingredients were in place it's been really like an upward trajectory for us we are much faster to build and deploy models We can bring in a new decisioning model much faster than the industry.

We can do things that a lot of other, you know, financial institutions have not been able to do because of that focus on platform enablement. From a business perspective, which I understand isn't your focus, but what is the competitive advantage that Capital One sees with this cloud strategy? Much faster speed to market. You know, being able to provision, compute, and storage resources on demand. And compute is essential for that. like very large, you know, GPU clusters, being able to run that very efficiently. And, you know, with all the latest innovation in networking and storage and how to speed those computations up, that's an advantage we have because of so much investment in technology.

And, you know, we consider ourselves as a tech company that does banking instead of the other way around. And that's really what the value proposition of what we are doing to our business growth. What are the critical architectural ingredients, data unification, features, stores, governance, automation, et cetera, needed to support AI at enterprise scale? So over the years, we learned a lot. You know, I will tell you some of the most fundamental lessons we have learned in, you know, in like what are the most critical ingredients. First of all, you know, you if you look at the ecosystem, there are a lot of open source capabilities out there today for machine learning and AI. There are closed solutions from a lot of companies and startups that are available.

And then you also have the choice of building your own IP where it makes sense. So fundamental to our design choices is a very flexible control plane of our platform where we can take any one of these three options and mix them together for a particular offering or a vertical on the platform. I was going to ask, you know, you had separate teams in different silos previously. Is it difficult to bring all those teams under one umbrella? Yeah, that's a great point. You know, I think one of the reasons we have been able to do so much, you know, in the last three, four years is the fact that at the very early stage, like maybe within the first year I joined the company, when I was just studying the kind of the landscape of, you know, all these different platforms and trying to figure out the strategy of how we are going to bring all of these things together.

we actually brought the major machine learning platform teams under one roof in my organization. That one move was so critical, you know, looking back. If we let these teams continue along, you know, it would have been very hard for us to converge. So that's definitely one of the lessons we have learned. you know assimilating all these teams with different cultures different you know speed of delivery different even talent levels that was kind of hard to you know for the first one or two years we had to invest in invest a lot in talent and culture they brought this kind of a mindset of central platforms and you know raised the bar in our technical design and scalability, along with a lot of other fundamental aspects that you have to worry about,

like multi-tenancy. We have different groups with their own unique data privacy requirements, so you cannot always mix all of the data on the platform. So all of those were possible because of this consolidation of teams and infusing the teams with new talent. And can you give us a tangible example of how platformization unlocks AI capabilities that weren't possible before or accelerated development, model development, for example? Maybe I'll give you a couple of examples. One here from the classic machine learning, you know, because that still drives a lot of business value to every bank, you know, even though a lot of that, those modeling techniques, you know, are changing to AI and, you know, and doing more with LLMs, you know, transformer technology, et cetera, architecture.

But in classic machine learning, you know, there are a couple of applications like loss forecasting, you know, like how much loss I'm going to have every quarter, so I have to set aside provision for that. That problem is a very difficult machine learning task. It has, you know, often like tens and tens of different models that come together in an ensemble process. It takes weeks and weeks of training time. And not only that, you have to simulate different economic scenarios, different, you know, customer situations. So it's bottom up kind of modeling. So even if you have the base model built, then you have to do all kinds of simulations to really get to the answer. You know, had a very complex workflow to do that, you know, in traditionally with our new platform

and all of the user-level libraries and workflows that we have built and with a lot of distributed, you know, parallel computing, now that whole task takes maybe a week end-to-end. You know, that's really kind of like unlocking a lot of value. And the second example I'll give you in this space is fraud modeling. As you know, like, you know, the speed at which the fraud vectors move You have to move faster than that. So the faster you can update your fraud models in production, and we have varying stages of, you know, models like some are updated daily, some are updated, you know, in hours, some are updated every few weeks. that whole thing is now possible because of our platform that we can actually update our fraud rules and models much faster and put them into production without needing engineers in the middle

to deploying them. On the fraud modeling that's interesting how automated is that you have models that are looking at new fraud strategies, I presume, and when it identifies something a potential strategy is a report generated to a human We get alerts you know and also we are up to a point where a lot of those strategies are also starting to being auto-generated. So we are in a really good place with respect to fraud now. But as you know, this is a never-ending game. Every time you think you, you know, have nailed something, you know, some other vector comes up. But we are very bullish on using AI, you know, to fight fraud and automatically kind of uncovering new rules from the data itself

with the help of, you know, G&AI techniques, for example. And where do most large enterprises underestimate complexity of deploying and maintaining ML systems in production at scale? I think a lot of companies, you know, deploy use cases. And it might be okay to deploy one use case in fraud or one use case in marketing. You know, I see that in Gen AI especially. They will have like an agentic application in one part of the organization, another agent application, you know, in somewhere else, in some other function. And but not having a standard way to orchestrate this, these applications, not having the platforms makes it really hard to, you know, scale them. Like, you know, today we are seeing the kind of the ushering of the agentic era. You know, if you think of that, oh, I'm going to have five agents in the company in the next year, you might be able to do that without a platform.

But if you are envisioning a future where you will have 20,000 agents running in your company, you know, doing all kinds of things, that is a hard problem to solve. That's where centralization and platform mindset come into play. This is, I think, the core kind of the challenge a lot of companies will have going forward is that not having a platform approach. you know they sometimes like take the speed to deploy something and get the immediate value but underestimating the complexity of scaling it through the enterprise you know and deploying and maintaining it these systems in in production like a lot of observability coming to play here as well. A lot of monitoring and observability across the ecosystem has to be in place and you cannot send data everywhere for your monitoring systems. You know, ideally all the data should be

in one place and then you monitor end-to-end, not one place for the data, not another place for the models, you know, et cetera, or cloud systems, you know. So we have a central place for all monitoring data that, you know, and then we can build dashboards off of that. A couple of questions. You mentioned that the platform allows you to sort of mix and match off the shelf or customize commercial agents or models and then agents or models that are built in-house. How much do you use off-the-shelf stuff or third-party agents or models? I would say, you know, we are moving from using AI to really building custom AI and emphatically integrated stack ourselves because we see a lot of leverage in that.

Where we do use third party solutions, you know, would be things that can be replicated by, you know, everyone down the line. You know, but core models, definitely, you know, that is our proprietary data and proprietary models. So we don't use, you know, we might use like LAMA, you know, as open source model for certain use cases, but you cannot just use LAMA for your fraud modeling use case, for example. You know, it just doesn't have the data. we will use our own proprietary models for fraud. For example, like in our, you know, system where you actually authoring a machine learning model, we use Jupyter Notebooks, which is a very well known, you know, open source platform. So we have incorporated that, but we, you know, put in a lot of governance around that and our own MLFs process around that.

So we do take advantage of open source and third-party solutions as needed. But where we see the most leverage using our proprietary models and data, we'll definitely custom build our infrastructure around that. Are you using the MCP model context protocol or ATSA protocols to plug stuff together? So this has been the last, you know, six months exciting work that we are doing. You know, believe it or not, when the MCP first came into the scene, we started seeing some green shoots of, you know, people wanting to build some MCP applications. Then we started rolling out the agent decoding tools last year, you know, like Winserv, Cloud Code, Cursor, GitHub Copilot, you know, all these tools. And obviously, they are, you know, one of the value proposition is that you will be able to connect to Jira and GitHub and a lot of MCP servers, you know, to automate, further automate some of the software engineering tasks.

So then, you know, when they asked me to figure out how to actually deploy MCP at the company, guess what, what our approach was? You know, we wanted to approach it as another platform, which is basically an agentic protocol, you know, platform for us. So we are very close actually to deploying this MCP platform where all the MCP services would be registered, discovered. We have our own stack, you know, based on that you build your MCP server. and then, you know, these MCP servers will be well-governed, monitored. We have a separate, you know, access management layer, which agents, meaning like AI agents, will have access to which MCP servers, you know, what kind of tools and services would be, resources would be exposed through them. So that took us a little bit of time, but again, you know,

We are just starting to see the value of it. Now we can scale to hundreds of MCP servers and services across the company now. It's something I've wondered because I talk to a lot of people about the penetration of production-grade agentic AI in enterprises. And there has been written over and over again. There are a lot of pilots, but very few of them make it into production. You mentioned earlier potentially having thousands of agents working together. What sort of scale are you at now with agents, and what's the biggest challenge in integrating agents into a system like Capital One, which depends on accuracy? Yeah. So I'll give you two examples where we have deployed agents in a customer facing way.

And I think we are probably one of the very few companies, you know, especially in financial space, you know, having done so. One of them is the Chat Concierge, you know, which is basically an app that runs on dealer websites or, you know, mobile apps. where you as a prospective car buyer can interact with the dealership 24 by 7, and it will look like an assistant, you know, helping you to schedule, look up inventory, and it's all natural language. And it's been very, you know, well-received. It's being deployed at, you know, many, many dealerships across the company through our auto business, where we provide auto loans and run a lot of the dealer back-end infrastructure for auction and their inventory management platforms. That's a classic kind of way we approach kind of like agent TKI.

It actually runs on our Gen AI platform on the back where we have multiple AI agents that coordinate together to figure out the best response to the customer intent And then the whole thing needs to be in real time, like without a lot of latency so that it feels like you're almost interacting with a human. So that's already life. You can, you know, work with that chat concierge online today. The other application, large-scale application we have built and deployed is a system where when you call our call center, we have you know 20,000 plus agents who are responding to customer questions or you know particular matters that they want us to to dissolve so there are a lot of knowledge bases

they have to look up these agents so we built a system where the call center agents will be able to to get their answers, you know, through our Gen AI platform. And initially it was a few agents, you know, we started putting in front of them. They loved the system. We learned a lot. You know, we collected, started collecting feedback data, like how good were our answers. And then slowly we, you know, built in different search techniques. you know, improve the algorithm to the point now that agents were literally asking us to enroll them into the system. And now we have, you know, rolled it out to all 20,000 plus agents that we have in the company. And there and this is 24 by 7. This is a classic example of, you know, LLM backed, you know, RAG application.

but it's reaching 20,000 agents now within the company and the answers are, you know, being shown, I mean, voiced over, you know, by our agents to the customers. And when you say 20,000 agents, do you mean 20,000 instances of an agent deployed across web? No, this is the human agents, I mean, like the call center agents. There are 20,000 call center agents that are using this system. The chat concierge that I mentioned earlier, that's an AI agent. Many, many instances of that, you know, for dealerships that are fine-tuned to those dealerships, you know, particular nature of business, inventory, et cetera. And what kind of guardrails or governance layers are essential when agentic AI systems begin interacting with internal data workflows or even other agents?

This is something I've wondered about because it's essentially invisible to humans. I mean, it's all code and, you know, binary bits, you know, chasing around inside computer systems, but actually taking action. So how do you keep track of all that? How do you make sure that agents aren't sharing data that they shouldn't be sharing with other agents? So we have built a framework called Macau. You know, there is public mention of it. Actually, Jensen Wong showcased it as part of GTC last year, the Macau framework. It's probably one of a kind in, you know, particularly in regulated industries where from the get go in the designs, you know, in the design space, we try to break up a complex task into smaller specialized tasks.

And then there is a specialized agent with its own LLM and fine-tuning to really, you know, specialize solving that one particular task that it specially stopped. And, you know, so if you look at the user request coming in or a question or a conversation that's happening, in behind the scenes, our AI, you know, agentic framework breaks it up into much smaller set of tasks. And then we solve each task, you know, by a specialist agent. And then there is a validation agent that actually even before going or taking an action on behalf of the user, it runs a check. You can think of this almost as a simulation environment where we use synthetic data to figure out whether the task would be, you know, well managed or not, or there could be an error in the task.

So those are some of the steps we have taken, along with guardrails, you know, on the platform itself, looking for, you know, any kind of data leakage or things that the LLM should not be, you know, responding back. Kind of a, you can think of like a tiered way of managing risk. In the design of our agents itself, you know, we have built this framework of validation and breaking complex tasks into smaller specialized tasks. guardrails, you know, and then a lot of monitoring in place so that in real time we can start to see any shift based on data, operational data of the agency framework. If you could give one piece of advice to enterprises just starting their AI platform journey, what should they prioritize to avoid sort of long-term technical debt?

I think one thing they have to prioritize is to really, you know, think long term about the design, like the foundational blocks of the platform, like the control plane that I talked about. You know, it took us some time to figure that out. But once it was in, it's now, you know, giving us a lot of value. That's one place. like think of the foundational blocks of the platform, you know, whether they should be open source or in-house built or a third-party solution. But each one of those decisions can have a huge impact, you know, on how the future of the AI platform will become in a couple of years. So that's a very high-judgment decision. The second thing, you know, I would say is that it's always good to have a few cornerstone kind of like use cases.

The type of, you know, like I'll tell you one story from the early days of our journey here. from the get-go, me and my product partner, we decided to take on the most complex machine learning tasks to bring on the platform. So the most complex users, you know, most complex machine learning models and AI use cases. The other approach could be that, oh, we are building a platform. let's just try to corner the market by moving, by bringing in all the easy use cases. You know, things that may have less value, but we could actually have a lot more use cases and a lot more users on the platform. But then you may not build the right platform, you know, that will really deliver value for the company. So, you know, so these two things, like control plane design and kind of the foundational part of the platform, how that will look like.

and then what use cases you focus on, you know, focusing on complexity and value, I think are two good places to start. Yeah, that's fascinating. If people want to learn more about Capital One's technology and scaling AI, where would they go? They can go to our tech blog. It's CapitalOne.com slash tech. And I believe the AI section is also there. Okay. And presumably Capital One is growing because of this activity. Yeah. We are looking for a lot of engineers, you know, not just for the whole company, you know, even for AIML. absolutely if anybody is interested they should absolutely reach out to me

or my team okay

番組の概要欄(原文)

Most enterprises are still trying to figure out how to get a handful of AI agents into production. Capital One is running tens of thousands. Abhijit Bose, Senior VP and leader of the company's AI/ML platform engineering organization, joins Craig Smith to explain how a decade-old bet on the cloud - made when the economics didn't yet make obvious sense - became the foundation for everything Capital One is doing in AI today, from proprietary GPU clusters built in a single quarter to a platform that took model deployment from months down to about a week. The conversation gets concrete fast: a 24/7 AI concierge now running across car dealership websites, a generative AI system serving all 20,000-plus of Capital One's call center agents - adopted so successfully that agents started asking to be enrolled - and the Macau framework, Capital One's answer to the governance problem that's keeping most regulated industries from putting agentic AI into production. Bose's closing argument is the one that should worry every enterprise still in pilot mode: deploying five agents is manageable without a platform. Deploying 20,000 is a different problem entirely, and the companies that didn't start building the foundation years ago are going to find that out the hard way. Subscribe to Eye on A.I. for weekly conversations with the people building and deploying the future of AI.

出典・この記事について

番組
Eye On A.I.(Craig S. Smith)
元の回
Are You Planning 20,000 AI Agents? Capital One Has a Warning | Abhijit Bose
配信日
要約作成日
公式サイト
www.eye-on.ai

この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について

X でシェアSpotify で聴くApple Podcasts で聴く

関連エピソード