Why Agent Systems Need a Control Plane

This article discusses the importance of a control plane architecture for agent systems that connect to external AI models and tools. It highlights the governance challenges that arise in production, such as provider failures, model hallucinations, and memory conflicts, and explains how a control plane can address these issues.

Details

The article argues that the typical agent architecture (user -> LLM -> tools -> response) looks clean on a whiteboard, but in production, the hard problems are not about making the LLM smarter, but about making the system controllable. It provides several examples of governance challenges, such as provider failures, model hallucinations, and memory conflicts. To address these issues, the article proposes a control plane architecture, borrowing concepts from networking and Kubernetes. The control plane manages the system's operation, handling concerns like provider routing, tool governance, request shaping, circuit breaking, observability, audit, and memory governance. The key insight is that governance must lead capability - building the control plane first before adding more capabilities is crucial to avoid uncontrolled blast radius. The article suggests that the pattern of building the control plane first, then adding capabilities inside it, is more effective than the other way around.