The Architecture of a Self-Hosted AI Gateway

This article explores the design decisions behind OpenClaw, an open-source AI agent gateway that connects chat platforms to AI models. It highlights key architectural choices such as single-instance deployment, embedded AI runtime, and a two-layer memory system.

Details

OpenClaw is an open-source AI agent gateway that allows connecting chat platforms like WhatsApp, Telegram, and Slack to AI models. The article highlights several non-obvious design decisions that reflect trade-offs faced when building AI infrastructure. One key constraint is that OpenClaw is designed for a single Gateway process per host, rather than horizontal scaling. This is because chat platform connections are stateful and tied to specific device pairings, so running multiple Gateway instances would cause issues like message duplication and state corruption. Another design choice is to embed the AI agent runtime directly within the Gateway process, rather than using a separate microservice architecture. This allows zero-latency communication, full control over the session lifecycle, and the ability to inject custom tools. However, it also means the Gateway and agent are tightly coupled, so a crash or memory leak in the agent affects the entire system. The agent's processing loop involves receiving input, assembling context from various sources, performing model inference, executing external tools, streaming the response, and persisting the conversation to disk. The context assembly step is particularly interesting, as it constructs a custom prompt for the model on every turn by pulling from workspace files, safety guardrails, and runtime information. OpenClaw's memory system has two layers: a 'working memory' Markdown file that is always included in the context, and 'daily memory' log files that are accessed on-demand. This distinction is designed to balance the token cost of including memory in every turn with the flexibility to search historical logs when needed.