Monitoring MCP Tool Call Metrics in Real Time
This article introduces benchmark-broccoli, a transparent MCP proxy that measures tokens, estimated cost, and latency for every tool call made by an AI client. It provides detailed observability into AI infrastructure costs.
Why it matters
Monitoring MCP tool call metrics is crucial for understanding and optimizing the costs of running AI applications at scale.
Key Points
- 1benchmark-broccoli is a TypeScript proxy that sits between an AI client and MCP servers, logging metrics like token counts, cost estimates, and latency
- 2It automatically groups related tool calls into sessions, providing visibility into per-prompt efficiency and cost
- 3It supports any MCP server, including custom implementations, and allows comparing costs across multiple AI models
Details
The Model Context Protocol (MCP) is Anthropic's standard for connecting AI applications to data sources and tools. However, out of the box, developers have limited visibility into the costs associated with these tool calls. benchmark-broccoli solves this by intercepting the communication between the AI client and MCP servers, logging detailed metrics on each tool call. This includes token counts, estimated costs across 12+ AI models, and latency tracking. The proxy automatically groups related calls into sessions, providing a high-level view of per-prompt efficiency and cost. This allows developers to identify which tools and prompts are most expensive, and optimize their AI infrastructure accordingly.
No comments yet
Be the first to comment