76% of Tool Calls in AI Agent Repos Lack Protective Checks

A study found that 76% of tool calls with real-world side effects in 16 open-source AI agent repositories had zero protective checks, such as input validation, rate limiting, or authentication. This poses a significant risk as AI agents can trigger unvalidated database writes, unchecked HTTP requests, or file deletions without confirmation.

Details

The researchers scanned 16 open-source AI agent repositories, including both agent frameworks and production applications with real business logic. They found that 76% of tool calls with potential real-world side effects, such as database writes, HTTP requests, or file deletions, had no input validation, rate limiting, authentication checks, or confirmation steps. This pattern held true not just in the generic framework code, but also in the production applications that build on top of these frameworks. The lack of guards means a single prompt injection or hallucination could trigger hundreds of unvalidated actions. While unguarded function calls exist in traditional codebases, the difference with AI agents is that the language model, not a human user, decides which functions to call and with what arguments, without understanding the business context and potential consequences.