The First Karpathy Loop for Production Coding Agents

This article introduces Backbeat, a tool that enables AI coding agents to autonomously iterate and optimize code, similar to Karpathy's autoresearch loop for AI research.

Details

The article discusses the challenges of working with AI coding agents, where the typical workflow of running the agent, checking the output, and then running it again with different instructions breaks down when trying to achieve more complex, iterative tasks. The author introduces Backbeat, a tool that implements a 'Karpathy loop' for production coding, allowing AI agents to autonomously iterate and optimize their work. Backbeat provides two main loop strategies: 'Retry' to run a task until a shell command returns a successful exit code, and 'Optimize' to score each iteration using an evaluation script and keep the best result. The article explains how Backbeat runs each iteration in a clean agent context, preventing baggage from previous failures, and how it can be used to loop entire pipelines with multiple steps. The scoring function is highlighted as the key to making the loop work, as it provides a way for the agent to evaluate and improve its own output. The article positions Backbeat as the first production implementation of this pattern for coding agents, with support for tools like Claude Code, Codex, and Gemini CLI.