Building a Self-Healing Codebase with NATS, Claude AI, and A/B Code Tournaments
The author built a distributed agent system called ChangeBus that automatically detects, fixes, and merges code changes using AI-generated fix variants that compete in A/B tournaments.
Why it matters
This system demonstrates how AI can be used to automate the code repair and deployment process, improving developer productivity and codebase health.
Key Points
- 1ChangeBus uses NATS event bus, Claude AI, and A/B tournaments to automatically fix and merge code changes
- 2It generates two competing fix variants (minimal vs. robust refactor) and selects the winner based on test pass rate, diff size, and clean application
- 3The system learns from feedback loops, improving its fix quality over time
- 4Human escalation is used for low-confidence fixes or security-related changes
Details
ChangeBus is built on three key ideas: detecting real changes (not just noise), using A/B tournaments to generate and select the best fix, and establishing feedback loops to improve the system over time. The architecture includes components like NATS event bus, Python async code, Claude API for AI, SQLite for event storage, and Git operations. The core workflow involves generating two competing fix variants (minimal vs. robust refactor) that are tested independently, scored, and merged if the winner scores above a threshold. The system tracks outcomes to learn which strategies produce better fixes, injecting this knowledge back into the generation prompts. When the AI is not confident enough, the system escalates to human review via a task routing system. The author envisions extending this architecture to open-source projects and other use cases.
No comments yet
Be the first to comment