Building a Cognitive AI Layer for Intelligent Routing and Autonomous Execution
The article describes the author's experience in building a cognitive AI layer called Akatskii, which routes inputs to the optimal LLM provider based on complexity and cost, executes tools autonomously in an iterative loop, and compresses memory to maintain context.
Why it matters
This approach to building a cognitive AI layer demonstrates how to create a high-performance, low-cost AI assistant that can handle a wide range of tasks intelligently.
Key Points
- 1Akatskii routes inputs using regex pattern matching before falling back to LLM classification
- 2It classifies inputs into five thought types, each mapped to a different LLM provider with varying cost and capabilities
- 3The system enters an iterative loop where the LLM calls tools, gets results, and decides when the task is complete
- 4This approach aims to achieve high-speed, low-cost AI assistance that can handle complex tasks autonomously
Details
The author built Akatskii, a cognitive AI layer, to address the limitations of existing AI frameworks. Akatskii can 'think at different speeds', using the appropriate LLM provider based on the complexity of the task. It uses regex pattern matching to route 80% of inputs instantly and for free, falling back to LLM classification only when necessary. Akatskii classifies inputs into five thought types - Reflex, Executive, Sensory, Creative, and Deep - and maps each to a different LLM provider, such as Groq Llama and Gemini. The key insight is that the more expensive Claude LLM is only invoked when the user explicitly requests it, keeping the overall cost close to zero. Akatskii also features an 'agentic loop' where the LLM reasons, calls tools, gets results, and decides when the task is complete, up to 15 iterations. This allows the system to handle complex, multi-step problems autonomously.
No comments yet
Be the first to comment