The Task Entropy Framework: Choosing Between Fast and Smart AI Models
This article introduces the 'task entropy' framework to help decide when to use fast or smart AI models based on the predictability and complexity of the task at hand.
Why it matters
This framework provides a systematic way to choose between fast and smart AI models based on the characteristics of the task, which can improve the efficiency and effectiveness of AI-powered workflows.
Key Points
- 1Low entropy tasks (e.g., boilerplate, refactoring) have a bounded outcome space and well-defined paths, so speed is more important.
- 2High entropy tasks (e.g., architecture decisions, debugging) have a vast outcome space and context-dependent 'right' answers, so smarts are more important.
- 3The 'routing agent' pattern can assess task entropy and route to the appropriate model.
Details
The author has developed a framework called 'task entropy' to help choose between fast and smart AI models for different types of tasks. Low entropy tasks have a predictable outcome space and well-defined paths, so speed is more important. High entropy tasks have a vast outcome space and context-dependent 'right' answers, so smarts are more important. The key signals to consider are reversibility (how easy is it to undo a mistake?) and blast radius (how many files/systems are affected). The ideal setup is to have a 'routing agent' that can assess the task entropy and route to the appropriate model - fast for low entropy, smart for high entropy.
No comments yet
Be the first to comment