Building Openbench, a local-first LLM workspace

The article discusses the development of Openbench, a desktop application for experimenting with local language models. It aims to provide a workspace for prompt iteration, model comparison, and inspecting model outputs.

Details

Openbench is a new desktop application being developed to provide a more robust workspace for working with local language models. The author found that existing tools like Ollama make it easy to run models, but lack features for more advanced tasks like comparing models, refining system prompts, and inspecting model outputs. Openbench aims to address these gaps by offering a dedicated environment for prompt iteration, side-by-side model comparison, and the ability to view raw request and response data. The app is built using a modern tech stack including React, TypeScript, Rust, and SQLite for local data storage. While still in early development, the core architecture is taking shape, and the author is seeking feedback from the community on what features are most needed for working with local language models.