Building a Pip-Installable RAG with Hybrid Search and Streaming

The article discusses the author's experience building a Retrieval Augmented Generation (RAG) pipeline using LangChain and the motivation behind creating a new library called 'ragwise' that provides hybrid search and streaming capabilities by default.

Details

The author explains that they spent a weekend integrating a RAG pipeline into a client project using LangChain, but ended up with a cumbersome setup and suboptimal retrieval quality. This was because the default was dense-only search, which misses exact keyword queries, error codes, and other specific information that is better captured by BM25 search. The article then discusses the two main types of RAG libraries available - those that are easy to start but weak on retrieval, and those that are strong on retrieval but require heavy infrastructure. The author saw a need for a pip-installable library that provides hybrid search as the default, which led to the creation of 'ragwise'. Ragwise uses a combination of BM25 and dense retrieval, with Reciprocal Rank Fusion to merge the ranked results. This approach is simple, robust, and does not require any training data. The article also highlights Ragwise's streaming capabilities, which enable real-time text generation in user interfaces.