ONNX Runtime + pgvector in Django: Semantic Search Without PyTorch or External APIs

This article discusses a solution for implementing semantic search in a Django-based web application without relying on external APIs or PyTorch dependencies. It leverages ONNX Runtime for efficient model inference and pgvector for vector storage.

Details

The article presents a solution for implementing semantic search in a Django-based web application without relying on external APIs or PyTorch dependencies. The author explains the limitations of the common approach of calling an external embedding API, which introduces costs, latency, and privacy concerns, especially for small-to-medium applications. To address these issues, the article introduces the use of ONNX (Open Neural Network Exchange) and ONNX Runtime. ONNX is a serialization format that allows running pre-trained models without the original framework, reducing deployment complexity. ONNX Runtime is a lean inference engine that can execute ONNX models on CPU or GPU, providing a more efficient alternative to PyTorch. The article showcases the use of the paraphrase-multilingual-MiniLM-L12-v2 model, exported to ONNX format, for semantic search. Additionally, the article discusses the use of pgvector for vector storage, eliminating the need for a separate vector database.